Feed: Slashdot
Feed Link: https://slashdot.org/
---

Title: How Python is Fighting Open Source's 'Phantom' Dependencies Problem

Link: https://developers.slashdot.org/story/25/08/1...

Since 2023 the Python Software Foundation has had a Security Developer-in-
Residence (sponsored by the Open Source Security Foundation's vulnerability-
finding "Alpha-Omega" project). And he's just published a new 11-page white
paper about open source's "phantom dependencies" problem - suggesting a way
to solve it. "Phantom" dependencies aren't tracked with packaging metadata,
manifests, or lock files, which makes them "not discoverable" by tools like
vulnerability scanners or compliance and policy tools. So Python security
developer-in-residence Seth Larson authored a recently-accepted Python
Enhancement Proposal offering an easy way for packages to provide metadata
through Software Bill-of-Materials (SBOMs). From the whitepaper: Python
Enhancement Proposal 770 is backwards compatible and can be enabled by
default by tools, meaning most projects won't need to manually opt in to
begin generating valid PEP 770 SBOM metadata. Python is not the only software
package ecosystem affected by the "Phantom Dependency" problem. The approach
using SBOMs for metadata can be remixed and adopted by other packaging
ecosystems looking to record ecosystem-agnostic software metadata... Within
Endor Labs' [2023 dependencies] report, Python is named as one of the most
affected packaging ecosystems by the "Phantom Dependency" problem. There are
multiple reasons that Python is particularly affected: - There are many
methods for interfacing Python with non-Python software, such as through the
C-API or FFI. Python can "wrap" and expose an easy-to-use Python API for
software written in other languages like C, C++, Rust, Fortran, Web Assembly,
and more. - Python is the premier language for scientific computing and
artificial intelligence, meaning many high-performance libraries written in
system languages need to be accessed from Python code. - Finally, Python
packages have a distribution type called a "wheel", which is essentially a
zip file that is "installed" by being unzipped into a directory, meaning
there is no compilation step allowed during installation. This is great for
being able to inspect a package before installation, but it means that all
compiled languages need to be pre-compiled into binaries before
installation... When designing a new package metadata standard, one of the
top concerns is reducing the amount of effort required from the mostly
volunteer maintainers of packaging tools and the thousands of projects being
published to the Python Package Index... By defining PEP 770 SBOM metadata as
using a directory of files, rather than a new metadata field, we were able to
side-step all the implementation pain... We'll be working to submit issues on
popular open source SBOM and vulnerability scanning tools, and gradually,
Phantom Dependencies will become less of an issue for the Python package
ecosystem. The white paper "details the approach, challenges, and insights
into the creation and acceptance of PEP 770 and adopting Software Bill-of-
Materials (SBOMs) to improve the measurability of Python packages," explains
an announcement from the Python Software Foundation. And the white paper ends
with a helpful note. "Having spoken to other open source packaging ecosystem
maintainers, we have come to learn that other ecosystems have similar issues
with Phantom Dependencies. We welcome other packaging ecosystems to adopt
Python's approach with PEP 770 and are willing to provide guidance on the
implementation."

Read more of this story at Slashdot.

---
VRSS v2.1.180528