Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PSF engagement report for Oct 2023 #265

Merged
merged 1 commit into from
Oct 28, 2023
Merged

Add PSF engagement report for Oct 2023 #265

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions alpha/engagements/2023/psf/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This engagement started in November 2022.

### Monthly Updates

* [October 2023](update-2023-10.md)
* [September 2023](update-2023-09.md)
* [August 2023](update-2023-08.md)
* [July 2023](update-2023-07.md)
Expand Down
80 changes: 80 additions & 0 deletions alpha/engagements/2023/psf/update-2023-10.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Update 2023-10

## Highlight: Security Developer-in-Residence 2023 Q3 Report

Wrote and published the report for Q3 in the Security Developer-in-Residence role.
Included all the highlighted accomplishments in the role like the PSF becoming a
CNA, advisories in an OSV database, improvements to the Python Security Response Team,
and more.

In this post I also detail the three areas that I want to focus on for the next quarter which are:

* Securing the CPython release process
* Metadata for bundled projects in Python packages
* SBOM for CPython release artifacts

[Read the full report on the PSF blog](https://pyfound.blogspot.com/2023/10/security-developer-in-residence-2023-q3-report.html).

## Highlight: CPython Release Process

I've made progress on improving the CPython release process. Starting with automating the process in GitHub Actions
and achieving a content-wise identical tarball to what was published for CPython v3.12.0. Using
tooling for reproducible builds like `diffoscope` and `reprotest` was able to remove all sources of
non-reproducibility from the CPython source builds and got the changes [merged into the CPython release process](https://github.com/python/release-tools/pull/62).
Verified that reproducibility had worked as expected for CPython 3.13.0a1.

Created [roadmap for future improvements to the CPython release process](https://github.com/python/release-tools/issues/66)
which include moving the source, docs, and macOS installer builds to an isolated platform (in this case, GitHub Actions).
From there being able to make at least the macOS installer and Windows installer builds reproducible would also be desirable.

The work was discussed on the [CPython core developer podcast "core.py"](https://podcasters.spotify.com/pod/show/corepy/episodes/Episode-1---Core-Sprint-in-Brno--Python-3-13-0-alpha-1-e2apebk)
by Łukasz Langa and Pablo Galindo Salgado.

## Highlight: Patching the libwebp vulnerability across the Python ecosystem

I set out to patch the libwebp vulnerability across the ecosystem. Python
packages tend to bundle third-party shared libraries in wheels. I documented
all the problems I ran into through this process, including the need for
metadata around bundled projects in Python packages.

[Full blog post documenting the experience](https://sethmlarson.dev/security-developer-in-residence-weekly-report-16).

## Highlight: Tracking how Python subcomponents change using SBOMs

Even though the Python APIs haven't changed, there's been lots of movement below the surface.
I created Software Bill-of-Materials (SBOMs) to track the subcomponents of CPython and to see how
this changes between releases.

I created an [SBOM for Python 3.12.0](https://github.com/sethmlarson/cpython-sbom/blob/main/sboms/Python-3.12.0.tgz.spdx.json) and then compared the components against the ones included in Python 3.11.6.
Comparing the two SBOM documents revealed the differences between the two release streams, including:

* [Python 3.12 removes `setuptools`](https://github.com/python/cpython/pull/101039) from the `ensurepip` module which was previously needed to bootstrap pip and venv in a Python environment.
Now in 3.12.0 there's only a bundled copy of `pip` which includes many other bundled dependencies like Requests and certifi.
These packages [still need to be captured in the CPython SBOM](https://github.com/sethmlarson/cpython-sbom/issues/10).
* [Python 3.12 replaces the implementation of hashlib algorithms using OpenSSL and tiny\_sha3](https://github.com/python/cpython/issues/99108) with a [new backend called HACL\*](https://github.com/hacl-star/hacl-star) (pronounced "H-A-C-L star")
which means the tiny_sha3 project can be removed. This new backend is formally verified using F\* to provide memory safety
and avoid many common security vulnerabilities.
* [Python 3.12 removes the small libffi stub for OSX](https://github.com/python/cpython/issues/100540) which was unused
after [adding support for macOS 11 and Apple Silicon](https://github.com/python/cpython/issues/72677).

[Read the full section in a blog post, which includes a visualization](https://sethmlarson.dev/security-developer-in-residence-weekly-report-13#tracking-how-python-subcomponents-change-using-sboms).

Next steps for the SBOM work will be to add it to the CPython release process and to upstream in a way that is maintainable by core developers.

## Other items

* Authored weekly reports and shared on social media:
* [Python 3.12.0 from a supply chain perspective](https://sethmlarson.dev/security-developer-in-residence-weekly-report-13) (HN frontpage, 8,000 analytics views)
* [Reproducible builds for CPython source tarballs](https://sethmlarson.dev/security-developer-in-residence-weekly-report-14)
* [Quarterly report for Q3 2023 on the PSF blog](https://sethmlarson.dev/security-developer-in-residence-weekly-report-15)
* [Patching the libwebp vulnerability across the Python ecosystem](https://sethmlarson.dev/security-developer-in-residence-weekly-report-16)
* The Python Software Foundation CVE Numbering Authority (CNA) [published its first CVE](https://www.cve.org/CVERecord?id=CVE-2023-5752), a medium severity vulnerability affecting pip when installing from a Mercurial repository.
* pip v23.3 released with the following relevant changes:
* Truststore was vendored which means you no longer need to bootstrap Truststore in order to use [pip's optional Truststore support](https://pip.pypa.io/en/stable/topics/https-certificates/#using-system-certificate-stores).
* Upgraded the vendored certifi to not be vulnerable to [GHSA-xqr8-7jwr-rhp7](https://github.com/advisories/GHSA-xqr8-7jwr-rhp7).
* Presented to the [Stockholm Python User Group](https://www.meetup.com/pysthlm/events/296576048/).
* Working with the OpenSSF SBOM Everywhere WG on finalizing the "[Best Practices for Naming and Directory conventions for SBOMs](https://docs.google.com/document/d/1-jFoh_R7FV4NhHuUkt4Atz3h4K9b4bnmolntSbytspE)" document.
* OpenSSF Day Europe recordings have been uploaded to YouTube. Here are two notable talks for Python:
* [We make Python safer than ever](https://www.youtube.com/watch?v=jhzv5RU56V4) by Cheuk Ting Ho (OpenSSF) and Seth Larson (PSF)
* [Trusted Publishing: Lessons from PyPI](https://www.youtube.com/watch?v=Cc7hl_tyKWE) by William Woodruff
* [Added redirect to readers of python-security.readthedocs.io](https://github.com/vstinner/python-security/pull/42) to the PSF Advisory Database.