Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OpenSSL versions for CI and Windows #131423

Open
4 tasks done
picnixz opened this issue Mar 18, 2025 · 18 comments
Open
4 tasks done

Update OpenSSL versions for CI and Windows #131423

picnixz opened this issue Mar 18, 2025 · 18 comments
Assignees
Labels
build The build process and cross-build dependencies Pull requests that update a dependency file extension-modules C modules in the Modules dir infra CI, GitHub Actions, buildbots, Dependabot, etc. topic-SSL type-security A security issue

Comments

@picnixz
Copy link
Member

picnixz commented Mar 18, 2025

OpenSSL v3.4.1 is out and contains some security patches (see https://github.com/openssl/openssl/releases/tag/openssl-3.4.1). There is one high vulnerabilty (CVE-2024-12797) that was fixed.

However, what I'm interested in, are the fixes that allow me to continue working on #128391 (see openssl/openssl#26388). Note that this high vulnerability does not affect the Windows build as the latter is still using OpenSSL 3.0.15 which is only affected by the following low vulnerabilities:

Those low vulnerabilities affect OpenSSL 1.1.1+ and 3.x versions that we currently use and were fixed in the February 2025 release.

Note: I don't think Python is directly affected by the low vulnerabilies and I just want the fixes that were included in those releases for my own work. Since the high vulnerability only affects 3.2+, Windows builds should not be affected.

cc @gpshead

Plan:

Linked PRs

@picnixz picnixz added build The build process and cross-build dependencies Pull requests that update a dependency file infra CI, GitHub Actions, buildbots, Dependabot, etc. topic-SSL type-security A security issue labels Mar 18, 2025
@picnixz picnixz self-assigned this Mar 18, 2025
@zooba
Copy link
Member

zooba commented Mar 18, 2025

Any reason we shouldn't move the Windows release forward to a newer version than 3.0 (for 3.14, that is - prior releases presumably stay where they are)?

@picnixz
Copy link
Member Author

picnixz commented Mar 18, 2025

Any reason we shouldn't move the Windows release forward to a newer version than 3.0 (for 3.14, that is - prior releases presumably stay where they are)?

I don't know. I guess we don't need the features added in 3.1. However, from a performance perspective, OpenSSL 3.0 seems to be way much slower compared to OpenSSL 3.1 on Windows builds: https://openssl-library.org/performance.html. However, OpenSSL 3.0 is the "LTS" version of OpenSSL so for Windows it may be better (see https://openssl-library.org/policies/releasestrat/index.html).

Now, in 2026, 3.0 would no more be LTS, so we might jump to the next LTS at that moment. WDYT? Otherwise, if there's no more LTS for OpenSSL, we might need to update Python much more frequently once a version becomes out-of-date (non-LTS releases are supported for 2 years, so I don't know how you want to proceed)

@zooba
Copy link
Member

zooba commented Mar 18, 2025

Hmm... yeah... let's stick to their LTS releases then. Presumably they'll have to nominate another LTS release before the current one expires, though it's a bit unfortunate we can't count on any to be supported for the same time as our own release.

@zooba
Copy link
Member

zooba commented Mar 18, 2025

The openssl-bin-3.0.16 tag is now available for integrating into the Windows build.

@picnixz
Copy link
Member Author

picnixz commented Mar 18, 2025

The openssl-bin-3.0.16 tag is now available for integrating into the Windows build.

Thanks! I'm sorry but I won't be able to update this until Sunday as I'm going offline for a few days.

@picnixz picnixz added the extension-modules C modules in the Modules dir label Mar 18, 2025
@hugovk
Copy link
Member

hugovk commented Mar 22, 2025

https://docs.python.org/3/using/configure.html says:

  • OpenSSL 1.1.1 is the minimum version and OpenSSL 3.0.9 is the recommended minimum version for the ssl and hashlib extension modules.

Shall we bump that 3.0.9 to 3.0.16?

non-LTS releases are supported for 2 years, so I don't know how you want to proceed

3.1 just went EOL on 2025-03-14, and 1.1.1 has been EOL since 2023-09-11.

Presumably they'll have to nominate another LTS release before the current one expires, though it's a bit unfortunate we can't count on any to be supported for the same time as our own release.

Yes, 3.5 (due for release next month, 2025-04-08) will be the next LTS:

https://openssl-library.org/post/2025-02-20-openssl-3.5-lts/

And:

In addition, the OpenSSL Corporation and Foundation have agreed to designate an LTS every two years. That means there will be an LTS release in April of 2027, another in 2029, and so on. As always, each LTS will be supported for 5 years with the final year’s support being security patches only.
...
Predictable LTS releases should help projects that depend on them plan their own development schedules. There will always be at least two supported LTS releases of OpenSSL in addition to predictably timed regular releases.

What is our policy around dropping EOL OpenSSL? Given there will always be two LTS, can we go as far as only supporting those?

@picnixz
Copy link
Member Author

picnixz commented Mar 22, 2025

1.1.1 has been EOL but I don't know about 1.1.1z for instance. It's still receiving updates I think and our build requirements are 1.1.1 at least. I'm not at home yet so I can't check it precisely but I'll investigate this evening of tomorrow

@hugovk
Copy link
Member

hugovk commented Mar 22, 2025

@picnixz
Copy link
Member Author

picnixz commented Mar 22, 2025

I think we can update the requirements for macOS and Windows as they are already out of date.

Concerning the minimal version and the policy for dropping EOL versions (which is partially related):

While OpenSSL 1.1.1w was released 2y ago some other 1.1.1X were released 6 months ago (like 1.1.1p). I think there are some backports being still done to 1.1.1 depending on the severity of the CVE (just a guess) and whether something applies to it or not.

So I don't know about whether we can drop 1.1.1 and make 3.x the minimum requirement. Note that my openSUSE openssl version is 1.1.1 and while I'm a bit out-of-date, I'm worried that there are still too many OSes with default openssl < 3.x. Also, I think we may already have if-guards in place to detect the disparities between 1.1.1 and 3.x API so I think we can keep them for now (I don't remember well but I don't think they are really disruptive on our side).

I don't know why my phone told my that the 1.1.1l release was actually 6 months ago. I can't find it anymore on the web UI so I guess it was a GH bug.

@gpshead
Copy link
Member

gpshead commented Mar 22, 2025

  • 1.1.1 is obsolete & EOL, no release builds from us would ever touch it. Thus it is irrelevant on macOS and Windows.

  • openssl 3.0 LTS is our official minimum requirement for 3.13 and 3.14. We should be able to ship 3.14 releases using the upcoming openssl 3.5 LTS. We always need to backport the ability to build using the latest openssl LTS to our past releases and security branches. That can be painful but, isn't a choice we get. (and really increases our wishing we didn't have an ssl module in the stdlib at all)

@gpshead
Copy link
Member

gpshead commented Mar 22, 2025

What about building against 1.1.1? (not relevant to this issue - but seems to be part of the conversation)

  • Merely attempt to "try" to retain the ability for our code to be compiled against some 1.1.1 fork derivative versions, but don't go out of our way to do so when it gets in the way of simpler code and CI setups. i.e. don't spend a lot of time attempting to keep the thing you rightly don't have an easy ability to test, and isn't in CI, working.

Why not rip 1.1.1-ish support out entirely? Things like AWS-LC and BoringSSL are 1.1.1-API-ish with a mix of additions/removals and do get used around the world in many critical places with all sorts of software, including CPython.

https://discuss.python.org/t/support-building-ssl-and-hashlib-modules-against-aws-lc/44505

Those 1.1.1 forks are not currently part of a tier supported platform to build against. They're best effort at most and I do not believe we build out of the box against those correctly today. People using them still maintain patches against CPython to do so. If users of those want more from us, they need to step up and provide support ala PEP-11 tiers to be officially tested as a tier'd platform.

This mostly winds up being "don't RIP out existing conditional compilation bits for aws-lc/boringssl" if you can help it. It isn't used by us, but others use it and occasionally contribute work to minimize the needs of their own CPython patches. ex: #131050 is a reasonable request along those lines for AWS-LC/BoringSSL purposes.

@picnixz
Copy link
Member Author

picnixz commented Mar 23, 2025

What about building against 1.1.1? (not relevant to this issue - but seems to be part of the conversation)

It's currently working in the CI. What the CI failed to do was related to incorrect mnemonics which were added in 3.x. So I actually didn't touch anything related to 1.1.1.

1.1.1 is obsolete & EOL, no release builds from us would ever touch it. Thus it is irrelevant on macOS and Windows.

That I agree.

openssl 3.0 LTS is our official minimum requirement for 3.13 and 3.14. We should be able to ship 3.14 releases using the upcoming openssl 3.5 LTS.

I would like to point that https://docs.python.org/3/using/configure.html still mentions that "OpenSSL 1.1.1 is the minimum version" and that "OpenSSL 3.0.9 is the recommended minimum version for the ssl and hashlib extension modules" (emphasis mine).

Why not rip 1.1.1-ish support out entirely?

I would have initially answered that it's because of some OSes, but I think I should upgrade my distribution as

openssl version
OpenSSL 1.1.1l-fips  24 Aug 2021 SUSE release 150500.17.37.1

So, if we drop 1.1.1, it could annoy some people like me who forgot to upgrade the distribution (or are late in doing it), but otherwise, I think it's probably a good idea to simplify the code (this should be tracked and correctly investigated in a separate issue though).

Now, I'm sorry to have said that openssl 1.1.1* still received updates but on my GH mobile app, the 1.1.1p release was marked as "6 months ago" which I found it weird. It might be a cache issue on my side but I can confirm that it's not updated at all :')

@zooba
Copy link
Member

zooba commented Mar 24, 2025

People using [1.1.1 backports] still maintain patches against CPython to do so.

Honestly, it would probably be easier for them to fork the _ssl module entirely and build it separately. No reason it won't just drop in over the stdlib version.

@zooba zooba assigned zooba and unassigned picnixz Mar 28, 2025
@zooba
Copy link
Member

zooba commented Mar 28, 2025

@picnixz Stealing this from you, since I'm making the change anyway as part of handling #131804

zooba added a commit to zooba/cpython that referenced this issue Mar 28, 2025
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix pythongh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
@picnixz
Copy link
Member Author

picnixz commented Mar 28, 2025

How about I give you the Windows part but I keep the Linux part as the mnemonics were updated in 3.4.1?

@zooba
Copy link
Member

zooba commented Mar 28, 2025

Okay yeah, you can keep that PR and I'll stick with the easy one 😆

@picnixz
Copy link
Member Author

picnixz commented Mar 28, 2025

Deal!

zooba added a commit that referenced this issue Mar 28, 2025
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix gh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
zooba added a commit to zooba/cpython that referenced this issue Mar 28, 2025
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix pythongh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
zooba added a commit to zooba/cpython that referenced this issue Mar 28, 2025
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix pythongh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
zooba added a commit that referenced this issue Mar 28, 2025
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix gh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
zooba added a commit to zooba/cpython that referenced this issue Mar 28, 2025
zooba added a commit that referenced this issue Mar 28, 2025
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix gh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
@zooba
Copy link
Member

zooba commented Apr 3, 2025

I have another update for Windows on this to use a build made with an older compiler version - one that doesn't have the compiler bug that caused problems with 3.0.16.1 and 3.0.16 (note that the fourth digit is ours, the OpenSSL sources are the same each time).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build The build process and cross-build dependencies Pull requests that update a dependency file extension-modules C modules in the Modules dir infra CI, GitHub Actions, buildbots, Dependabot, etc. topic-SSL type-security A security issue
Projects
None yet
Development

No branches or pull requests

4 participants