Update Python to 3.13.0 and add support to Windows on AMD64. #1477
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,15 +10,19 @@ | |
| We only supply binaries for windows and macOS, but we do it very different ways for those two OSes. | ||
|
There was a problem hiding this comment. Perhaps add another sentence here: "On linux, we simply depend on the system version of python". |
||
|
|
||
| Windows recipe: | ||
| 1. Download precompiled version of python from NuGet package manager, | ||
| either the package "python" for AMD64, or "pythonarm64" for ARM64. | ||
| 2. Set up pip and install pywin32 and psutil via pip for emrun to work. | ||
| 3. Re-zip and upload to storage.google.com | ||
|
|
||
| macOS recipe: | ||
| 1. Clone cpython | ||
| 2. Use homebrew to install and configure openssl (for static linking!) | ||
| 3. Build cpython from source and use `make install` to create archive. | ||
|
|
||
| Raspberry Pi Debian 12 (Bookworm): | ||
| 1. Before calling this script, run "sudo apt install libssl-dev", or otherwise | ||
| Python won't be able to use SSL. | ||
| """ | ||
|
|
||
| import glob | ||
|
|
@@ -32,29 +36,35 @@ | |
| from subprocess import check_call | ||
| from zip import unzip_cmd, zip_cmd | ||
|
|
||
| version = '3.13.0' | ||
|
There was a problem hiding this comment. Do we need to upgrade to such a recent version? i.e. what as the first version of which arm64 packages are available. The slight downside of using a bleeding edge version of python like this is that we accidentally start depending of very new python features that folks on older/stable versions might not have. There was a problem hiding this comment. I was pondering about this too. My thinking here is that since we do these updates so rarely, maybe we would for once update to a modern version. Python 3.9, 3.10 and 3.11 will no longer receive bugfixes to any issues one might run into. I don't think it makes sense to update to any of these versions, even if one of these would be the earliest ones to support Windows on ARM64. Python 3.14 is in feature development. So we definitely will not want to update to that branch. Python 3.12 and Python 3.13 have finished feature development and are actively receiving bufixes. It feels that either of these would be the ones to focus on. I.e.
I picked 3.13 since it is newer, and I could not find any mentions of OS support cutoff differences between 3.12 and 3.13.
3.13 is stable. (or rather, actually, every python version is broken in different ways. But Emscripten test suite is fortunately evolved to be good at finding ways that Python works or doesn't work out) We do bundle python for the very purpose to normalize which version we provide to people? So this covers Windows and macOS users. On Linuxes where this currently is not the case, I understand updating to a newer version should be possible easier with their package managers? There was a problem hiding this comment. I suppose the other alternative could be 3.12.7 . At the end of the day I don't feel that strongly, but generally would strive to standardize to latest version, especially if Emscripten test suite will pass with 3.13.0. (Testing that out on Windows, although I am seeing Windows failures already before this update, making it a bit more time consuming to compare) |
||
| major_minor_version = '.'.join(version.split('.')[:2]) # e.g. '3.9.2' -> '3.9' | ||
| # This is not part of official Python version, but a repackaging number appended by emsdk | ||
| # when a version of Python needs to be redownloaded. | ||
| revision = '0' | ||
|
|
||
| PSUTIL = 'psutil==6.0.0' | ||
|
|
||
| upload_base = 'gs://webassembly/emscripten-releases-builds/deps/' | ||
|
|
||
|
|
||
| # Detects whether current python interpreter architecture is ARM64 or AMD64 | ||
| # If running AMD64 python on an ARM64 Windows, this still intentionally returns AMD64 | ||
| def find_python_arch(): | ||
| import sysconfig | ||
| arch = sysconfig.get_platform().lower() | ||
| if 'amd64' in arch: | ||
| return 'amd64' | ||
| if 'arm64' in arch: | ||
| return 'arm64' | ||
| raise f'Unknown Python sysconfig platform "{arch}" (neither AMD64 or ARM64)' | ||
|
|
||
|
|
||
| def make_python_patch(): | ||
| python_arch = find_python_arch() | ||
| package_name = 'pythonarm64' if python_arch == 'arm64' else 'python' | ||
| download_url = f'https://www.nuget.org/api/v2/package/{package_name}/{version}' | ||
| filename = f'python-{version}-win-{python_arch}.zip' | ||
| out_filename = f'python-{version}-{revision}-win-{python_arch}.zip' | ||
|
|
||
| if not os.path.exists(filename): | ||
| print(f'Downloading python: {download_url} to {filename}') | ||
|
|
@@ -64,19 +74,17 @@ def make_python_patch(): | |
| check_call(unzip_cmd() + [os.path.abspath(filename)], cwd='python-nuget') | ||
| os.remove(filename) | ||
|
|
||
| src_dir = os.path.join('python-nuget', 'tools') | ||
| python_exe = os.path.join(src_dir, 'python.exe') | ||
| check_call([python_exe, '-m', 'ensurepip', '--upgrade']) | ||
| check_call([python_exe, '-m', 'pip', 'install', 'pywin32==308']) | ||
| check_call([python_exe, '-m', 'pip', 'install', PSUTIL]) | ||
|
|
||
| check_call(zip_cmd() + [os.path.join('..', '..', out_filename), '.'], cwd=src_dir) | ||
| print('Created: %s' % out_filename) | ||
|
|
||
| # cleanup if everything went fine | ||
| shutil.rmtree('python-nuget') | ||
|
|
||
| if '--upload' in sys.argv: | ||
| upload_url = upload_base + out_filename | ||
|
|
@@ -94,7 +102,7 @@ def build_python(): | |
| check_call(['brew', 'install', 'openssl', 'xz', 'pkg-config']) | ||
| if platform.machine() == 'x86_64': | ||
| prefix = '/usr/local' | ||
| min_macos_version = '10.13' | ||
| elif platform.machine() == 'arm64': | ||
| prefix = '/opt/homebrew' | ||
| min_macos_version = '11.0' | ||
|
|
@@ -149,7 +157,7 @@ def build_python(): | |
|
|
||
| # Install psutil module. This is needed by emrun to track when browser | ||
| # process quits. | ||
| check_call([pybin, pip, 'install', PSUTIL]) | ||
|
|
||
| dirname = 'python-%s-%s' % (version, revision) | ||
| if os.path.isdir(dirname): | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure we want to drop support for python 3.9 just because emsdk bundles a bleeding edge version. I think the idea is that we still want to support the system version of python on older/stable linux releases.