-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade Python to 3.10 in app VM and psycopg2 to 2.9 #1378
Conversation
@@ -86,6 +86,9 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
end | |||
|
|||
config.vm.define "database" do |database| | |||
# SSH issues bringing up DB VM with base box 22.04 | |||
# Explicitly define downgraded version for use by database until fix | |||
database.vm.box = "bento/ubuntu-20.04" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When configuring all machines with a "bento/ubuntu-22.04" base box I ran into issues successfully provisioning and bringing up the database VM. As part of booting the VM it would hang on database: SSH auth method: private key
before failing with a timeout error. This has been a reported issue on Vagrant before, but all the previous solutions recommended in comments don't work. I previously ran into this issue once with Cicero and I was able to solve it by explicitly defining the ssh auth.username
and ssh.auth.password
in the Vagrantfile, but that did not work here either.
I settled on configuring a less-upgraded base box that doesn't have ssh issues (20.04) for the database and then configuring the more-upgraded (22.04) base box version for otp and app. I'm not sure if we're going to run into trouble with these two being different, but I struggled to manually install Python 3.10 for use in the Ubuntu 20.04 version so this was the only way I could figure out how to bring Python up past 3.8 in the app while also avoiding VirtualBox issues.
# Tasks and related artifacts copied from azavea.nginx role | ||
# Customized to use Nginx Stable PPA that works with Ubuntu 22.04 | ||
- name: Configure the Nginx PPA | ||
apt_repository: repo=ppa:ondrej/nginx state=present |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hard to tell since it's a new file, but line 5 has been changed from apt_repository: repo=ppa:nginx/{{ nginx_version }} state=present
. See notes for details on why this was needed.
@@ -0,0 +1,2 @@ | |||
--- | |||
nginx_delete_default_site: False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In addition to updating the nginx repo in the "Configure the Nginx PPA" task, I removed the line nginx_version: "stable"
here in the copied over defaults.yml
since it's no longer used by this role.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! Lots of detail and detours.
I tried provisioning database
with ubuntu-22.04
and ran into the same issue. I suspect it's a VirtualBox Guest Additions mismatch, but I'm not sure why it works for app
and otp
but not database
. Nevertheless, I think it's okay to keep database
at an older version for now. It's only used in development, and the database API is pretty consistent.
Thanks for the review! Added one more commit to clarify how to run migrations following our slack discussion. |
Overview
Upgrades the app VM from Python 3.6.9 to 3.10.12 as well as upgrades
psycopg2
from 2.8.6 to 2.9 in both app and database VM. Both of these upgrades support upcoming Django and PostgreSQL upgrades in the #1372 task list.Ideally we would get to Python version 3.12 since that is currently in
bugfix
maintenance status into 2025, but I ran into a variety of issues trying to upgrade to this version so am leaving it at 3.10.12. This version is still receiving security support through October 2026. (see notes for more detail)Demo
Notes
Context on the nginx role changes:
Upgrading to Python 3.10 failed app provisioning with the following error:
After digging it appears upgrading the upgraded Ubuntu version was too modern for the repo defined in the
azavea.nginx
role. Unfortunately, this is a separate repo so I had to copy over the entire role and update app's roledependencies in order to update this task to use the correct stable PPA.
Notes on issues upgrading to Python 3.12:
The Ubuntu 22.04 base box brings the Python default to 3.10.12, but ideally we would upgrade to Python 3.12 to give us the most runway before contract end. I tried to manually install Python 3.12 from PPA as part of app provisioning and then update the
ansible_python_interpreter
to locate the correct version to use, but ran into incompatibilities with the ansible version:The upgraded Python version got rid of some variables, like
cert_file
, that are used by ansible in theazavea.packer
role and we would need to upgrade localansible-core
to version >=2.15 for these to be compatible, see this ansible issue for more detail. This project depends on local installation ofansible
, but we have historically had issues using upgraded ansible versions beyond 6.0 with this project. The latest version ofansible
v 5.0 only includes up toansible-core
v 2.12.7. I could target an update foransible-core
to get these fixes, but I suspect this could open a can of worms and given our remaining budget and that we're able to at least get to Python 3.10 I think we should leave this for down the road. Curious if you think it's worth it!Testing Instructions
vagrant destroy
(Ensure you haveGraph.obj
in theotp_data
dir before destroying the otp VM so it doesn't rebuild the graph on up)vagrant up
(you may need to runCAC_APP_SHARED_FOLDER_TYPE=virtualbox vagrant up
)python3
version andpsycopg2-binary
versionpsycopg2
versionChecklist
Connects #1356