Assess various upgrades for continued application stability

[rachelekm]

We need to run various upgrades spring 2024 to support application stability through the end of the contract. We should generate a task list of the different upgrades, their priority levels and effort estimates. Of note, from Derek:

• Django: We are on the previous LTS version, 3.2, and we need to upgrade to 4.2. This will be supported with security fixes through April 2026. This has been tricky in the past because of the custom admin theme that we maintain.
• Boto3: This is part of the deployment scripts. As far as I can tell it's just used to update the DNS. Boto 2 is deprecated and unmaintained.
• PostgreSQL: We are currently on Postgresql 11 which is supported through this November. Version 14 will be supported through November 2026. This will likely include downtime.

[rachelekm]

The immediate updates that affect the stability of the application through the end of the contract are Boto3, Django, Python, and PostgreSQL and PostGIS, as noted above.
We are also using a version of Nodejs many versions behind the current LTS, but this should be deferred in favor of other upgrades that have greater impact on maintaining the application through contract end (There is a standing issue to setup nvm which captures a needed update for our frontend development process). Specific details into individual upgrades are below, listed in order of proposed completion.

TL;DR The most immediate task is the Boto->Boto3 upgrade as this impacts our monthly deployment. Following that upgrade, the remaining tasks are more dependent on each other and the plan is to complete the below as 3-4 tasks on a separate branch that we can merge in once all properly tested together(updated, see slack discussion). Python upgrades should be completed first as it blocks Django and PostgreSQL upgrades. Django has immediate support deadlines with less effort than following upgrades so we should complete this task next. Upgrading PostgreSQL from 12.7 to 14 is not expected to have difficulties but will be a significant upgrade given downtime of the production database and lack of a staging database. To lower risk and potential developer time, we could create a separate task to create a mock staging database and run these updates there before running on the production database. Regardless there is a task to gain clarity of database upgrade steps with ops and potentially create temporary staging database.

Upgrade deployment dependency Boto to Boto3:

• Switch from boto from boto3 #1116

This package is installed as part of a local virtual environment we create during the deployment process. Boto is no longer supported, we should upgrade to Boto3 and confirm no issues with deployments. As in Derek's notes above its only use is for Route53 so hopefully limited issues on upgrade.

Upgrade Django 3.2.13 to 4.2 and Python 3.6.9 to 3.8.18:

• Upgrade Python in the app VM #1356
• Upgrade Django from 3.2.13 to 4.2 #1373

We are currently using the last Django LTS version (3.2.13) which ends security support April 1,2024. We should upgrade to the next version, 4.2, which we will not be able to do until we upgrade Python to 3.8.18. We should upgrade to 3.12 to give us maximum runway, see slack discussion. Upgrading Python is covered in 1356 and a separate task is created for upgrading Django. Even though it is a major version change some depreciation warnings have already been addressed in previous work,1320 , but we should still anticipate running into issues with our django-wpadmin dependency since this is an upstream package no longer maintained and forked long ago.

As part of this task we will also need to upgrade psycopg2 2.8.6 to 2.9.0 as the current version limits our ability to upgrade from PostgreSQL 12 in the next task.

Upgrade PostgreSQL 12.7 to 14 and PostGIS 3.0 to 3.1:

• Run PostgreSQL upgrades locally & in new testing environment #1374
• Upgrade PostgreSQL 12 to 14 and PostGIS 3.0 to 3.1 in Production #1375

The last PostgreSQL update was completed by first testing on develop and then running updates on the Production database. See step-by-step here. There shouldn't be difficulties between versions given the last few PostgreSQL upgrades and available documentation. However, we have limited developer familiarity with production database updates, last run 2 yeas ago, and at one point our development environment had gotten out of sync with our RDS environment (though this should have been addressed in last update). In addition, we are upgrading Python versions, psycopg2, and PostGIS in other tasks. Given scope of upgrades and a potential lack of parity and familiarity, a temporary staging database made from a duplicate of production to run upgrades would lower risk and production downtime.

To upgrade PostgreSQL, if not creating a separate staging db for testing, follow instructions from last upgrade here. PostgreSQL 14 has community and Amazon RDS support through 2026 and we will need to upgrade PostGIS to 3.1 to support. In the last few PostgreSQL upgrades there have been few difficulties and similarly for PostGIS because this is a minor upgrade and the last one did not have difficulties we can expect limited issues. In past upgrade we estimated ~2 hours of production database downtime.

[rachelekm]

Closing this issue after updating task list with recommendations from slack discussion.