Changing the version of Magic Castle used by MC Hub

This guide will go over the steps necessary to do an update of the version of Magic Castle used in MC Hub.

1. Setup

If you don't already have MC Hub setup locally, do the following steps.

1.1 Clone MC Hub locally

git clone https://github.com/ComputeCanada/mc-hub.git

1.2 Configure MC Hub

Follow the guide here.

2. Update MC Hub with your code

2.1 Updating the MC version number

Open the file app/models/constants.py and change the line containing the currently used version of Magic Castle.

For example, change the following line:

MAGIC_CASTLE_VERSION_TAG = "10.0"

To this:

MAGIC_CASTLE_VERSION_TAG = "10.1"

2.2 Updating the Terraform required version

If the version of Terraform required to run MC changed, do the following. Still in the file app/models/constants.py, change the line containing the Terraform required version.

For example, change the following line:

TERRAFORM_REQUIRED_VERSION = ">= 0.13.0"

To this:

TERRAFORM_REQUIRED_VERSION = ">= 0.14.5"

2.3 Changing the structure of the Magic Castle configuration (optional)

If you need to change the schema of a Magic Castle configuration, which contains a subset of the variables in the main.tf file, read this section. If the structure of main.tf did not change, you can skip this section.

Note: a good example of configuration schema change can be found in this commit.

Add the required parameter to the Marshmallow schema in app/models/magic_castle/magic_castle_configuration_schema.py. For example:

new_field = fields.Str(missing="")

As you can see the field is not required and has a default value of "". This is done to ensure MC Hub is backwards compatible with clusters created without the parameter.

You will also need to update the tests to use the new configuration parameter. This includes the following files:

• app/tests/integration/test_api_no_auth.py
• app/tests/integration/test_api_with_auth.py
• app/tests/unit/magic_castle/test_magic_castle.py
• app/tests/unit/magic_castle/test_magic_castle_configuration.py

Finally, you will need to update the frontend Vue Js code to display a new input field.

Go to frontend/src/components/cluster/ClusterDisplay.vue and update the DEFAULT_MAGIC_CASTLE constant to include the new parameter.

Then, go to frontend/src/components/cluster/ClusterEditor.vue and add an input field supported by the Vuetify components library. For example, it could be the following:

<v-list-item>
  <v-text-field v-model="magicCastle.new_field" label="New field" />
</v-list-item>

2.4 Last changes

Update the CHANGELOG:

## [v7.5.1] 2021-03-05

### Changed
- Bumped MC to 10.1

Before pushing the changes, you can should verify that all tests from the basic test suite are passed.

cd app
python -m pytest

Then, you can run the full test suite, which includes tests that build a real cluster on the cloud. These will take a few minutes to complete.

python -m pytest --build-live-cluster

3. Publish a new release of MC Hub

If the tests pass and you are satisfied by your changes, you can push the changes and add the tag v7.5.1 to your git commit. Adding a tag to the commit will have the effect of triggering the creation of a Docker image on Docker Hub.

git tag v7.5.1
git push --follow-tags origin master

Now, you will have to wait for Travis CI to build the MC Hub Docker image and run the tests on it. This includes the creation of a live cluster on the ipm-500 project on Arbutus (make sure that the quotas are respected). Finally, if all tests pass, Travis CI will automatically deploy the built image on the Docker Hub repository. When this is done, you should be able to see the new version tag on Docker Hub.

Then, go to GitHub to publish a new release associated with the version tag v7.5.1. Go to https://github.com/ComputeCanada/mc-hub/releases/new.

The release title should have the format MC Hub v7.5.1 and the release description should match exactly the CHANGELOG for the latest release. Then, click on Publish release.

3. Deploying MC Hub with Ansible

Warning: When updating MC Hub, all existing cluster clusters that get modified will be automatically upgraded to the new version of MC you are using.

If you are an administrator of mc.computecanada.dev, follow the instructions to run the Ansible playbook.

Otherwise, if you have your own deployment, you can follow the instructions here.

In both cases, you will need to update the attribute mc_hub_version with the new MC Hub tag in the hosts.yml file.

all:
  hosts:
    example.com:
      ...
      mc_hub_version: "v7.5.1"