DCMAW-11057: Push STS test image to dev and build #338

sandymay-dd · 2025-01-15T18:02:35Z

What changed

Containerises the API test suite and updates the STS mock push-to-main.yml action to build/tag/publish the test image to Dev and Build.

Why did it change

This partially implements the Testing step in the Secure Pipeline methodology, whereby automated tests are run and are used to determine whether to promote the application to the next environment. The other part of the implementation will be done in the https://github.com/govuk-one-login/mobile-id-check-async-infra repository (updating the secure pipeline and test image repository Cloudformation stacks).

This is a mock asset that isn't promoted to Production; however, it is still good to run these tests so that we are alerted in case there are issues in Dev or Build that we need to address.

This also introduces a new pattern for the team - Github reusable workflows (https://docs.github.com/en/actions/sharing-automations/reusing-workflows). The intent here is to create re-usable abstracted assets helping with maintainability and improves speed of adoption for new applications. This has only been done for functionality that builds/pushes/tags the docker image used in the test containers (AWS CodeBuild)

Testing

Action successfully runs for the Dev environment. It fails for Build due to IAM policy restrictions (only the main branch can push assets to Build`; this is expected.

run-tests-locally.sh passes, when run against the mob-sts-mock stack

Checklists

There is a ticket raised for this PR that is present in the branch name
No PII data logged. See guidance here
Demo to a BA, TA, and the team.
Update README with any new instructions or tasks

sandymay-dd · 2025-01-15T18:12:42Z

.github/workflows/shared-build-and-push-test-image.yml

+      - name: Setup nodeJS v20
+        uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af #v4.1.0
+        with:
+          node-version-file: '.nvmrc'


Enforces the use of an .nvmrc file (a standard we use in all our async stacks), but I think the easiest way to ensure this is running on the version of Node that we develop with. Happy for suggestions.

sandymay-dd · 2025-01-15T18:13:10Z

.github/workflows/shared-build-and-push-test-image.yml

+jobs:
+  build-and-push:
+    name: build-and-push
+    runs-on: ubuntu-24.04


Note this is a later version than what we use elsewhere (ubuntu-22)

sandymay-dd · 2025-01-15T18:23:50Z

.github/workflows/shared-build-and-push-test-image.yml

+          version: 1.132.0
+
+      - name: Install npm packages
+        run: npm install


note: also enforces the use of npm. This is a programme standard now and is preferred to yarn.

sandymay-dd · 2025-01-15T18:27:30Z

.github/workflows/shared-build-and-push-test-image.yml

annoyingly - I don't think you can have sub-directories for workflows. Happy for naming suggestions to make this file stand out.

A cheeky prefix?

_name.yml
or
action-${name}.yaml

or potentially job-{name}.yaml

(ideas for tomorrow me)

sandymay-dd · 2025-01-15T18:28:28Z

.github/workflows/sts-mock-push-to-main.yml


  build-and-upload-sam-artifact-to-dev:
    name: Validate & upload S3 artifact to dev
    runs-on: ubuntu-22.04
-    needs: sonar-scan


I don't believe our route to live should be blocked by a Sonar main branch analysis scan. We check the quality gate has been met on our pull-request workflows.

sandymay-dd · 2025-01-16T16:25:34Z

.github/workflows/shared-build-and-push-test-image.yml

+    secrets:
+      GH_ACTIONS_ROLE_ARN:
+        required: true
+      TEST_IMAGE_REPOSITORY:
+        required: true
+      CONTAINER_SIGN_KMS_KEY:
+        required: true


These aren't strictly secrets; however, I had issues passing these through as inputs. I would prefer to keep these as secrets in this template, to help understand provenance and maintain consistency.

Keeping them as secrets for the moment matches the other pipelines. I'm happy with this atm.

to clarify: I had technical issues, not moral ones 😀

sonarqubecloud · 2025-01-16T16:28:56Z

Quality Gate passed for 'mobile-id-check-async-sts-mock'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

sandymay-dd · 2025-01-16T16:34:09Z

sts-mock/Dockerfile

+COPY package.json package-lock.json /
+RUN npm install --ignore-scripts
+
+FROM node:iron-alpine AS final


I don't think there's any value of a multi-stage build given that we're copying all the node_modules over. Will remove. Thought this was saving space at the time (oops).

sandymay-dd · 2025-01-16T16:36:01Z

sts-mock/run-tests-locally.sh

+
+stack_name=${1:-mob-sts-mock}
+
+echo "Running test against ${stack_name}"


Suggested change

echo "Running test against ${stack_name}"

echo "Running tests against ${stack_name}"

sandymay-dd · 2025-01-16T17:08:21Z

.github/workflows/sts-mock-push-to-main.yml

-  #     id: cosign-image             
-  #     run: |
-  #       cosign sign --key awskms:///$BUILD_CONTAINER_SIGN_KMS_KEY $STS_MOCK_BUILD_TEST_IMAGE_REPOSITORY_URI:$IMAGE_TAG
+  test-image-build:


this calls the reusable workflow job

sandymay-dd · 2025-01-16T17:08:45Z

.github/workflows/sts-mock-push-to-main.yml

+    uses:
+      ./.github/workflows/shared-build-and-push-test-image.yml
+    with:
+      WORKING_DIRECTORY: sts-mock


will see if there's a way to avoid needing to pass this through

sandymay-dd · 2025-01-16T18:04:10Z

sts-mock/run-tests-locally.sh

+docker build --tag testcontainer .
+
+docker run --rm --interactive --tty \
+  --user root \


note to me: I think this is run as the root in the pipeline and we can't change that in ID Check.

sandymay-dd · 2025-01-17T11:46:27Z

.github/workflows/shared-build-and-push-test-image.yml

+          cosign-release: 'v1.9.0'
+
+      - name: Install SAM CLI
+        uses: aws-actions/setup-sam@819220f63fb333a9a394dd0a5cab2d8303fd17e2 #v2


Not needed!

sandymay-dd · 2025-01-17T11:46:53Z

.github/workflows/shared-build-and-push-test-image.yml

+          version: 1.132.0
+
+      - name: Install npm packages
+        run: npm install


could we just install --dev-dependencies

sandymay-dd · 2025-01-17T11:48:21Z

.github/workflows/shared-build-and-push-test-image.yml

+          version: 1.132.0
+
+      - name: Install npm packages
+        run: npm install


npm clean-install (alias for npm ci)

sandymay-dd · 2025-01-17T11:58:45Z

sts-mock/Dockerfile

+FROM node:iron-alpine AS builder
+
+COPY package.json package-lock.json /
+RUN npm install --ignore-scripts


sandymay-dd added 2 commits January 15, 2025 18:06

DCMAW-11057: Push STS test image to dev and build

dc49f1a

DCMAW-11057: Use nvmrc for Node version

f0bf11e

sandymay-dd commented Jan 15, 2025

View reviewed changes

sandymay-dd force-pushed the DCMAW-11057 branch from 25dd116 to f0bf11e Compare January 15, 2025 18:13

sandymay-dd commented Jan 15, 2025

View reviewed changes

sandymay-dd added 8 commits January 15, 2025 19:08

DCMAW-11057: move directory to env parameter

f031718

DCMAW-11057: move from secrets to inputs

1e85802

DCMAW-10057: env fix

e1db5a3

DCMAW-11057: fix working directory

b81efa7

DCMAW-11057: try secrets

f36a7d3

DCMAW-11057: fix cosign ref

f8d1cd9

DCMAW-11057: uncomment for testing

80b3a59

DCMAW-11057: fix nvmrc dir

911a814

sandymay-dd force-pushed the DCMAW-11057 branch from e22659f to 911a814 Compare January 15, 2025 19:08

sandymay-dd added 2 commits January 16, 2025 13:33

DCMAW-11057: add containerised test suite

b5a94f7

Merge branch 'main' into DCMAW-11057

b43d657

sandymay-dd commented Jan 16, 2025

View reviewed changes

DCMAW-11057: re-enable the rest of the workflow post testing

1b11f75

sandymay-dd marked this pull request as ready for review January 16, 2025 16:27

sandymay-dd requested review from a team as code owners January 16, 2025 16:27

sandymay-dd commented Jan 16, 2025

View reviewed changes

dothomson approved these changes Jan 16, 2025

View reviewed changes

sandymay-dd commented Jan 16, 2025

View reviewed changes

sandymay-dd commented Jan 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DCMAW-11057: Push STS test image to dev and build #338

DCMAW-11057: Push STS test image to dev and build #338

sandymay-dd commented Jan 15, 2025 •

edited

Loading

sandymay-dd Jan 15, 2025

sandymay-dd Jan 15, 2025

sandymay-dd Jan 15, 2025

sandymay-dd Jan 15, 2025

dothomson Jan 16, 2025

sandymay-dd Jan 16, 2025

sandymay-dd Jan 16, 2025

sandymay-dd Jan 15, 2025

sandymay-dd Jan 16, 2025 •

edited

Loading

dothomson Jan 16, 2025

sandymay-dd Jan 16, 2025

sonarqubecloud bot commented Jan 16, 2025

sandymay-dd Jan 16, 2025 •

edited

Loading

sandymay-dd Jan 16, 2025

sandymay-dd Jan 16, 2025

sandymay-dd Jan 16, 2025

sandymay-dd Jan 16, 2025 •

edited

Loading

sandymay-dd Jan 17, 2025

sandymay-dd Jan 17, 2025

sandymay-dd Jan 17, 2025 •

edited

Loading

sandymay-dd Jan 17, 2025


		stack_name=${1:-mob-sts-mock}

		echo "Running test against ${stack_name}"

	echo "Running test against ${stack_name}"
	echo "Running tests against ${stack_name}"

DCMAW-11057: Push STS test image to dev and build #338

Are you sure you want to change the base?

DCMAW-11057: Push STS test image to dev and build #338

Conversation

sandymay-dd commented Jan 15, 2025 • edited Loading

What changed

Why did it change

Checklists

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandymay-dd Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Jan 16, 2025

Quality Gate passed for 'mobile-id-check-async-sts-mock'

sandymay-dd Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandymay-dd Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandymay-dd Jan 17, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandymay-dd commented Jan 15, 2025 •

edited

Loading

sandymay-dd Jan 16, 2025 •

edited

Loading

sandymay-dd Jan 16, 2025 •

edited

Loading

sandymay-dd Jan 16, 2025 •

edited

Loading

sandymay-dd Jan 17, 2025 •

edited

Loading