Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

717 test production mode in integration test #736

Merged
merged 22 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
f3ec015
update github workflow to include prod testing 9might be inn the wron…
Bdegraaf1234 Oct 9, 2023
24dc919
insert the e2e-prod script into the github workflow as that is what w…
Bdegraaf1234 Oct 9, 2023
68fc25a
let it run on the ci for now
Bdegraaf1234 Oct 9, 2023
f97b720
baseurl for cypress should match the ALDS
Bdegraaf1234 Oct 9, 2023
448bfba
shutdown the docker containers, it feels like there are some persiste…
Bdegraaf1234 Oct 9, 2023
1e2e7ff
or de we just need to force click?
Bdegraaf1234 Oct 9, 2023
fa61351
or both?
Bdegraaf1234 Oct 9, 2023
2460903
or both?
Bdegraaf1234 Oct 9, 2023
836c19d
no more alternate docker command needed for windows
Bdegraaf1234 Oct 10, 2023
b2a1549
apply dev context from the app.yml docker config. This removes the ne…
Bdegraaf1234 Oct 10, 2023
f1c2787
The test prod task fails when ran as a docker image, but pass when ra…
Bdegraaf1234 Oct 10, 2023
7ef184e
The test prod task fails when ran as a docker image, but pass when ra…
Bdegraaf1234 Oct 10, 2023
8ec99b1
remove some configurations that are already inherited from the defaul…
Bdegraaf1234 Oct 10, 2023
c1dfac7
prod docker image testing is a stub in the docker CI task
Bdegraaf1234 Oct 10, 2023
6753bb5
Change the run-prod-e2e.sh script to reflect the new changes. This sc…
Bdegraaf1234 Oct 10, 2023
5c6f52b
minor updates to the README to reflect the new situation
Bdegraaf1234 Oct 10, 2023
25e5f9c
Merge branch 'dev' into 717-test-production-mode-in-integration-test
Bdegraaf1234 Nov 7, 2023
01c1292
Revert "let it run on the ci for now"
Bdegraaf1234 Nov 7, 2023
dc2f453
update yml indentation
Bdegraaf1234 Nov 8, 2023
14b5ce2
faulty test config
Bdegraaf1234 Nov 9, 2023
293fc4a
enable appserver downstream testing
Bdegraaf1234 Nov 10, 2023
cb1a53e
update codeowners
Bdegraaf1234 Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,16 @@ jobs:
org.opencontainers.image.vendor=RADAR-base
org.opencontainers.image.licenses=Apache-2.0

# # placeholder for testing
# - name: End to end tests (docker image)
# id: e2e-prod-docker
# run: |
# cp src/test/resources/config/keystore.p12 src/main/docker/etc/config
# ./gradlew -Pprod buildDocker -x test -x javadocJar
# docker-compose -f src/main/docker/app.yml up -d
# yarn run e2e-prod
# docker-compose -f src/main/docker/app.yml down -v

# Temp fix
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
Expand Down
10 changes: 4 additions & 6 deletions .github/workflows/pre-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,14 +51,12 @@ jobs:
run: |
yarn install

- name: Run full end to end tests
- name: Run full production end to end tests
run: |
cp src/test/resources/config/keystore.p12 src/main/resources/config/keystore.p12
docker-compose -f src/main/docker/postgresql.yml up -d
./gradlew bootRun &>mp.log </dev/null &
yarn run wait-for-managementportal
./gradlew generateOpenApiSpec
yarn e2e
./gradlew --stop
yarn run e2e-prod
Bdegraaf1234 marked this conversation as resolved.
Show resolved Hide resolved
docker-compose -f src/main/docker/postgresql.yml down -v

- name: Upload screenshots of failed e2e tests
if: always()
Expand Down
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Management Portal is an application which is used to manage clinical studies for
- [Testing](#testing)
* [Client tests](#client-tests)
* [Other tests](#other-tests)
- [Using Docker to simplify development (optional)](#using-docker-to-simplify-development--optional-)
- [Using Docker to simplify development (optional)](#using-docker-to-simplify-development-optional)
- [Documentation](#documentation)
- [Client libraries](#client-libraries)

Expand Down Expand Up @@ -49,7 +49,7 @@ docker-compose files.
3. Now, we can start the stack with `docker-compose -f src/main/docker/management-portal.yml up -d`.

This will start a Postgres database and ManagementPortal. The default password for the `admin`
account is `admin`.
account is `admin`. An angular live development server to access the managementportal can be started using the `yarn start` command (see [Development](#development)).

### Build from source

Expand All @@ -68,15 +68,14 @@ You must install and configure the following dependencies on your machine to run
main differences between the profiles. Configure the application using the property file at `src/main/resources/config/application-<profile>.yml`.Read more about configurations [here](#configuration)

5. Run ManagementPortal by running `./gradlew bootRun -Pprod` or `./gradlew bootRun -Pdev`. Development mode will start an in
memory database and ManagementPortal.
memory database and ManagementPortal. An angular live development server to access the managementportal can be started using the `yarn start` command (see [Development](#development)).
6. You can log in to the application using `admin:admin`. Please don't forgot to change the password of `admin`, if you are using the application on production environment.


| | Development | Production |
|----------------------------------|-----------------|-----------------------------------|
| Database type | In-memory | Postgres |
| Demo data loaded | Yes | No |
| Context path of the application | `/` | `/managementportal` |
Bdegraaf1234 marked this conversation as resolved.
Show resolved Hide resolved



Expand Down Expand Up @@ -249,6 +248,8 @@ auto-refreshes when files change on your hard drive.

Then open <http://localhost:8081/> to start the interface and sign in with admin/admin.

### Managing dependencies

[Yarn][] is also used to manage CSS and JavaScript dependencies used in this application. You can upgrade dependencies by
specifying a newer version in [package.json](package.json). You can also run `yarn update` and `yarn install` to manage dependencies.
Add the `help` flag on any command to see how you can use it. For example, `yarn help update`.
Expand Down Expand Up @@ -294,15 +295,15 @@ To launch your application's tests, run:

### Client tests

Unit tests are run by [Karma][] and written with [Jasmine][]. They're located in [src/test/javascript/](src/test/javascript/) and can be run with:
Unit tests are run by [Karma][] and written with [Jasmine][]. They're located in `src/test/javascript/` and can be run with:

yarn test

UI end-to-end tests are powered by [Cypress][], which is built on top of WebDriverJS. They're located in [src/test/javascript/e2e](src/test/javascript/e2e)
and can be run by starting Spring Boot in one terminal (`./gradlew bootRun`) and running the tests (`yarn run e2e`) in a second one.
### Other tests

Performance tests are run by [Gatling][] and written in Scala. They're located in [src/test/gatling](src/test/gatling) and can be run with:
Performance tests are run by [Gatling][] and written in Scala. They're located in `src/test/gatling` and can be run with:

./gradlew gatlingRunAll

Expand Down
4 changes: 1 addition & 3 deletions angular.json
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,7 @@
},
"configurations": {
"production": {
"devServerTarget": "management-portal:serve:production",
"watch": false,
"headless": true
"devServerTarget": "management-portal:serve:production"
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion cypress.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@
"screenshotsFolder": "src/test/javascript/e2e/cypress/screenshots",
"pluginsFile": false,
"fixturesFolder": false,
"baseUrl": "http://localhost:9000"
"baseUrl": "http://localhost:8081"
}
6 changes: 1 addition & 5 deletions gradle/docker.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,5 @@ task buildDocker(type: Exec, dependsOn: bootWar) {
include "*.war*"
}
}
if (OperatingSystem.current().isWindows()) {
commandLine 'cmd', '/c', 'docker', 'image', 'build', '-f', 'build/docker/Dockerfile', '-t', 'managementportal', 'build/docker/'
} else {
commandLine 'docker', 'image', 'build', '-f', 'build/docker/Dockerfile', '-t', 'managementportal', 'build/docker/'
}
commandLine 'docker', 'image', 'build', '-f', 'build/docker/Dockerfile', '-t', 'managementportal', 'build/docker/'
}
1 change: 1 addition & 0 deletions src/main/docker/app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ services:
- SPRING_DATASOURCE_URL=jdbc:postgresql://managementportal-postgresql:5432/managementportal
- SPRING_DATASOURCE_USERNAME=radarbase
- SPRING_DATASOURCE_PASSWORD=radarbase
- SPRING_LIQUIBASE_CONTEXTS=dev #includes testing_data, remove for production builds
- MANAGEMENTPORTAL_FRONTEND_CLIENT_SECRET=secret
- JHIPSTER_SLEEP=10 # gives time for the database to boot before the application
- JAVA_OPTS=-Xmx512m # maximum heap size for the JVM running ManagementPortal, increase this as necessary
Expand Down
23 changes: 23 additions & 0 deletions src/main/resources/config/application-e2e-prod-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# ===================================================================
# Spring Boot configuration for the "dev" profile.
#
# This configuration overrides the application.yml file.
#
# More information on profiles: https://jhipster.github.io/profiles/
# More information on configuration properties: https://jhipster.github.io/common-application-properties/
# ===================================================================

# ===================================================================
# Standard Spring Boot properties.
# Full reference is available at:
# http://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
# ===================================================================

spring:
profiles:
active: prod,api-docs,e2e-prod-test
datasource:
username: radarbase
password: radarbase
liquibase:
contexts: dev
14 changes: 6 additions & 8 deletions src/main/resources/config/application-prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ spring:
livereload:
enabled: false
datasource:
type: com.zaxxer.hikari.HikariDataSource
driverClassName: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/managementportal
# name: managementportal
username: <username>
password: <password>
type: com.zaxxer.hikari.HikariDataSource
Bdegraaf1234 marked this conversation as resolved.
Show resolved Hide resolved
driverClassName: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/managementportal
# name: managementportal
# username: radarbase
# password: radarbase
# hikari:
# data-source-properties:
# cachePrepStmts: true
Expand Down Expand Up @@ -53,8 +53,6 @@ spring:
starttls.enable: false
thymeleaf:
cache: true
liquibase:
contexts: prod
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be removed? Would it not be overridden by the e2e-prod-test profile?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it to the apllication.yml, the reasoning being that it is the default unless overriddedn by one of the dev profiles. Don't feel super strongly on this.


# ===================================================================
# To enable SSL, generate a certificate using:
Expand Down
3 changes: 3 additions & 0 deletions src/main/resources/config/application.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,9 @@ spring:
basename: i18n/messages
thymeleaf:
mode: HTML
liquibase:
contexts: prod


springdoc:
api-docs:
Expand Down
26 changes: 5 additions & 21 deletions src/test/bash/run-prod-e2e.sh
Original file line number Diff line number Diff line change
@@ -1,38 +1,22 @@
#!/bin/bash

# For testing production mode e2e tests we need some tweaks:
# 1) The angular app needs to run in dev mode for protractor to work
# 2) We update the basehref of the angular app to the context path of the backend
# 3) We update the protractor configuration to the new path
#
# Then we start up a docker stack with a postgres server since production mode is configured for a
# For testing production mode e2e tests we start up a docker postgres server since production mode is configured for a
# postgres database instead of in-memory database.

set -e

# only run on the release branch and master branch if it's not a tag build
echo "Running production e2e tests"
cp src/test/resources/config/keystore.p12 src/main/docker/etc/config
cp src/test/resources/config/keystore.p12 src/main/resources/config

# set liquibase context to dev so it loads demo data
sed -i "s|contexts: prod|contexts: dev|" src/main/resources/config/application-prod.yml
./gradlew -Pprod buildDocker -x test -x javadocJar
# recover the prod liquibase context
git checkout src/main/resources/config/application-prod.yml

docker-compose -f src/main/docker/app.yml up -d # spin up production mode application
./gradlew bootrun -Pe2e-prod-test -x test -x javadocJar &
docker-compose -f src/main/docker/postgresql.yml up -d # spin up production mode application
set +e

# wait for app to be up
yarn run wait-for-managementportal
# run e2e tests against production mode
if ./gradlew generateOpenApiSpec && yarn run e2e; then
if ./gradlew generateOpenApiSpec && yarn run e2e-prod; then
EXIT_STATUS=0
else
EXIT_STATUS=1
fi
docker-compose -f src/main/docker/app.yml logs # show output of app startup
docker-compose -f src/main/docker/app.yml down -v # clean up containers and volumes

docker-compose -f src/main/docker/postgresql.yml down -v # clean up containers and volumes
exit $EXIT_STATUS