Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update e2e ui tests, harmonize the inputs and test running #77

Merged
merged 1 commit into from
Feb 6, 2025
+90 −38
Merged

Update e2e ui tests, harmonize the inputs and test running #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 45 additions & 15 deletions .github/workflows/global-ci-bundle.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,12 @@ on:
required: false
type: string
default: main
api_tests_tiers:
description: |
The make command to execute from go-konveyor-tests repository
required: false
type: string
default: make test-tier0 test-tier1
api_hub_tests_ref:
description: |
The branch or PR of the Hub API tests from tackle2-hub repository to clone.
Expand All @@ -111,6 +117,15 @@ on:
required: false
type: string
default: main
ui_test_tags:
description: |
A comma separated list of test tags/tiers to select the ui tests to run. Each tag/tier
needs to be explicitly included in the list. For example, to use tiers 0, 1, and 2,
the value should be: "@tier0,@tier1,@tier2".
required: false
type: string
default: "@tier0"

workflow_dispatch:
inputs:
operator_bundle:
Expand Down Expand Up @@ -155,6 +170,12 @@ on:
required: false
type: string
default: main
api_tests_tiers:
description: |
The make command to execute from go-konveyor-tests repository
required: false
type: string
default: make test-tier0 test-tier1
ui_tests_ref:
description: |
The branch or PR of the tackle-ui-tests repository to clone.
Expand All @@ -167,6 +188,14 @@ on:
required: false
type: string
default: main
ui_test_tags:
description: |
A comma separated list of test tags/tiers to select the ui tests to run. Each tag/tier
needs to be explicitly included in the list. For example, to use tiers 0, 1, and 2,
the value should be: "@tier0,@tier1,@tier2".
required: false
type: string
default: "@tier0"

env:
operator_bundle: ttl.sh/konveyor-operator-bundle-${{ github.sha }}:2h
Expand Down Expand Up @@ -310,19 +339,20 @@ jobs:
run: |
export HUB_BASE_URL="http://$(minikube ip)/hub"
export HUB_TESTS_REF="${{ inputs.api_hub_tests_ref }}"
make test-tier0 test-tier1
${{ inputs.api_tests_tiers }}
working-directory: go-konveyor-tests

e2e-ui-integration-tests:
needs: check-images
if: ${{ inputs.run_ui_tests }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# TODO: Support cypress-split in main tackle-ui-test project
# split: [0, 1, 2, 3, 4, 5, 6, 7, 8]
tier: [tier1, tier2, tier3]

# Note: We can split the input string by comma if we want to run the tags in parallel
# strategy:
# fail-fast: false
# matrix:
# test-tags:
# - ${{ inputs.ui_test_tags }}

steps:
- name: Extract pull request number from inputs or PR description
Expand Down Expand Up @@ -413,7 +443,7 @@ jobs:
echo "UI_URL=https://$(minikube ip)" >>$GITHUB_ENV

- name: Run login tests
uses: cypress-io/github-action@v5
uses: cypress-io/github-action@v6
env:
CYPRESS_user: admin
CYPRESS_pass: password
Expand All @@ -423,27 +453,27 @@ jobs:
spec: "cypress/e2e/tests/login.test.ts"

- name: Run UI tests
uses: cypress-io/github-action@v5
uses: cypress-io/github-action@v6
env:
CYPRESS_INCLUDE_TAGS: "@${{ matrix.tier }}"
CYPRESS_INCLUDE_TAGS: "@${{ input.ui_test_tags }}"
CYPRESS_user: "admin"
CYPRESS_pass: "Dog8code"
CYPRESS_tackleUrl: "${{ env.UI_URL }}"
CYPRESS_git_user: "fakeuser"
CYPRESS_git_password: "${{ secrets.GITHUB_TOKEN }}"
CYPRESS_git_key: "${{ secrets.GITHUB_TOKEN }}"
# TODO: Support cypress-split in main tackle-ui-test project
# CYPRESS_split: "${{ strategy.job-total }}"
# CYPRESS_splitIndex: "${{ strategy.job-index }}"
with:
working-directory: tackle-ui-tests
# spec can be removed once https://github.com/konveyor/tackle-ui-tests/pull/1331 merges
spec: "**/*.test.ts"

- name: Upload cypress report data as artifact
- name: Upload cypress report data as artifact
uses: actions/upload-artifact@v4
if: always()
with:
name: tackle-ui-test-reports-${{ matrix.tier }}
name: tackle-ui-test-reports-${{ input.ui_test_tags }}
path: |
tackle-ui-tests/cypress/reports
tackle-ui-tests/cypress/screenshots
tackle-ui-tests/cypress/videos

68 changes: 45 additions & 23 deletions .github/workflows/global-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ on:
required: false
type: string
default: main
ui_test_tags:
description: |
A comma separated list of test tags/tiers to select the ui tests to run. Each tag/tier
needs to be explicitly included in the list. For example, to use tiers 0, 1, and 2,
the value should be: "@tier0,@tier1,@tier2".
required: false
type: string
default: "@tier0"
workflow_dispatch:
inputs:
tag:
Expand Down Expand Up @@ -149,6 +157,14 @@ on:
required: false
type: string
default: main
ui_test_tags:
description: |
A comma separated list of test tags/tiers to select the ui tests to run. Each tag/tier
needs to be explicitly included in the list. For example, to use tiers 0, 1, and 2,
the value should be: "@tier0,@tier1,@tier2".
required: false
type: string
default: "@tier0"

jobs:
e2e-api-integration-tests:
Expand Down Expand Up @@ -211,13 +227,14 @@ jobs:
memory: 'max'
cpus: 'max'

# TODO: Could just load all images found in this artifact so that people can rebuild multiple components if needed
- name: Load image
if: "${{ inputs.component_name != '' }}"
run: |
export SHELL=/bin/bash
eval $(minikube -p minikube docker-env)
docker load --input /tmp/${{ inputs.component_name }}.tar
for image in $(ls /tmp/images/*.tar); do
docker load --input ${image}
; done

- name: install konveyor
uses: konveyor/tackle2-operator/.github/actions/[email protected]
Expand Down Expand Up @@ -284,11 +301,13 @@ jobs:
e2e-ui-integration-tests:
if: ${{ inputs.run_ui_tests }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# TODO enable customizing the number of tiers run for nightlies
tier: [tier0] #, tier1, tier2, tier3]

# Note: We can split the input string by comma if we want to run the tags in parallel
# strategy:
# fail-fast: false
# matrix:
# test-tags:
# - ${{ inputs.ui_test_tags }}

steps:
- name: Extract pull request number from inputs or PR description
Expand Down Expand Up @@ -328,13 +347,14 @@ jobs:
memory: 'max'
cpus: 'max'

# TODO: Could just load all images found in this artifact so that people can rebuild multiple components if needed
- name: Load image
if: "${{ inputs.component_name != '' }}"
run: |
export SHELL=/bin/bash
eval $(minikube -p minikube docker-env)
docker load --input /tmp/${{ inputs.component_name }}.tar
for image in $(ls /tmp/images/*.tar); do
docker load --input ${image}
; done

- name: install konveyor
uses: konveyor/tackle2-operator/.github/actions/[email protected]
Expand Down Expand Up @@ -397,20 +417,20 @@ jobs:
echo $external_ip;
echo "UI_URL=https://$(minikube ip)" >>$GITHUB_ENV

# - name: Run login tests
# uses: cypress-io/github-action@v5
# env:
# CYPRESS_user: admin
# CYPRESS_pass: password
# CYPRESS_tackleUrl: "${{ env.UI_URL }}"
# with:
# working-directory: tackle-ui-tests
# spec: "cypress/e2e/tests/login.test.ts"
- name: Run login tests
uses: cypress-io/github-action@v6
env:
CYPRESS_user: admin
CYPRESS_pass: password
CYPRESS_tackleUrl: "${{ env.UI_URL }}"
with:
working-directory: tackle-ui-tests
spec: "cypress/e2e/tests/login.test.ts"

- name: Run UI tests
uses: cypress-io/github-action@v5
- name: Run UI tests (${{ inputs.ui_test_tags }})
uses: cypress-io/github-action@v6
env:
CYPRESS_INCLUDE_TAGS: "@${{ matrix.tier }}"
CYPRESS_INCLUDE_TAGS: "${{ inputs.ui_test_tags }}"
CYPRESS_user: "admin"
CYPRESS_pass: "Dog8code"
CYPRESS_tackleUrl: "${{ env.UI_URL }}"
Expand All @@ -419,13 +439,15 @@ jobs:
CYPRESS_git_key: "${{ secrets.GITHUB_TOKEN }}"
with:
working-directory: tackle-ui-tests
# spec can be removed once https://github.com/konveyor/tackle-ui-tests/pull/1331 merges
spec: "**/*.test.ts"

- name: Upload cypress report data as artifact
- name: Upload cypress report data as artifact
uses: actions/upload-artifact@v4
if: always()
with:
name: tackle-ui-test-reports-${{ matrix.tier }}
name: tackle-ui-test-reports-${{ inputs.test-tags }}
path: |
tackle-ui-tests/cypress/reports
tackle-ui-tests/cypress/screenshots
tackle-ui-tests/cypress/videos
Loading