Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "PR tests reporting" #5387

Merged
merged 1 commit into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .changeset/olive-pianos-matter.md

This file was deleted.

47 changes: 0 additions & 47 deletions .github/actions/flaky-tests-report/action.yml

This file was deleted.

22 changes: 0 additions & 22 deletions .github/actions/run-pw-tests/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,6 @@ inputs:
E2E_ENCODE_PASS:
description: "Password for encoding credentials"
required: true
BRANCH_NAME:
description: "Branch name"
required: true
SALEOR_CLOUD_SERVICE:
description: "Saleor Cloud service"
required: true
DASHBOARD_VERSION:
description: "Dashboard version"
required: false

runs:
using: "composite"
Expand Down Expand Up @@ -91,18 +82,13 @@ runs:
WORKERS: ${{ inputs.PW_WORKERS }}
RETRIES: ${{ inputs.PW_RETRIES }}
PROJECT: ${{ inputs.PROJECT }}
BRANCH_NAME: ${{ inputs.BRANCH_NAME }}
SALEOR_CLOUD_SERVICE: ${{ inputs.SALEOR_CLOUD_SERVICE }}
DASHBOARD_VERSION: ${{ inputs.DASHBOARD_VERSION }}

run: |
PROJECTS=($PROJECT)
PROJECT_PARAMS=""
for PROJECT in "${PROJECTS[@]}"; do
PROJECT_PARAMS+="--project=${PROJECT} "
done
echo "reportName=all-blob-reports-${SHARD_NUMBER%%/*}" >> $GITHUB_ENV
echo "ctrfReport=ctrf-report-${SHARD_NUMBER%%/*}" >> $GITHUB_ENV
npx playwright test --grep @e2e $PROJECT_PARAMS --shard "$SHARD_NUMBER"

- name: Upload blob report to GitHub Actions Artifacts
Expand All @@ -111,12 +97,4 @@ runs:
with:
name: ${{ env.reportName }}
path: blob-report
retention-days: 1

- name: Upload ctrf report to GitHub Actions Artifacts
uses: actions/upload-artifact@v4
if: always()
with:
name: ${{ env.ctrfReport }}
path: ctrf/ctrf-report-*.json
retention-days: 1
21 changes: 2 additions & 19 deletions .github/workflows/pr-automation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ jobs:
BACKUP_VER: ${{ steps.cloud_variables.outputs.BACKUP_VER }}
BACKUP_NAME: ${{ steps.cloud_variables.outputs.BACKUP_NAME }}
SHARD_MATRIX: ${{ steps.generate-shard-matrix.outputs.SHARD_MATRIX }}
SALEOR_CLOUD_SERVICE: ${{ steps.cloud_variables.outputs.SALEOR_CLOUD_SERVICE }}
steps:
- uses: actions/checkout@v4
with:
Expand Down Expand Up @@ -75,7 +74,6 @@ jobs:
outputs:
ACCOUNTS: ${{ steps.accounts.outputs.ACCOUNTS }}
TESTMO_RUN_ID: ${{ steps.init-testmo.outputs.testmo-run-id }}
DASHBOARD_VERSION: ${{ steps.build-dashboard.outputs.DASHBOARD_VERSION }}
runs-on: ubuntu-22.04
needs: initialize-cloud
permissions:
Expand Down Expand Up @@ -113,7 +111,6 @@ jobs:
run: npm ci

- name: Build dashboard
id: build-dashboard
env:
## backward compatibility for older versions
API_URI: ${{ needs.initialize-cloud.outputs.API_URL }}
Expand All @@ -127,13 +124,7 @@ jobs:
IS_CLOUD_INSTANCE: true
ENABLED_SERVICE_NAME_HEADER: true
ONBOARDING_USER_JOINED_DATE_THRESHOLD: ${{ vars.STAGING_ONBOARDING_USER_JOINED_DATE_THRESHOLD }}
run: |
npm run build
# Extract version from package.json
VERSION=$(node -p "require('./package.json').version")
echo "VERSION=$VERSION"
echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "DASHBOARD_VERSION=$VERSION" >> $GITHUB_OUTPUT
run: npm run build

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
Expand Down Expand Up @@ -212,9 +203,6 @@ jobs:
PW_RETRIES: ${{ vars.PW_RETRIES }}
ACCOUNTS: ${{ needs.deploy-dashboard.outputs.ACCOUNTS }}
E2E_ENCODE_PASS: ${{ secrets.E2E_ENCODE_PASS }}
DASHBOARD_VERSION: ${{ needs.deploy-dashboard.outputs.DASHBOARD_VERSION }}
BRANCH_NAME: ${{ github.ref}}
SALEOR_CLOUD_SERVICE: ${{ needs.initialize-cloud.outputs.SALEOR_CLOUD_SERVICE }}

- name: submit-results-to-testmo
if: always()
Expand All @@ -240,9 +228,4 @@ jobs:
with:
testmoUrl: ${{ secrets.TESTMO_URL }}
testmoToken: ${{ secrets.TESTMO_TOKEN }}
testmoRunId: ${{ needs.deploy-dashboard.outputs.TESTMO_RUN_ID }}

- name: Create flaky report
uses: ./.github/actions/flaky-tests-report
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
testmoRunId: ${{ needs.deploy-dashboard.outputs.TESTMO_RUN_ID }}
14 changes: 0 additions & 14 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,6 @@
"mochawesome": "^7.1.3",
"mochawesome-merge": "^4.3.0",
"mochawesome-report-generator": "^6.2.0",
"playwright-ctrf-json-reporter": "^0.0.18",
"playwright-testmo-reporter": "^1.8.4",
"rollup-plugin-polyfill-node": "^0.11.0",
"typescript": "^5.0.4",
Expand Down
40 changes: 15 additions & 25 deletions playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ dotenv.config();

const env = process.env;
const DEFAULT_WORKERS = "2";
const shardNumber = env.SHARD_NUMBER?.match(/^\d*/)?.[0] || "0";

// const DEFAULT_RETRIES = "1";

// FIXME: High timeouts are a temporary solution to handle slower CI environments.
Expand All @@ -23,31 +21,23 @@ export default defineConfig({
// as the test will most likely fail again. We can enable it later if needed.
// retries: parseInt(env.RETRIES || DEFAULT_RETRIES),
workers: parseInt(env.WORKERS || DEFAULT_WORKERS),

reporter: process.env.CI
? [
["blob"],
["github"],
[
"playwright-testmo-reporter",
{
outputFile: "testmo/testmo.xml", // Optional: Output file path. Defaults to 'testmo.xml'.
embedBrowserType: true, // Optional: Embed browser type in the XML file. Defaults to false.
embedTestSteps: true, // Optional: Embed test steps in the XML file. Defaults to true.
testStepCategories: ["hook", "expect", "pw:api", "test.step"], // Optional: Test step categories to include in the XML file. Defaults to ["hook","expect","pw:api","test.step"]. Possible options are "hook", "expect", "pw:api", "test.step".
testTitleDepth: 1, // Optional: Test case title depth to report in the XML file. Defaults to 1. Increase this to 2 include suite name. Increase this even further to include the path.
attachmentBasePathCallback: () =>
process.env.URL_TO_RUN ? process.env.URL_TO_RUN : "",
},
],
['playwright-ctrf-json-reporter', {
outputFile: `ctrf-report-${shardNumber}.json`, // Optional: Output file name. Defaults to 'ctrf-report.json'.
minimal: true, // Optional: Generate a minimal report. Defaults to 'false'. Overrides screenshot and testType when set to true
appName: 'Saleor Dashboard', // Optional: Specify the name of the application under test.
appVersion: env.DASHBOARD_VERSION || '', // Optional: Specify the version of the application under test.
branchName: env.BRANCH_NAME || '', // Optional: Specify the branch name.
testEnvironment: env.SALEOR_CLOUD_SERVICE || '' // Optional: Specify the test environment (e.g. staging, production).
}]]
["blob"],
["github"],
[
"playwright-testmo-reporter",
{
outputFile: "testmo/testmo.xml", // Optional: Output file path. Defaults to 'testmo.xml'.
embedBrowserType: true, // Optional: Embed browser type in the XML file. Defaults to false.
embedTestSteps: true, // Optional: Embed test steps in the XML file. Defaults to true.
testStepCategories: ["hook", "expect", "pw:api", "test.step"], // Optional: Test step categories to include in the XML file. Defaults to ["hook","expect","pw:api","test.step"]. Possible options are "hook", "expect", "pw:api", "test.step".
testTitleDepth: 1, // Optional: Test case title depth to report in the XML file. Defaults to 1. Increase this to 2 include suite name. Increase this even further to include the path.
attachmentBasePathCallback: () =>
process.env.URL_TO_RUN ? process.env.URL_TO_RUN : "",
},
],
]
: [["html"], ["list"]],
expect: { timeout: 10 * 1000 },
maxFailures: 10,
Expand Down
Loading