Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(demo-playwright): add Playwright #4992

Merged
merged 1 commit into from
Aug 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions .github/actions/variables/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,11 @@ runs:
export IS_DEPENDABOT="${{ github.actor == 'dependabot[bot]' || github.actor == 'dependabot-preview[bot]' }}"
export CYPRESS_failOnSnapshotDiff="0"
export CYPRESS_API_URL="http://localhost:1234/"
export SNAPSHOTS_PATH="./projects/demo-integrations/cypress/snapshots"
export CYPRESS_SNAPSHOTS_PATH="./projects/demo-integrations/cypress/snapshots"
export PLAYWRIGHT_SNAPSHOTS_PATH="./projects/demo-playwright/tests-results"
export SNAPSHOTS_CACHE_KEY="e2e-cache--${{ github.event.pull_request.head.sha || github.sha }}-${{ github.event.number }}"
export SNAPSHOTS_ARTIFACTS_KEY="e2e-artifacts--${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_id }}-${{ github.event.number }}"
export CYPRESS_SNAPSHOTS_ARTIFACTS_KEY="cypress-e2e-artifacts--${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_id }}-${{ github.event.number }}"
export PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY="playwright-e2e-artifacts--${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_id }}-${{ github.event.number }}"

if [[ "$IS_FORK" == "false" && "$IS_DEPENDABOT" == "false" ]]; then
export IS_OWNER_MODE="true"
Expand Down Expand Up @@ -74,6 +76,8 @@ runs:
echo "SUPPORT_AUTO_PUSH=$SUPPORT_AUTO_PUSH" >> $GITHUB_ENV
echo "CYPRESS_failOnSnapshotDiff=$CYPRESS_failOnSnapshotDiff" >> $GITHUB_ENV
echo "CYPRESS_API_URL=$CYPRESS_API_URL" >> $GITHUB_ENV
echo "SNAPSHOTS_PATH=$SNAPSHOTS_PATH" >> $GITHUB_ENV
echo "SNAPSHOTS_ARTIFACTS_KEY=$SNAPSHOTS_ARTIFACTS_KEY" >> $GITHUB_ENV
echo "CYPRESS_SNAPSHOTS_PATH=$CYPRESS_SNAPSHOTS_PATH" >> $GITHUB_ENV
echo "PLAYWRIGHT_SNAPSHOTS_PATH=$PLAYWRIGHT_SNAPSHOTS_PATH" >> $GITHUB_ENV
echo "CYPRESS_SNAPSHOTS_ARTIFACTS_KEY=$CYPRESS_SNAPSHOTS_ARTIFACTS_KEY" >> $GITHUB_ENV
echo "PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY=$PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY" >> $GITHUB_ENV
echo "SNAPSHOTS_CACHE_KEY=$SNAPSHOTS_CACHE_KEY" >> $GITHUB_ENV
3 changes: 2 additions & 1 deletion .github/screenshot-bot.config.toml
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
# array of RegExp strings to match workflow names
# which should be watched by bot
workflowWithTests = [
'.*E2E Summary.*', # all workflows with sub-string "e2e" in their names will be watched by bot
'.*E2E*', # all workflows with sub-string "e2e" in their names will be watched by bot
]

# array of RegExp strings to match images inside artifacts (by their path or file name)
# which shows difference between two screenshot and which will be added to bot report comment
screenshotsDiffsPaths = [
'.*__diff_output__.*', # it is default cypress folder name into which snapshot diffs are put
'.*-diff.png'
]

# RegExp string to match images inside artifacts (by their path or file name)
Expand Down
53 changes: 0 additions & 53 deletions .github/workflows/e2e-summary.yml

This file was deleted.

108 changes: 93 additions & 15 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
uses: ./.github/actions/nodejs

- name: Building demo-app of git-branch without cache
run: npx nx build demo -c next --output-path ${{ env.DIST }}
run: npx nx build demo -c next --output-path ${{ env.DIST }} --base-href='/'

- name: Upload cache / ${{ env.CACHE_DIST_KEY }}
uses: actions/cache/[email protected]
Expand Down Expand Up @@ -97,40 +97,118 @@ jobs:
- name: Clean up resources
run: npx kill-port --port ${{ env.NG_SERVER_PORT }}

- name: Upload artifacts / ${{ env.SNAPSHOTS_ARTIFACTS_KEY }}
- name: Debug output
continue-on-error: true
run: tree ${{ env.CYPRESS_SNAPSHOTS_PATH }}

- name: Upload artifacts / ${{ env.CYPRESS_SNAPSHOTS_ARTIFACTS_KEY }}
uses: actions/[email protected]
with:
path: ${{ env.CYPRESS_SNAPSHOTS_PATH }}/**/*.diff.png
name: ${{ env.CYPRESS_SNAPSHOTS_ARTIFACTS_KEY }}
if-no-files-found: ignore
retention-days: 1

playwright:
if: ${{ !contains(github.head_ref , 'release/') }}
runs-on: ubuntu-latest
needs: [build-demo]
steps:
- uses: actions/[email protected]
- name: Setup global variables
uses: ./.github/actions/variables
- name: Setup Node.js and Cache
uses: ./.github/actions/nodejs

- name: Install Playwright Browsers
run: npx playwright install --with-deps

- name: Download cache / ${{ env.CACHE_DIST_KEY }}
uses: actions/cache/[email protected]
with:
path: dist/demo
key: ${{ env.CACHE_DIST_KEY }}

- name: Serve ${{ env.DIST }} in background
run: npx nx serve-compiled demo --path ${{ env.DIST }} --port ${{ env.NG_SERVER_PORT }}

- name: Run screenshot tests on ${{ env.DIST }}
run: npx nx e2e demo-playwright -- --update-snapshots

- name: Clean up resources
run: npx kill-port --port ${{ env.NG_SERVER_PORT }}

- name: Download ${{ env.DIST_NEXT }} for serve locally
run: |
git clone \
--depth 1 \
--branch snapshots/demo/next/${{ github.base_ref }} \
https://github.com/Tinkoff/taiga-ui.git ${{ env.DIST_NEXT }}

- name: Find and replace baseHref for next snapshot
uses: jacobtomlinson/gha-find-replace@v2
with:
find: '<base href="./">'
replace: '<base href="/">'
include: '${{ env.DIST_NEXT }}/index.html'
regex: false

- name: Serve ${{ env.DIST_NEXT }} in background
run: npx nx serve-compiled demo --path ${{ env.DIST_NEXT }} --port ${{ env.NG_SERVER_PORT }}

- name: Run screenshot tests on ${{ env.DIST_NEXT }}
continue-on-error: true
run: npx nx e2e demo-playwright

- name: Clean up resources
run: npx kill-port --port ${{ env.NG_SERVER_PORT }}

- name: Debug output
continue-on-error: true
run: tree ${{ env.PLAYWRIGHT_SNAPSHOTS_PATH }}

- name: Upload artifacts / ${{ env.PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY }}
uses: actions/[email protected]
with:
path: ${{ env.SNAPSHOTS_PATH }}/**/*.diff.png
name: ${{ env.SNAPSHOTS_ARTIFACTS_KEY }}
path: ${{ env.PLAYWRIGHT_SNAPSHOTS_PATH }}/**/*-diff.png
name: ${{ env.PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY }}
if-no-files-found: ignore
retention-days: 1

result:
if: ${{ !contains(github.head_ref , 'release/') }}
name: result
needs: [cypress]
needs: [cypress, playwright]
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- name: Setup global variables
uses: ./.github/actions/variables

- name: Download artifacts / ${{ env.SNAPSHOTS_ARTIFACTS_KEY }}
- name: Download artifacts / ${{ env.CYPRESS_SNAPSHOTS_ARTIFACTS_KEY }}
continue-on-error: true
uses: actions/[email protected]
with:
name: ${{ env.SNAPSHOTS_ARTIFACTS_KEY }}
path: ${{ env.SNAPSHOTS_PATH }}
name: ${{ env.CYPRESS_SNAPSHOTS_ARTIFACTS_KEY }}
path: ./total/cypress

- name: Upload cache / ${{ env.SNAPSHOTS_CACHE_KEY }}
uses: actions/cache/[email protected]
- name: Download artifacts / ${{ env.PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY }}
continue-on-error: true
uses: actions/[email protected]
with:
path: ${{ env.SNAPSHOTS_PATH }}
key: ${{ env.SNAPSHOTS_CACHE_KEY }}
name: ${{ env.PLAYWRIGHT_SNAPSHOTS_ARTIFACTS_KEY }}
path: ./total/playwright

- name: Debug output
continue-on-error: true
run: tree ${{ env.SNAPSHOTS_PATH }}
- name: Check if diff-output exists
id: diff-checker
run: |
echo "diff_exist=$(find ./total -regex '.*diff\.png$' | wc -l | sed -e 's/^[[:space:]]*//')" >> $GITHUB_OUTPUT

- name: Fall with an error if diff-output exists
if: ${{ steps.diff-checker.outputs.diff_exist != '0' }}
run: |
find ./total -regex '.*diff\.png$' -exec echo "{}" \;
exit 1

concurrency:
group: e2e-${{ github.workflow }}-${{ github.ref }}
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,7 @@ dist
RELEASE_BODY.md
*tsbuildinfo
.angular
/projects/demo-playwright/tests-results/
/projects/demo-playwright/tests-report/
/projects/demo-playwright/snapshots/
/projects/demo-playwright/playwright/.cache/
32 changes: 32 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@
"@nx/jest": "16.6.0",
"@nx/node": "16.6.0",
"@nx/workspace": "16.6.0",
"@playwright/test": "^1.36.2",
"@testing-library/cypress": "9.0.0",
"@tinkoff/prettier-config": "1.52.1",
"@tinkoff/stylelint-config": "1.52.1",
Expand Down
39 changes: 39 additions & 0 deletions projects/demo-playwright/playwright.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import {defineConfig, devices} from '@playwright/test';
import {ViewportSize} from 'playwright-core';

const DEFAULT_VIEWPORT: ViewportSize = {width: 700, height: 700};

/**
* See https://playwright.dev/docs/test-configuration.
*/
export default defineConfig({
testDir: __dirname,
testMatch: `**/*.spec.ts`,
outputDir: `tests-results`,
snapshotDir: `snapshots`,
reporter: process.env.CI ? `github` : [[`html`, {outputFolder: `tests-report`}]],
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 0 : 0,
use: {
baseURL: `http://localhost:${process.env.NG_SERVER_PORT || 3333}`,
trace: `on-first-retry`,
viewport: DEFAULT_VIEWPORT,
},
projects: [
{
name: `chromium`,
use: {
...devices[`Desktop Chrome`],
viewport: DEFAULT_VIEWPORT,
},
},
],
expect: {
toHaveScreenshot: {
animations: `disabled`,
caret: `hide`,
},
},
});
33 changes: 33 additions & 0 deletions projects/demo-playwright/project.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"name": "demo-playwright",
"$schema": "../../node_modules/nx/schemas/project-schema.json",
"sourceRoot": "projects/demo-playwright/cypress",
"projectType": "application",
"prefix": "app",
"targets": {
"lint": {
"executor": "nx:run-commands",
"options": {
"command": "eslint --no-error-on-unmatched-pattern \"**/projects/demo-playwright/**/*\""
}
},
"stylelint": {
"executor": "nx:run-commands",
"options": {
"command": "stylelint \"**/demo-playwright/**/*.{css,less}\" --allow-empty-input"
}
},
"e2e": {
"executor": "nx:run-commands",
"options": {
"command": "playwright test --config projects/demo-playwright/playwright.config.ts"
}
},
"e2e-ui": {
"executor": "nx:run-commands",
"options": {
"command": "nx e2e demo-playwright -- --ui"
}
}
}
}
17 changes: 17 additions & 0 deletions projects/demo-playwright/tests/example.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import {expect, test} from '@playwright/test';

test(`has title`, async ({page}) => {
await page.goto(`/getting-started`);

await expect(page).toHaveTitle(`Taiga UI: Getting started`);
});

test(`debug screenshot`, async ({page}) => {
await page.goto(`/components/mobile-calendar`);

await page.locator(`tui-mobile-calendar-example-1 button`).click();

await expect(page.locator(`tui-dialog tui-mobile-calendar`)).toHaveScreenshot(

Check failure on line 14 in projects/demo-playwright/tests/example.spec.ts

View workflow job for this annotation

GitHub Actions / playwright

[chromium] › tests/example.spec.ts:9:5 › debug screenshot

1) [chromium] › tests/example.spec.ts:9:5 › debug screenshot ───────────────────────────────────── Error: Screenshot comparison failed: 4579 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(test-playwright-screenshot.png) with timeout 5000ms - verifying given screenshot expectation - waiting for locator('tui-dialog tui-mobile-calendar') - locator resolved to <tui-mobile-calendar _nghost-tui-demo-c344="" _ngcontent-tui-demo-c34…>…</tui-mobile-calendar> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - 4430 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('tui-dialog tui-mobile-calendar') - locator resolved to <tui-mobile-calendar _nghost-tui-demo-c344="" _ngcontent-tui-demo-c34…>…</tui-mobile-calendar> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - 4319 pixels (ratio 0.01 of all image pixels) are different. - waiting 250ms before taking screenshot - waiting for locator('tui-dialog tui-mobile-calendar') - locator resolved to <tui-mobile-calendar _nghost-tui-demo-c344="" _ngcontent-tui-demo-c34…>…</tui-mobile-calendar> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - 4579 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/taiga-ui/taiga-ui/projects/demo-playwright/tests-results/tests-example-debug-screenshot-chromium/test-playwright-screenshot-expected.png Received: /home/runner/work/taiga-ui/taiga-ui/projects/demo-playwright/tests-results/tests-example-debug-screenshot-chromium/test-playwright-screenshot-actual.png Diff: /home/runner/work/taiga-ui/taiga-ui/projects/demo-playwright/tests-results/tests-example-debug-screenshot-chromium/test-playwright-screenshot-diff.png 12 | await page.locator(`tui-mobile-calendar-example-1 button`).click(); 13 | > 14 | await expect(page.locator(`tui-dialog tui-mobile-calendar`)).toHaveScreenshot( | ^ 15 | `test-playwright-screenshot.png`, 16 | ); 17 | }); at /home/runner/work/taiga-ui/taiga-ui/projects/demo-playwright/tests/example.spec.ts:14:66
`test-playwright-screenshot.png`,
);
});
Loading
Loading