Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Me 18059 esm tests over preview build #727

Merged
merged 38 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
5e0cc4f
vp test: support esm test running on preview deploy
ShayLevi Oct 27, 2024
a6edc9a
vp test: support esm test running on preview deploy increase timeout
ShayLevi Oct 27, 2024
6409060
vp test: support esm test running on preview deploy revert timeout
ShayLevi Oct 27, 2024
96b8aab
vp test: adding waitFor function for checking if preview URL is ready
ShayLevi Oct 28, 2024
105d995
vp test: adding waitFor function for checking if preview URL is ready
ShayLevi Oct 28, 2024
384f252
vp test: adding waitFor function for checking if preview URL is ready
ShayLevi Oct 28, 2024
a8cf2a3
vp test: modify changes
ShayLevi Oct 28, 2024
4b68609
vp test: modify changes
ShayLevi Oct 28, 2024
ede3ca1
vp test: modify waitForDeployPreviewUrl
ShayLevi Oct 28, 2024
0ac494e
vp test: modify waitForDeployPreviewUrl
ShayLevi Oct 28, 2024
38a291d
vp test: modify changes
ShayLevi Oct 28, 2024
41a9c3f
vp test: modify changes
ShayLevi Oct 28, 2024
e7c3d53
vp test: remove beforeAll
ShayLevi Oct 28, 2024
b73bfe1
vp test: modify waitForDeployPreviewUrl function
ShayLevi Oct 28, 2024
064cd30
vp test: modify waitForDeployPreviewUrl function
ShayLevi Oct 28, 2024
f69490c
vp test: modify waitForDeployPreviewUrl function
ShayLevi Oct 28, 2024
0903c8c
vp test: modify waitForDeployPreviewUrl function adding a print
ShayLevi Oct 28, 2024
a911323
vp test: modify changes
ShayLevi Oct 28, 2024
c3c769f
vp test: adding print to log + increasing timeout
ShayLevi Oct 28, 2024
4475231
vp test: revert timeout
ShayLevi Oct 28, 2024
1687533
vp test: adding timeout
ShayLevi Oct 28, 2024
95b9b42
vp test: adding test for is deploy URL ready
ShayLevi Oct 28, 2024
f8ea78a
vp test: revert test
ShayLevi Oct 28, 2024
21a560e
vp test: change assertion
ShayLevi Oct 28, 2024
34323b9
vp test: change assertion
ShayLevi Oct 28, 2024
ff388d8
vp test: modify waitForDeployPreviewUrl
ShayLevi Oct 29, 2024
744e67e
vp test: modify waitForDeployPreviewUrl assertion
ShayLevi Oct 29, 2024
513c68a
vp test: adding beforeEach and modify waitForDeployPreviewUrl
ShayLevi Oct 29, 2024
c163e38
vp test: modify waitForDeployPreviewUrl
ShayLevi Oct 29, 2024
cc8de1c
vp test: adding logs for debugging
ShayLevi Oct 30, 2024
47e85ad
vp test: adding logs for debugging
ShayLevi Oct 30, 2024
c31ebfb
vp test: adding logs for debugging
ShayLevi Oct 30, 2024
dfdfa21
vp test: adding logs for debugging
ShayLevi Oct 30, 2024
f0e31f9
vp test: adding logs for debugging
ShayLevi Oct 30, 2024
44deeab
vp test: modify waitForDeployPreviewUrl and revert debugging prints
ShayLevi Oct 30, 2024
3fca8c0
vp test: modify waitForDeployPreviewUrl
ShayLevi Oct 30, 2024
b1286a0
vp test: refactor based on review comments
ShayLevi Oct 31, 2024
636a3ca
vp test: remove console log print
ShayLevi Nov 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/e2e_pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,17 @@ jobs:
- name: NPM Install
run: npm install

- name: Extract PR Number
id: extract_pr_number
run: echo "PR_NUMBER=${{ github.event.pull_request.number }}" >> $GITHUB_ENV

- name: Set Deploy Preview URL
run: echo "PREVIEW_URL=https://deploy-preview-${{ env.PR_NUMBER }}--cld-vp-esm-pages.netlify.app" >> $GITHUB_ENV

- name: E2E tests
run: npm run test:e2e
env:
PREVIEW_URL: ${{ env.PREVIEW_URL }}

- name: Upload report to artifact
uses: actions/upload-artifact@v4
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testMatch: /test\/e2e\/specs\/.*(\.spec.ts)/,

timeout: 45000,
timeout: 150000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you have to increase the timeout?
(just curious why you needed this change)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok i see now it is for the deployment preview to be loaded

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The page can take some time to be loaded so the timeout was increased.

/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
Expand Down
30 changes: 25 additions & 5 deletions test/e2e/specs/linksConsoleErrorsEsmPage.spec.ts
Original file line number Diff line number Diff line change
@@ -1,19 +1,26 @@
import { ConsoleMessage, expect } from '@playwright/test';
import { ConsoleMessage, expect, Page } from '@playwright/test';
import { vpTest } from '../fixtures/vpTest';
import { ESM_LINKS } from '../testData/esmPageLinksData';
import { waitForPageToLoadWithTimeout } from '../src/helpers/waitForPageToLoadWithTimeout';
import { validatePageErrors } from '../src/helpers/validatePageErrors';
import { ExampleLinkType } from '../types/exampleLinkType';

const EDGE_ESM_URL = 'https://cld-vp-esm-pages.netlify.app/';
// On PR level it will use the preview deploy URL and locally it will use the latest EDGE.
const ESM_URL = process.env.PREVIEW_URL || EDGE_ESM_URL;
// Flag to indicate if the deploy preview URL is ready
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const ESM_URL = process.env.PREVIEW_URL || EDGE_ESM_URL;
const ESM_URL = process.env.PREVIEW_URL ?? EDGE_ESM_URL;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed. Thanks.

let PREVIEW_URL_LOADED = false;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is a changing variable i think it is better to name it like isPreviewURLLoaded... since a snake case is often associated with constant variables

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed. Thanks

const ESM_URL = 'https://cld-vp-esm-pages.netlify.app/';
/**
* Console error test generated by LINKS object array data.
*/
for (const link of ESM_LINKS) {
vpTest(`Test console errors on link ${link.name}`, async ({ page, consoleErrors, vpExamples }) => {
vpTest.skip(link.name === 'Adaptive Streaming', 'Flaky on CI');
/**
* Navigate to ESM Imports examples page
*/
//Wait for deploy URL to be available if PREVIEW_URL is set, and it is not available yet
if (process.env.PREVIEW_URL && !PREVIEW_URL_LOADED) {
await waitForDeployPreviewUrl(link, page);
}
await page.goto(ESM_URL);
await vpExamples.clickLinkByName(link.name);
await waitForPageToLoadWithTimeout(page, 5000);
Expand Down Expand Up @@ -50,3 +57,16 @@ function handleCommonEsmBrowsersErrors(linkName: string, consoleErrors: ConsoleM
validatePageErrors(consoleErrors, [], ['the server responded with a status of 404']);
}
}

/**
* Waits for a deploy preview URL to become available by making repeated requests and check that link is visible.
*/
async function waitForDeployPreviewUrl(link: ExampleLinkType, page: Page): Promise<void> {
console.log('Waiting for deploy preview to be ready...');
await expect(async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
console.log('Waiting for deploy preview to be ready...');

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The print was added to get indication that we're waiting for the preview and not just wait long time without any indication which can feel like something is stuck.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Print was removed

await page.goto(process.env.PREVIEW_URL);
const linkLocator = page.getByRole('link', { name: link.name, exact: true });
await expect(linkLocator).toBeVisible({ timeout: 10000 });
PREVIEW_URL_LOADED = true;
}).toPass({ intervals: [1_000], timeout: 120000 });
}
Loading