Development
: Run tests with submission builds sequentially
#9623
+114
−39
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
General
Client
Motivation and Context
During E2E tests, when multiple exercise submissions trigger simultaneous CI builds, CI servers may get too loaded and builds may take significantly longer than normal to finish. This causes inconsistent test execution times and often results in timeouts. Executing tests that involve submission builds sequentially could solve this issue.
Description
This PR introduces annotations (
@fast
and@slow
) to categorize Playwright e2e tests.Fast Tests (
@fast
): Tests that do not involve exercise submissions triggering CI builds. These are executed in a fully parallel manner to maximize test speed.Slow Tests (
@slow
): Tests that include programming exercise submissions which initiate CI builds. Due to the increased server load these tests create, they are executed sequentially to prevent simultaneous builds.Steps for Testing
Prerequisites:
Exam Mode Testing
Prerequisites:
Testserver States
Note
These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.
Review Progress
Performance Review
Code Review
Manual Tests
Exam Mode Test
Performance Tests
Test Coverage
Screenshots