Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor GitHub Actions Workflows for Improved CI/CD Performance #1760

Merged
merged 3 commits into from
Jul 19, 2024

Conversation

MartinMinkov
Copy link
Contributor

@MartinMinkov MartinMinkov commented Jul 16, 2024

Summary

This PR refactors the GitHub Actions workflows to improve the CI/CD pipeline performance for o1js by introducing parallel testing and optimized caching strategies.

All CI tests now take around 1h, down from around 3h, a 3x speed up!

Key Changes:

  1. New Prepare Job:

    • Sets up the repository and Node.js environment.
    • Caches dependencies and build outputs (including all recursive git submodules).
    • Counts unit tests and calculates chunks for parallel execution.
    • Caches the entire repository for reuse.
  2. Updated Build-And-Test-Server Job:

    • Restores repository and dependencies from cache.
    • Runs various integration tests and adds results to the job summary.
  3. New Run-Unit-Tests Job:

    • Executes unit tests in parallel chunks
    • Uploads test results and adds them to the job summary.
  4. Enhanced Build-And-Test-Web Job:

    • Caches Playwright browsers and runs end-to-end tests.
    • Uploads E2E test artifacts.

Benefits:

  1. Faster Builds: Parallel test execution reduces overall testing time.
  2. Efficient Caching: Speeds up jobs by reusing cached dependencies and repository states.
  3. Enhanced Visibility: Detailed job summaries and artifact uploads provide better insights into test results.

Additional Work

  • Each test completes at a different time, some being more computationally expensive. It would be great to spread these out over all chunks so we can avoid hot spots where there is an imbalance of easy/difficult tests for each chunk
  • We could explore removing some proof tests or conditionally turn them on/off depending on changes

@MartinMinkov MartinMinkov force-pushed the feat/speed-up-ci branch 2 times, most recently from 69e71f9 to e371d0a Compare July 16, 2024 22:27
@MartinMinkov MartinMinkov changed the title Test: see if running unit tests in parallel is faster in ci Test: Exploratory Implementation of Parallel Unit Tests in CI Jul 16, 2024
@MartinMinkov MartinMinkov force-pushed the feat/speed-up-ci branch 6 times, most recently from a7c7d95 to 7865905 Compare July 17, 2024 23:06
@MartinMinkov MartinMinkov changed the title Test: Exploratory Implementation of Parallel Unit Tests in CI Add and Refactor GitHub Actions for Unit Tests Execution Jul 17, 2024
@MartinMinkov MartinMinkov changed the title Add and Refactor GitHub Actions for Unit Tests Execution Refactor GitHub Actions Workflows for Improved CI/CD Performance Jul 17, 2024
…ced caching

- Introduced a `Prepare` job to set up the repository, cache dependencies, count unit tests, and calculate chunks for parallel execution.
- Updated `Build-And-Test-Server` job to restore repository and dependencies from cache, run integration tests, and add results to the job summary.
- Added `Run-Unit-Tests` job to execute unit tests in parallel chunks, upload test results, and add them to the job summary.
- Enhanced `Build-And-Test-Web` job to cache Playwright browsers, run end-to-end tests, and upload E2E test artifacts.
- Refined `Release-on-NPM` and `Release-mina-signer-on-NPM` jobs to build and publish the project if the version has changed.
@shimkiv
Copy link
Member

shimkiv commented Jul 18, 2024

For some reason it still waits for status:

image

The unit tests have been removed from the CI test script. This change was likely made because the unit tests are now being run separately or have been moved to a different part of the CI pipeline. Removing them from this script helps to keep the CI process more focused and efficient.
…to run after all unit tests have completed successfully

refactor(build-action.yml): make Build-And-Test-Web job depend on Prepare job instead of directly on unit tests for better separation of concerns and maintainability
Copy link
Member

@shimkiv shimkiv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From PR description:

Enhanced Build-And-Test-Web Job:
...
Uploads E2E test artifacts.

Not a big deal, but I think it was always the case (uploading).

@MartinMinkov MartinMinkov merged commit 9a1d10c into main Jul 19, 2024
22 of 24 checks passed
@MartinMinkov MartinMinkov deleted the feat/speed-up-ci branch July 19, 2024 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants