Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Model Repro CI #2

Merged
merged 77 commits into from
Feb 21, 2024
Merged

Model Repro CI #2

merged 77 commits into from
Feb 21, 2024

Commits on Dec 19, 2023

  1. Added config/ and repro-ci.yml: release branch configurations

    @CodeGat
    CodeGat committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    346e865 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2024

  1. README.md: Fixed markdown violations

    @CodeGat
    CodeGat committed Jan 14, 2024
    Configuration menu
    Copy the full SHA
    0812279 View commit details
    Browse the repository at this point in the history

  2. Added test/: Pulled in bitwise repro scripts from cosima/access-om2

    @CodeGat
    CodeGat committed Jan 14, 2024
    Configuration menu
    Copy the full SHA
    7c45198 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2024

  1. Added pr-1-ci.yml for PR-specific CI logic

    @CodeGat
    CodeGat committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    0f9e999 View commit details
    Browse the repository at this point in the history

  2. Added schedule-*.yml for Schedule-specific CI logic

    @CodeGat
    CodeGat committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    73bd123 View commit details
    Browse the repository at this point in the history

  3. repro-ci.yml: Modified into a reusable workflow, added stub steps

    @CodeGat
    CodeGat committed Jan 15, 2024
    Configuration menu
    Copy the full SHA
    ebdff14 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2024

  1. pr-1-ci.yml: Added rough draft of steps for this workflow

    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    9db9f1c View commit details
    Browse the repository at this point in the history

  2. Added bump-1-ci.yml, will be responsible for tagging a new ground tru… 

    …th version in main from the VERSION file
    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    05505c9 View commit details
    Browse the repository at this point in the history

  3. repro-ci.yml: Focus of this workflow is now to generate and upload ch… 

    …ecksums, rather than to compare checksums
    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    5ee7b21 View commit details
    Browse the repository at this point in the history

  4. schedule-2-start.yml: Added a stub check-repro job and stub outcomes … 

    …for that check
    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    344762c View commit details
    Browse the repository at this point in the history

  5. released-configs/json: Added invalid JSON comment on tag formation

    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    bfad339 View commit details
    Browse the repository at this point in the history

  6. bump-1-ci.yml: Added code to add tag to main based on VERSION

    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    98a5422 View commit details
    Browse the repository at this point in the history

  7. Added check-repro.yml, a reusable workflow responsible for comparing … 

    …checksums
    @CodeGat
    CodeGat committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    2f21183 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2024

  1. pr-1-ci.yml: References new check-repro.yml workflow

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    ec8a4e2 View commit details
    Browse the repository at this point in the history

  2. schedule-2-start.yml: References new check-repro.yml workflow

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    bcc8ea4 View commit details
    Browse the repository at this point in the history

  3. repro-ci.yml: Formatted rsync command correctly

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    8d28870 View commit details
    Browse the repository at this point in the history

  4. Updated files with TODOs

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    445d343 View commit details
    Browse the repository at this point in the history

  5. Updated all uses of config-branch to config-tag

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    43a1289 View commit details
    Browse the repository at this point in the history

  6. check-repro.yml: Added logic for checking checksums

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    273182b View commit details
    Browse the repository at this point in the history

  7. Removed some TODOs

    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    5831f78 View commit details
    Browse the repository at this point in the history

  8. pr-1-ci.yml: Added different messages depending on release or dev con… 

    …figs
    @CodeGat
    CodeGat committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    98b8f40 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2024

  1. Converted reusable check-repo.yml into an action to account for diffe… 

    …ring uses across PR and Scheduled checks
    @CodeGat
    CodeGat committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    fc4b7a7 View commit details
    Browse the repository at this point in the history

  2. Moved repro-ci.yml and check-repro action to access-nri/reproducibili… 

    …ty, workflows reference this updated location
    @CodeGat
    CodeGat committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    001ecf1 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2024

  1. Added new workflow to bump VERSION file, renamed bump-1-ci, updated p… 

    …r-1-ci logic
    @CodeGat
    CodeGat committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    6b57523 View commit details
    Browse the repository at this point in the history

  2. Added prevent-merge.yml: This forces a failure of checks when VERSION… 

    … isn't bumped accordingly
    @CodeGat
    CodeGat committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    65c07e1 View commit details
    Browse the repository at this point in the history

  3. schedule-*: Added issue creation as a drastic measure, and made the m… 

    …atrix job not fail fast
    @CodeGat
    CodeGat committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    d01df93 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. schedule-1-ci: Inheritied the appropriate permissions

    @CodeGat
    CodeGat committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    945582b View commit details
    Browse the repository at this point in the history

  2. schedule-2-start: Updated call to checksum-check action with checksum… 

    …-name and checked-against fields
    @CodeGat
    CodeGat committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    edcb680 View commit details
    Browse the repository at this point in the history

  3. pr-1-ci: Added checksum-name field to checksum-check action

    @CodeGat
    CodeGat committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    5eea597 View commit details
    Browse the repository at this point in the history

  4. bump-version-file.yml: Formatting

    @CodeGat
    CodeGat committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    fcfbc0e View commit details
    Browse the repository at this point in the history

  5. pr-1-ci.yml: Added TODOs

    @CodeGat
    CodeGat committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    9abaf14 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2024

  1. Added CONTRIBUTING.md, README-DEV.md and updated README.md to referen… 

    …ce these new documents
    @CodeGat
    CodeGat committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    3ae70bc View commit details
    Browse the repository at this point in the history

  2. schedule-1-ci.yml: Added workflow_dispatch trigger

    @CodeGat
    CodeGat committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    63b6622 View commit details
    Browse the repository at this point in the history

  3. Updated CI to use repro-ci's new experiment-location output

    @CodeGat
    CodeGat committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    ce7b813 View commit details
    Browse the repository at this point in the history

  4. bump-version-file.yml: Updated regex for <type>-<experiment>-<tag> fo… 

    …rmat, added error annotations, added more variable expansion safety
    @CodeGat
    CodeGat committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    26b987d View commit details
    Browse the repository at this point in the history

  5. Add pytests for checking model reproducibility. 

    These are adapted from COSIMA's pytests for access-om2 found here: https://github.com/COSIMA/access-om2/blob/master/test/test_bit_reproducibility.py
so they can be run as part of repro-ci workflows.
    @jo-basevi
    jo-basevi committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    f84ce7b View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2024

  1. Update pytests: Add pytest option --checksum-path for path of checksu… 

    …m to compare against.

The default checksum path is the /testing/checksum/CHECKSUM file stored on each model configuration.
    @jo-basevi
    jo-basevi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    ccd47ed View commit details
    Browse the repository at this point in the history

  2. Add pytest marker for checksum and update test readme

    @jo-basevi
    jo-basevi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    f6dab99 View commit details
    Browse the repository at this point in the history

  3. Fixes to updated reproducibility call, TODOs, PR comments

    @CodeGat
    CodeGat committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    4c7455a View commit details
    Browse the repository at this point in the history

  4. Added validate-json.yml for config validation, updated README-DEV.md … 

    …to reference this new workflow
    @CodeGat
    CodeGat committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    254e5bb View commit details
    Browse the repository at this point in the history

  5. README.md: Fixed formatting errors

    @CodeGat
    CodeGat committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    70ebaf3 View commit details
    Browse the repository at this point in the history

  6. Added comments to all workflows

    @CodeGat
    CodeGat committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    b5383ee View commit details
    Browse the repository at this point in the history

  7. pr-1-ci.yml: Added proper on.paths-ignore, updated checksum-check out… 

    …put name across file
    @CodeGat
    CodeGat committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    7f5a394 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. Appended 'pr-*-' to pr-adjacent workflows

    @CodeGat
    CodeGat committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    c09e0d9 View commit details
    Browse the repository at this point in the history

  2. TODOs and fixes from partial review

    @CodeGat
    CodeGat committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    9b98418 View commit details
    Browse the repository at this point in the history

  3. Refactored workflow to commit and push at the same point, removing th… 

    …e need for merge prevention, in line with @jo-basevi solutions
    @CodeGat
    CodeGat committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    b12a91d View commit details
    Browse the repository at this point in the history

  4. Update tests: Add checksum schema versioning support 

    - Added test for checksum extraction and schema validation
- Added more TODOs for once schema is versioned on a different repo
    @jo-basevi
    jo-basevi committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    4690031 View commit details
    Browse the repository at this point in the history

  5. Add a test output access-om2.out file

    @jo-basevi
    jo-basevi committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    2eb95b9 View commit details
    Browse the repository at this point in the history

  6. Consolidated the commit and push logic into the bump comment workflow

    @CodeGat
    CodeGat committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    e456645 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. Update .github/workflows/pr-1-ci.yml 

    Co-authored-by: Aidan Heerdegen <[email protected]>
    @CodeGat @aidanheerdegen
    CodeGat and aidanheerdegen authored Feb 2, 2024
    Configuration menu
    Copy the full SHA
    d6b9558 View commit details
    Browse the repository at this point in the history

  2. Updated pr-1-ci.yml to reference a section on config tags in the READ… 

    …ME.md
    @CodeGat
    CodeGat committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    ab8af5b View commit details
    Browse the repository at this point in the history

  3. README-DEV.md: Formatting

    @CodeGat
    CodeGat committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    5fa57c4 View commit details
    Browse the repository at this point in the history

  4. Update .github/workflows/pr-2-confirm.yml 

    Co-authored-by: Aidan Heerdegen <[email protected]>
    @CodeGat @aidanheerdegen
    CodeGat and aidanheerdegen authored Feb 2, 2024
    Configuration menu
    Copy the full SHA
    2584d99 View commit details
    Browse the repository at this point in the history

  5. pr-2-confirm.yml: Added comment regarding regex

    @CodeGat
    CodeGat committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    5c76c5f View commit details
    Browse the repository at this point in the history

  6. Update .github/workflows/schedule-2-start.yml 

    Co-authored-by: Aidan Heerdegen <[email protected]>
    @CodeGat @aidanheerdegen
    CodeGat and aidanheerdegen authored Feb 2, 2024
    Configuration menu
    Copy the full SHA
    1705db2 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2024

  1. Removed all references to the VERSION file, instead referencing metad… 

    …ata.yaml or modifications to this file
    @CodeGat
    CodeGat committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    1686a9f View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Apply suggestions from code review 

    Update test/README.md

Co-authored-by: Aidan Heerdegen <[email protected]>
    @jo-basevi @aidanheerdegen
    jo-basevi and aidanheerdegen authored Feb 7, 2024
    Configuration menu
    Copy the full SHA
    3b76634 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. Updated pr-* with fixes from final test

    @CodeGat
    CodeGat committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    52bd32a View commit details
    Browse the repository at this point in the history

  2. pr-1-ci.yml: Changed double to single quotes for comments to allow co… 

    …de formatting
    @CodeGat
    CodeGat committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    450aff3 View commit details
    Browse the repository at this point in the history

  3. pr-2-confirm.yml: changed mv to cp

    @CodeGat
    CodeGat committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    d4ba6f2 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. Update test/test_bit_reproducibility.py

    @CodeGat
    CodeGat authored Feb 14, 2024
    Configuration menu
    Copy the full SHA
    8a1a096 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. exp_test_helper.py: removed __future__ import

    @CodeGat
    CodeGat committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    54f4bfa View commit details
    Browse the repository at this point in the history

  2. accessom2.py: Added examples of checksum regex

    @CodeGat
    CodeGat committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    78cdcc3 View commit details
    Browse the repository at this point in the history

  3. test_bit_reproducibility.py: Renamed output checksum file from `CHECK… 

    …SUM` to `historical-3hr-checksum.yaml`
    @CodeGat
    CodeGat committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    ef9b19e View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2024

  1. removed schema dir, updated models/accessom2 and test/test/test_acces… 

    …s_om2_extract_checksums to use schema
    @CodeGat
    CodeGat committed Feb 18, 2024
    Configuration menu
    Copy the full SHA
    a26f780 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2024

  1. Added requirements.txt for tests

    @CodeGat
    CodeGat committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    961df03 View commit details
    Browse the repository at this point in the history

  2. Updated filenames for test resources and now use get_schema_from_url()

    @CodeGat
    CodeGat committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    399d733 View commit details
    Browse the repository at this point in the history

  3. Updated requirements.txt to have jsonschema

    @CodeGat
    CodeGat committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    dcd42ab View commit details
    Browse the repository at this point in the history

  4. Added new skippable 'test' marker to conftest for test tests

    @CodeGat
    CodeGat committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    7d6943b View commit details
    Browse the repository at this point in the history

  5. Removed skip marker from test_extract_checksums

    @CodeGat
    CodeGat committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    b153021 View commit details
    Browse the repository at this point in the history

  6. Modified path to checksum resource in test test

    @CodeGat
    CodeGat committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    27877fb View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. test_bit_reproducibility: Added a check for checksum_path existence, … 

    …using a default schema instead

Co-authored-by: Aidan Heerdegen <[email protected]>
    @CodeGat @aidanheerdegen
    CodeGat and aidanheerdegen committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    a958592 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. Modified checksum style to be dict[str, any] rather than list[dict[st… 

    …r, any]].
    @CodeGat
    CodeGat committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    933b161 View commit details
    Browse the repository at this point in the history

  2. Changed yaml functions to their appropriate json equivalents, ren… 

    …amed files from `.yaml` to `.json`
    @CodeGat
    CodeGat committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    a9b65f3 View commit details
    Browse the repository at this point in the history

  3. output_checksums are now a defaultdict[str, list[any]]

    @CodeGat
    CodeGat committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    bae7e47 View commit details
    Browse the repository at this point in the history

  4. JSON dumps are now pretty-printed

    @CodeGat
    CodeGat committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    00d6cbc View commit details
    Browse the repository at this point in the history

  5. Added updated test/resources

    @CodeGat
    CodeGat committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    f3810d4 View commit details
    Browse the repository at this point in the history