Environment for accessing intake python package

[anton-seaice]

Progress :)

Thanks to a new build on vk83 we can now trigger model runs through the CI:

This fails as the service user doesn't access hh5. We currently module use hh5 for an environment to build the intake-esm datastore for model output:

We could:

• put an exception into https://github.com/ACCESS-NRI/model-config-tests/blob/main/src/model_config_tests/models/accessom3.py to skip userscripts
• add access-nri intake catalog python package to the payu conda environment (this might be an intermediate step to adding intake-esm datastore generation to payu)
• build a new environment containing the access-nri intake builders and put it in vk83

Keen to hear your thoughts @aidanheerdegen , @jo-basevi, @CodeGat

[aidanheerdegen]

Yes we need to remove dependencies on hh5 ASAP.

This issue probably belongs on the https://github.com/ACCESS-NRI/access-om3-configs repo (or https://github.com/ACCESS-NRI/model-config-tests). If you agree move/copy/dupe and we can discuss.

[anton-seaice]

We typically collect our OM3 & config related issues here because its hard to know where (and possibly in multiple places) the fix will be applied and to make the issues easier to find.

[anton-seaice]

Related payu-org/payu#521

[anton-seaice]

As an interim patch - I think we can generate the datastore in a postscript instead of a userscript. The postscript starts a new PBS job, so when that fails due to not finding the storage data, it won't impact the CI.

[anton-seaice]

om3-scripts was updated to move the intake datastore generation into a seperate PBS job in COSIMA/om3-scripts#37

these PRs removed the hh5 dependency from the main payu job:

• 1deg_ryf Initial checksums for Repro-CI :) ACCESS-NRI/access-om3-configs#146
• 1deg_iaf Cherry-pick commits from #146 ACCESS-NRI/access-om3-configs#147
• 1deg_ryf_wombatlite Cherry-pick commits from #146 ACCESS-NRI/access-om3-configs#148
• 025deg_ryf Cherry-pick commits from #146 ACCESS-NRI/access-om3-configs#149

and with WW3:

• 1deg_ryf Remove dependency on hh5 in model run  ACCESS-NRI/access-om3-wav-configs#56
• 1deg_iaf Cherry-pick commits from #56 ACCESS-NRI/access-om3-wav-configs#58

[anton-seaice]

Closed through linked PRs