From e7b07d5191c23232ad2353b702304340c3d6c125 Mon Sep 17 00:00:00 2001 From: Eric Sinsky - NOAA <48259628+EricSinsky-NOAA@users.noreply.github.com> Date: Mon, 28 Oct 2024 13:51:06 -0400 Subject: [PATCH] Bring recent updates from develop and configure reforecast (#3015) # Description This PR brings recent changes from the develop branch to the GEFS reforecast branch. This PR updates the GEFS reforecast branch to develop hash ac3cde510d4aeed632e3b080b6227cb5a26ef6d0 (10/11/2024). This version of global-workflow uses the ufs-weather-model hash [6a4e09e](https://github.com/ufs-community/ufs-weather-model/tree/6a4e09e94773ffa39ce7ab6a54a885efada91f21) (9/9/2024). Furthermore, this PR ensures the following adjustments for the reforecast: - [x] Speed up rocoto by grouping post job - [x] Optimize PE configuration - [x] Remove duplicate OCNSPPT and EPBL settings - [x] Set restart_interval to fhmax - [x] Turn off SHUM in config.efcs - [x] Set FHMIN_WAV to 3 in config.base - [x] Turn off ATM history file output - [x] Change HMS=${cyc}0000 to HMS=030000 in Wavepostpnt script (#2788) - [x] Include YYYYMMDDHH (PDY) in job name - [x] Change CA seed based on case and cyc for control member and perturbed members - [x] Fix post ensemble info - [x] Add tob to ocean products (#2995 ) - [x] Move PEVPR from b group to a group for atmos products (#2995) - [x] Add option to download initial condition from HPSS - [x] Add ability to download and stage replay analysis from AWS, which is needed for the repair_replay task - [x] Add capability to run forecasts in 7-day intervals (#2928) - [x] Update defaults.yaml so that many of the reforecast-specific settings can be used by default # Type of change - [ ] Bug fix (fixes something broken) - [ ] New feature (adds functionality) - [x] Maintenance (code refactor, clean-up, new CI test, etc.) # Change characteristics - Is this a breaking change (a change in existing functionality)? NO - Does this change require a documentation update? NO - Does this change require an update to any of the following submodules? NO - [ ] EMC verif-global - [ ] GDAS - [ ] GFS-utils - [ ] GSI - [ ] GSI-monitor - [ ] GSI-utils - [ ] UFS-utils - [ ] UFS-weather-model - [ ] wxflow # How has this been tested? This branch is being tested on WCOSS2. When testing has succeeded, this PR will be marked as ready for review. # Checklist - [ ] Any dependent changes have been merged and published - [ ] My code follows the style guidelines of this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have documented my code, including function, input, and output descriptions - [ ] My changes generate no new warnings - [ ] New and existing tests pass with my changes - [ ] This change is covered by an existing CI test or a new one has been added - [ ] I have made corresponding changes to the system documentation if necessary --------- Co-authored-by: Wei Huang Co-authored-by: Kate Friedman Co-authored-by: Cory Martin Co-authored-by: Andrew.Tangborn Co-authored-by: Walter Kolczynski - NOAA Co-authored-by: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com> Co-authored-by: DavidBurrows-NCO <82525974+DavidBurrows-NCO@users.noreply.github.com> Co-authored-by: AnningCheng-NOAA <48297505+AnningCheng-NOAA@users.noreply.github.com> Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com> Co-authored-by: Rahul Mahajan Co-authored-by: AntonMFernando-NOAA <167725623+AntonMFernando-NOAA@users.noreply.github.com> Co-authored-by: BoCui-NOAA <53531984+BoCui-NOAA@users.noreply.github.com> Co-authored-by: DavidNew-NOAA <134300700+DavidNew-NOAA@users.noreply.github.com> Co-authored-by: Jeffrey Whitaker Co-authored-by: mingshichen-noaa <48537176+mingshichen-noaa@users.noreply.github.com> Co-authored-by: Jiarui Dong Co-authored-by: David Huber Co-authored-by: Guillaume Vernieres Co-authored-by: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Co-authored-by: Innocent Souopgui <162634017+InnocentSouopgui-NOAA@users.noreply.github.com> Co-authored-by: Neil Barton <103681022+NeilBarton-NOAA@users.noreply.github.com> --- .github/ISSUE_TEMPLATE/NCO_bug_report.yml | 2 +- .github/ISSUE_TEMPLATE/bug_report.yml | 14 +- .github/ISSUE_TEMPLATE/dump_request.yml | 37 + .github/ISSUE_TEMPLATE/feature_request.yml | 17 +- .github/ISSUE_TEMPLATE/fix_file.yml | 97 - .github/ISSUE_TEMPLATE/production_update.yml | 29 +- .github/ISSUE_TEMPLATE/static_data.yml | 134 ++ .github/pull_request_template.md | 63 +- .gitignore | 43 +- ci/cases/gfsv17/C384mx025_3DVarAOWCDA.yaml | 2 +- ci/cases/gfsv17/ocnanal.yaml | 2 +- ci/cases/pr/C48_S2SWA_gefs.yaml | 4 +- ci/cases/pr/C48mx500_3DVarAOWCDA.yaml | 2 +- ci/cases/pr/C96C48_hybatmDA.yaml | 2 +- ci/cases/pr/C96C48_hybatmaerosnowDA.yaml | 2 +- ci/cases/pr/C96C48_ufs_hybatmDA.yaml | 4 +- ci/cases/pr/C96_S2SWA_gefs_replay_ics.yaml | 22 + ci/cases/pr/C96_atm3DVar.yaml | 2 +- ci/cases/pr/C96_atm3DVar_extended.yaml | 2 +- ci/cases/weekly/C384C192_hybatmda.yaml | 2 +- ci/cases/weekly/C384_atm3DVar.yaml | 2 +- ...ci_defaults.yaml => gefs_defaults_ci.yaml} | 0 ci/cases/yamls/gefs_replay_ci.yaml | 14 + docs/note_fixfield.txt | 2 +- docs/source/components.rst | 2 +- docs/source/errors_faq.rst | 9 +- docs/source/index.rst | 5 +- docs/source/init.rst | 4 +- docs/source/jobs.rst | 2 +- docs/source/run.rst | 1 - docs/source/setup.rst | 18 +- env/AWSPW.env | 16 +- env/AZUREPW.env | 52 +- env/CONTAINER.env | 4 +- env/GAEA.env | 51 +- env/GOOGLEPW.env | 56 +- env/HERA.env | 75 +- env/HERCULES.env | 67 +- env/JET.env | 57 +- env/ORION.env | 68 +- env/S4.env | 53 +- env/WCOSS2.env | 67 +- gempak/ush/gfs_meta_comp.sh | 2 +- gempak/ush/gfs_meta_mar_comp.sh | 2 +- jobs/JGDAS_AERO_ANALYSIS_GENERATE_BMATRIX | 46 + jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT | 58 - jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_POST | 49 - jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_VRFY | 2 +- jobs/JGEFS_ATMOS_ACC | 5 +- jobs/JGFS_ATMOS_CYCLONE_GENESIS | 5 - jobs/JGFS_ATMOS_CYCLONE_TRACKER | 5 - jobs/JGFS_ATMOS_VERIFICATION | 6 - jobs/JGLOBAL_AERO_ANALYSIS_FINALIZE | 16 +- jobs/JGLOBAL_AERO_ANALYSIS_INITIALIZE | 10 +- ..._RUN => JGLOBAL_AERO_ANALYSIS_VARIATIONAL} | 4 +- jobs/JGLOBAL_ATMOS_UPP | 7 +- jobs/JGLOBAL_ATMOS_VMINMON | 18 +- ...RUN => JGLOBAL_MARINE_ANALYSIS_CHECKPOINT} | 13 +- jobs/JGLOBAL_MARINE_ANALYSIS_FINALIZE | 43 + ...REP => JGLOBAL_MARINE_ANALYSIS_INITIALIZE} | 36 +- jobs/JGLOBAL_MARINE_ANALYSIS_VARIATIONAL | 38 + jobs/JGLOBAL_MARINE_BMAT | 16 +- jobs/JGLOBAL_STAGE_IC | 12 + jobs/JGLOBAL_WAVE_POST_PNT | 2 +- jobs/rocoto/{ocnanalrun.sh => aeroanlgenb.sh} | 5 +- jobs/rocoto/{aeroanlrun.sh => aeroanlvar.sh} | 4 +- jobs/rocoto/arch_test.sh | 3 - jobs/rocoto/atmos_ensstat.sh | 18 +- jobs/rocoto/atmos_products.sh | 20 +- jobs/rocoto/marineanlchkpt.sh | 18 + .../{ocnanalchkpt.sh => marineanlfinal.sh} | 4 +- .../{ocnanalprep.sh => marineanlinit.sh} | 2 +- .../{ocnanalpost.sh => marineanlvar.sh} | 4 +- jobs/rocoto/oceanice_products.sh | 14 +- modulefiles/module_base.gaea.lua | 9 + modulefiles/module_base.hercules.lua | 6 +- modulefiles/module_base.jet.lua | 4 +- modulefiles/module_base.orion.lua | 6 +- modulefiles/module_base.wcoss2.lua | 3 +- modulefiles/module_gwsetup.jet.lua | 2 +- parm/archive/enkf.yaml.j2 | 22 +- parm/archive/gdas.yaml.j2 | 53 +- parm/archive/gdas_restarta.yaml.j2 | 2 + parm/archive/gefs_arcdir.yaml.j2 | 44 + parm/archive/gefs_extracted_atmos.yaml.j2 | 86 + parm/archive/gefs_extracted_ice.yaml.j2 | 33 + parm/archive/gefs_extracted_ocean.yaml.j2 | 40 + parm/archive/gefs_extracted_wave.yaml.j2 | 51 + .../{arcdir.yaml.j2 => gfs_arcdir.yaml.j2} | 0 parm/archive/gfsa.yaml.j2 | 5 +- parm/archive/master_gefs.yaml.j2 | 12 + parm/config/gefs/config.atmos_ensstat | 3 + parm/config/gefs/config.atmos_products | 9 +- parm/config/gefs/config.base | 59 +- parm/config/gefs/config.cleanup | 1 + parm/config/gefs/config.efcs | 6 +- parm/config/gefs/config.extractvars | 2 +- parm/config/gefs/config.fcst | 2 +- parm/config/gefs/config.oceanice_products | 2 +- parm/config/gefs/config.resources | 20 +- parm/config/gefs/config.resources.AZUREPW | 11 + parm/config/gefs/config.resources.GOOGLEPW | 11 + parm/config/gefs/config.ufs | 51 +- parm/config/gefs/yaml/defaults.yaml | 19 +- parm/config/gfs/config.aero | 2 +- parm/config/gfs/config.aeroanl | 24 +- parm/config/gfs/config.aeroanlgenb | 29 + parm/config/gfs/config.aeroanlrun | 11 - parm/config/gfs/config.aeroanlvar | 11 + parm/config/gfs/config.atmensanl | 6 +- parm/config/gfs/config.base | 22 +- parm/config/gfs/config.com | 1 + parm/config/gfs/config.marineanl | 20 + parm/config/gfs/config.marineanlchkpt | 11 + parm/config/gfs/config.marineanlfinal | 10 + parm/config/gfs/config.marineanlinit | 10 + parm/config/gfs/config.marineanlvar | 11 + parm/config/gfs/config.marinebmat | 8 + parm/config/gfs/config.ocnanal | 20 - parm/config/gfs/config.ocnanalchkpt | 11 - parm/config/gfs/config.ocnanalpost | 10 - parm/config/gfs/config.ocnanalprep | 10 - parm/config/gfs/config.ocnanalrun | 11 - parm/config/gfs/config.prepoceanobs | 2 +- parm/config/gfs/config.prepsnowobs | 3 - parm/config/gfs/config.resources | 111 +- parm/config/gfs/config.resources.GAEA | 6 + parm/config/gfs/config.resources.HERA | 21 +- parm/config/gfs/config.resources.JET | 5 + parm/config/gfs/config.resources.ORION | 25 +- parm/config/gfs/config.resources.S4 | 8 +- parm/config/gfs/config.snowanl | 1 + parm/config/gfs/config.ufs | 22 +- parm/config/gfs/config.wave | 2 +- parm/config/gfs/yaml/defaults.yaml | 7 +- parm/gdas/aero_finalize_bmatrix_bkg.yaml.j2 | 19 + parm/gdas/aero_finalize_variational.yaml.j2 | 24 + parm/gdas/aero_stage_bmatrix_bkg.yaml.j2 | 38 + parm/gdas/aero_stage_variational.yaml.j2 | 50 + parm/post/oceanice_products_gefs.yaml | 2 +- parm/post/upp.yaml | 36 +- .../{bufr_ij9km.txt => bufr_ij_gfs_C1152.txt} | 0 .../{bufr_ij13km.txt => bufr_ij_gfs_C768.txt} | 0 parm/product/bufr_ij_gfs_C96.txt | 2115 +++++++++++++++++ parm/product/gefs.0p25.fFFF.paramlist.a.txt | 4 +- parm/product/gefs.0p25.fFFF.paramlist.b.txt | 60 +- parm/stage/analysis.yaml.j2 | 9 +- ...2 => atmosphere_ens_perturbations.yaml.j2} | 6 +- parm/stage/atmosphere_replay.yaml.j2 | 7 + parm/stage/master_gefs.yaml.j2 | 10 +- parm/stage/ocean.yaml.j2 | 2 +- ...aml.j2 => ocean_ens_perturbations.yaml.j2} | 6 +- parm/ufs/gocart/ExtData.other | 20 +- .../exgdas_aero_analysis_generate_bmatrix.py | 27 + scripts/exgdas_enkf_earc.py | 6 +- scripts/exgfs_aero_init_aerosol.py | 16 +- scripts/exgfs_atmos_postsnd.sh | 140 +- scripts/exgfs_wave_post_pnt.sh | 4 +- ... => exglobal_aero_analysis_variational.py} | 6 +- scripts/exglobal_archive.py | 15 +- scripts/exglobal_atm_analysis_finalize.py | 2 + .../exglobal_atm_analysis_fv3_increment.py | 12 +- scripts/exglobal_atm_analysis_initialize.py | 7 +- scripts/exglobal_atm_analysis_variational.py | 6 +- scripts/exglobal_atmens_analysis_finalize.py | 2 + .../exglobal_atmens_analysis_fv3_increment.py | 12 +- .../exglobal_atmens_analysis_initialize.py | 10 +- scripts/exglobal_atmens_analysis_letkf.py | 10 +- scripts/exglobal_atmens_analysis_obs.py | 7 +- scripts/exglobal_atmens_analysis_sol.py | 8 +- scripts/exglobal_atmos_upp.py | 2 +- scripts/exglobal_cleanup.sh | 25 +- .../exglobal_marine_analysis_checkpoint.py | 29 + scripts/exglobal_marine_analysis_finalize.py | 27 + .../exglobal_marine_analysis_initialize.py | 24 + .../exglobal_marine_analysis_variational.py | 24 + scripts/exglobal_prep_snow_obs.py | 1 - scripts/exglobal_stage_ic.py | 7 +- sorc/build_all.sh | 10 +- sorc/gdas.cd | 2 +- sorc/gfs_utils.fd | 2 +- sorc/gsi_enkf.fd | 2 +- sorc/gsi_monitor.fd | 2 +- sorc/gsi_utils.fd | 2 +- sorc/link_workflow.sh | 26 +- sorc/ufs_model.fd | 2 +- sorc/ufs_utils.fd | 2 +- sorc/verif-global.fd | 2 +- ush/check_ice_netcdf.sh | 43 - ush/detect_machine.sh | 2 +- ush/extractvars_tools.sh | 3 +- ush/forecast_postdet.sh | 30 +- ush/forecast_predet.sh | 53 +- ush/gfs_bufr.sh | 91 +- ush/ocnice_extractvars.sh | 14 +- ush/parsing_model_configure_FV3.sh | 14 +- ush/parsing_namelists_FV3.sh | 18 - ush/parsing_namelists_MOM6.sh | 6 + ush/python/pygfs/__init__.py | 2 + ush/python/pygfs/jedi/__init__.py | 1 + ush/python/pygfs/jedi/jedi.py | 336 +++ ush/python/pygfs/task/aero_analysis.py | 139 +- ush/python/pygfs/task/aero_bmatrix.py | 294 +++ ush/python/pygfs/task/analysis.py | 47 +- ush/python/pygfs/task/archive.py | 5 +- ush/python/pygfs/task/atm_analysis.py | 272 ++- ush/python/pygfs/task/atmens_analysis.py | 287 +-- ush/python/pygfs/task/bmatrix.py | 28 + ush/python/pygfs/task/marine_analysis.py | 483 ++++ ush/python/pygfs/task/marine_bmat.py | 78 +- ush/python/pygfs/task/oceanice_products.py | 16 +- ush/python/pygfs/task/snow_analysis.py | 84 +- ush/python/pygfs/task/stage_ic.py | 25 +- ush/python/pygfs/task/upp.py | 8 +- ush/python/pygfs/utils/marine_da_utils.py | 122 +- versions/build.gaea.ver | 3 +- versions/build.hera.ver | 2 +- versions/build.hercules.ver | 2 +- versions/build.jet.ver | 6 +- versions/build.noaacloud.ver | 2 +- versions/build.orion.ver | 2 +- versions/build.s4.ver | 2 +- versions/build.spack.ver | 27 - versions/fix.ver | 1 + versions/run.gaea.ver | 4 +- versions/run.hera.ver | 2 +- versions/run.hercules.ver | 3 +- versions/run.jet.ver | 10 +- versions/run.noaacloud.ver | 2 +- versions/run.orion.ver | 3 +- versions/run.s4.ver | 2 +- versions/run.wcoss2.ver | 7 +- versions/{run.spack.ver => spack.ver} | 18 +- workflow/applications/applications.py | 132 +- workflow/applications/gefs.py | 18 +- workflow/applications/gfs_cycled.py | 87 +- workflow/applications/gfs_forecast_only.py | 18 +- workflow/hosts.py | 2 +- workflow/hosts/awspw.yaml | 1 + workflow/hosts/azurepw.yaml | 3 +- workflow/hosts/gaea.yaml | 1 + workflow/hosts/googlepw.yaml | 3 +- workflow/hosts/hera.yaml | 1 + workflow/hosts/hercules.yaml | 1 + workflow/hosts/jet.yaml | 23 +- workflow/hosts/orion.yaml | 3 +- workflow/hosts/s4.yaml | 1 + workflow/hosts/wcoss2.yaml | 1 + workflow/rocoto/gefs_tasks.py | 274 ++- workflow/rocoto/gefs_xml.py | 12 +- workflow/rocoto/gfs_cycled_xml.py | 33 +- workflow/rocoto/gfs_forecast_only_xml.py | 37 +- workflow/rocoto/gfs_tasks.py | 649 ++--- workflow/rocoto/rocoto.py | 22 + workflow/rocoto/tasks.py | 26 +- workflow/rocoto/workflow_xml.py | 3 +- workflow/setup_expt.py | 49 +- 257 files changed, 6999 insertions(+), 2508 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/dump_request.yml delete mode 100644 .github/ISSUE_TEMPLATE/fix_file.yml create mode 100644 .github/ISSUE_TEMPLATE/static_data.yml create mode 100644 ci/cases/pr/C96_S2SWA_gefs_replay_ics.yaml rename ci/cases/yamls/{gefs_ci_defaults.yaml => gefs_defaults_ci.yaml} (100%) create mode 100644 ci/cases/yamls/gefs_replay_ci.yaml create mode 100755 jobs/JGDAS_AERO_ANALYSIS_GENERATE_BMATRIX delete mode 100755 jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT delete mode 100755 jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_POST rename jobs/{JGLOBAL_AERO_ANALYSIS_RUN => JGLOBAL_AERO_ANALYSIS_VARIATIONAL} (83%) rename jobs/{JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN => JGLOBAL_MARINE_ANALYSIS_CHECKPOINT} (65%) create mode 100755 jobs/JGLOBAL_MARINE_ANALYSIS_FINALIZE rename jobs/{JGDAS_GLOBAL_OCEAN_ANALYSIS_PREP => JGLOBAL_MARINE_ANALYSIS_INITIALIZE} (60%) create mode 100755 jobs/JGLOBAL_MARINE_ANALYSIS_VARIATIONAL rename jobs/rocoto/{ocnanalrun.sh => aeroanlgenb.sh} (82%) rename jobs/rocoto/{aeroanlrun.sh => aeroanlvar.sh} (83%) delete mode 100755 jobs/rocoto/arch_test.sh create mode 100755 jobs/rocoto/marineanlchkpt.sh rename jobs/rocoto/{ocnanalchkpt.sh => marineanlfinal.sh} (82%) rename jobs/rocoto/{ocnanalprep.sh => marineanlinit.sh} (89%) rename jobs/rocoto/{ocnanalpost.sh => marineanlvar.sh} (82%) create mode 100644 parm/archive/gefs_arcdir.yaml.j2 create mode 100644 parm/archive/gefs_extracted_atmos.yaml.j2 create mode 100644 parm/archive/gefs_extracted_ice.yaml.j2 create mode 100644 parm/archive/gefs_extracted_ocean.yaml.j2 create mode 100644 parm/archive/gefs_extracted_wave.yaml.j2 rename parm/archive/{arcdir.yaml.j2 => gfs_arcdir.yaml.j2} (100%) create mode 100644 parm/archive/master_gefs.yaml.j2 create mode 120000 parm/config/gefs/config.cleanup create mode 100644 parm/config/gefs/config.resources.AZUREPW create mode 100644 parm/config/gefs/config.resources.GOOGLEPW create mode 100644 parm/config/gfs/config.aeroanlgenb delete mode 100644 parm/config/gfs/config.aeroanlrun create mode 100644 parm/config/gfs/config.aeroanlvar create mode 100644 parm/config/gfs/config.marineanl create mode 100644 parm/config/gfs/config.marineanlchkpt create mode 100644 parm/config/gfs/config.marineanlfinal create mode 100644 parm/config/gfs/config.marineanlinit create mode 100644 parm/config/gfs/config.marineanlvar delete mode 100644 parm/config/gfs/config.ocnanal delete mode 100644 parm/config/gfs/config.ocnanalchkpt delete mode 100644 parm/config/gfs/config.ocnanalpost delete mode 100644 parm/config/gfs/config.ocnanalprep delete mode 100644 parm/config/gfs/config.ocnanalrun create mode 100644 parm/gdas/aero_finalize_bmatrix_bkg.yaml.j2 create mode 100644 parm/gdas/aero_finalize_variational.yaml.j2 create mode 100644 parm/gdas/aero_stage_bmatrix_bkg.yaml.j2 create mode 100644 parm/gdas/aero_stage_variational.yaml.j2 rename parm/product/{bufr_ij9km.txt => bufr_ij_gfs_C1152.txt} (100%) rename parm/product/{bufr_ij13km.txt => bufr_ij_gfs_C768.txt} (100%) create mode 100644 parm/product/bufr_ij_gfs_C96.txt rename parm/stage/{atmosphere_perturbation.yaml.j2 => atmosphere_ens_perturbations.yaml.j2} (79%) create mode 100644 parm/stage/atmosphere_replay.yaml.j2 rename parm/stage/{ocean_replay.yaml.j2 => ocean_ens_perturbations.yaml.j2} (79%) create mode 100755 scripts/exgdas_aero_analysis_generate_bmatrix.py rename scripts/{exglobal_aero_analysis_run.py => exglobal_aero_analysis_variational.py} (84%) create mode 100755 scripts/exglobal_marine_analysis_checkpoint.py create mode 100755 scripts/exglobal_marine_analysis_finalize.py create mode 100755 scripts/exglobal_marine_analysis_initialize.py create mode 100755 scripts/exglobal_marine_analysis_variational.py delete mode 100755 ush/check_ice_netcdf.sh create mode 100644 ush/python/pygfs/jedi/__init__.py create mode 100644 ush/python/pygfs/jedi/jedi.py create mode 100644 ush/python/pygfs/task/aero_bmatrix.py create mode 100644 ush/python/pygfs/task/bmatrix.py create mode 100644 ush/python/pygfs/task/marine_analysis.py delete mode 100644 versions/build.spack.ver rename versions/{run.spack.ver => spack.ver} (66%) diff --git a/.github/ISSUE_TEMPLATE/NCO_bug_report.yml b/.github/ISSUE_TEMPLATE/NCO_bug_report.yml index cc53205807..79632779aa 100644 --- a/.github/ISSUE_TEMPLATE/NCO_bug_report.yml +++ b/.github/ISSUE_TEMPLATE/NCO_bug_report.yml @@ -1,4 +1,4 @@ -name: NCO Bug report +name: NCO Bug Report description: Report something that is incorrect or broken labels: ["nco-bug", "triage"] assignees: diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 216293781c..d43d2eb3d7 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -1,4 +1,4 @@ -name: Bug report +name: Bug Report description: Report something that is incorrect or broken labels: ["bug", "triage"] @@ -9,6 +9,8 @@ body: Your bug may already be reported! Please search on the [Issue tracker](https://github.com/NOAA-EMC/global-workflow/issues) before creating one. + Is this actually a workflow bug? If not, please open an issue in the appropriate repository first. + - type: textarea id: current_behavior attributes: @@ -46,6 +48,13 @@ body: validations: required: true + - type: input + id: hash + attributes: + label: What global-workflow hash are you using? + validations: + required: true + - type: textarea id: reproduction_steps attributes: @@ -63,8 +72,9 @@ body: attributes: label: Additional information description: Provide context or any additional information about the bug. + placeholder: Optional validations: - required: true + required: false - type: textarea id: propsed_implementation diff --git a/.github/ISSUE_TEMPLATE/dump_request.yml b/.github/ISSUE_TEMPLATE/dump_request.yml new file mode 100644 index 0000000000..4481e7f6e0 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/dump_request.yml @@ -0,0 +1,37 @@ +name: Global Observation Dump Request +description: Request additional dates be added to a machine's global dump archive (GDA) or introduce experimental dump data to the GDA +labels: ["Static Data Mgmt"] +assignees: + - KateFriedman-NOAA + - WalterKolczynski-NOAA + +body: + - type: dropdown + attributes: + label: Machine + options: + - WCOSS2 + - Hera/Ursa + - Orion/Hercules + - Jet + multiple: true + validations: + required: true + + - type: input + attributes: + label: Start date + validations: + required: true + + - type: input + attributes: + label: End date + validations: + required: true + + - type: textarea + attributes: + label: Additional information + placeholder: | + Any additional information needed (experimental obs, etc.) diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml index 3aed58c520..7e0ddb2459 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.yml +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -1,4 +1,4 @@ -name: Feature request +name: Feature Request description: Request new capability labels: ["feature", "triage"] @@ -10,8 +10,7 @@ body: - Something is wrong or broken - It is a request from NCO - It is initiating a production update - - Files need to be updated or added to fix - + - Files need to be updated in a directory managed by workflow CMs under "glopara" Please search on the [Issue tracker](https://github.com/NOAA-EMC/global-workflow/issues) to make sure the feature has not already been requested to avoid duplicates. @@ -34,7 +33,17 @@ body: id: criteria attributes: label: Acceptance Criteria - placeholder: What does it mean for this issue to be complete? + description: | + A checklist of criteria to be satisfied before this feature is considered complete. + + Examples: + - Forecast with option X completes successfully + - File X produced + - Output unchanged + placeholder: | + - [ ] Criterion #1 + - [ ] Criterion #2 + - [ ] Criterion #3 validations: required: true diff --git a/.github/ISSUE_TEMPLATE/fix_file.yml b/.github/ISSUE_TEMPLATE/fix_file.yml deleted file mode 100644 index 3f5b69cd1d..0000000000 --- a/.github/ISSUE_TEMPLATE/fix_file.yml +++ /dev/null @@ -1,97 +0,0 @@ -name: Fix File Update -description: Request fix files be added or updated -labels: ["Fix Files", "triage"] -assignees: - - KateFriedman-NOAA - - WalterKolczynski-NOAA - -body: - - type: dropdown - attributes: - label: Target fix directory - options: - - NEW - - aer - - am - - chem - - cice - - cpl - - datm - - gdas/crtm - - gdas/fv3jedi - - gdas/gsibec - - gldas - - glwu - - gsi - - lut - - mom6 - - orog - - raw - - reg2grb2 - - sfc_climo - - ugwd - - verif - - wave - multiple: true - validations: - required: true - - - type: checkboxes - attributes: - label: Type of change - options: - - label: Update existing files - - label: Remove files - - label: Add new files to existing directory - - label: Add new fix directory - validations: - required: true - - - type: dropdown - attributes: - label: Any workflow changes needed? - description: | - Any change other than adding files to an existing directory will require at least a new fix version. - options: - - No change needed - - Fix version only - - Fix version + additional changes - validations: - required: true - - - type: textarea - attributes: - label: Related issues - description: Please link any related issues in other repositories - placeholder: NOAA-EMC/repo#1234 - validations: - required: false - - - type: input - attributes: - label: Pickup location - description: | - Please provide a path to the data on either Hera or WCOSS. - - If this requires a version update, please provide the *entire* directory, including unchanged files from current fix. - placeholder: '/path/to/new/fix' - validations: - required: true - - - type: input - attributes: - label: Size - description: How much bigger/smaller is the new set from the existing set (in MB)? - placeholder: '0' - validations: - required: true - - - type: markdown - attributes: - value: | - # Tasks - - [ ] Discuss needs with global-workflow developer assigned to request. - - [ ] Add/update/remove fix file(s) in fix sets on supported platforms (global-workflow assignee task). - - [ ] Update "Fix File Management" spreadsheet (https://docs.google.com/spreadsheets/d/1BeIvcz6TO3If4YCqkUK-oz_kGS9q2wTjwLS-BBemSEY/edit?usp=sharing). - - [ ] Make related workflow/component updates. - diff --git a/.github/ISSUE_TEMPLATE/production_update.yml b/.github/ISSUE_TEMPLATE/production_update.yml index ac9ada505a..cb1fb588d8 100644 --- a/.github/ISSUE_TEMPLATE/production_update.yml +++ b/.github/ISSUE_TEMPLATE/production_update.yml @@ -33,18 +33,19 @@ body: validations: required: true - - type: markdown + - type: checkboxes attributes: - value: | - ** Tasks ** - - [ ] Create release branch - - [ ] Make workflow changes for upgrade in release branch (add additional checklist items as needed) - - [ ] Create release notes - - [ ] Cut hand-off tag for CDF - - [ ] Submit CDF to NCO - - [ ] Implementation into operations complete - - [ ] Merge release branch into operational branch - - [ ] Cut version tag from operational branch - - [ ] Release new version tag - - [ ] Announce to users - - [ ] Update Read-The-Docs operations status version in develop + label: Tasks + description: List of tasks to complete update. For use after issue is created. + options: + - label: Create release branch + - label: Make workflow changes for upgrade in release branch (add additional checklist items as needed) + - label: Create release notes + - label: Cut hand-off tag for CDF + - label: Submit CDF to NCO + - label: Implementation into operations complete + - label: Merge release branch into operational branch + - label: Cut version tag from operational branch + - label: Release new version tag + - label: Announce to users + - label: Update Read-The-Docs operations status version in develop diff --git a/.github/ISSUE_TEMPLATE/static_data.yml b/.github/ISSUE_TEMPLATE/static_data.yml new file mode 100644 index 0000000000..f29f155cf8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/static_data.yml @@ -0,0 +1,134 @@ +name: Static Data Update +description: Request static data be added or updated +labels: ["Static Data Mgmt"] +assignees: + - KateFriedman-NOAA + - WalterKolczynski-NOAA + +body: + - type: dropdown + attributes: + label: Type of static data + description: | + - Fix: large static files needed to run global-workflow + - Initial conditions: ICs needed to run one of global-workflow's included tests + - Experimental data: Temporary datasets needed for development that will be replaced by other sources for operations (e.g. data that will be in DCOM) + options: + - Fix + - Initial conditions + - Experimental data + validations: + required: true + + - type: dropdown + attributes: + label: Target directory + options: + - "Fix: aer" + - "Fix: am" + - "Fix: chem" + - "Fix: cice" + - "Fix: cpl" + - "Fix: datm" + - "Fix: gdas/crtm" + - "Fix: gdas/fv3jedi" + - "Fix: gdas/gsibec" + - "Fix: gldas" + - "Fix: glwu" + - "Fix: gsi" + - "Fix: lut" + - "Fix: mom6" + - "Fix: orog" + - "Fix: raw" + - "Fix: reg2grb2" + - "Fix: sfc_climo" + - "Fix: ugwd" + - "Fix: verif" + - "Fix: wave" + - "Fix: *NEW*" + - "ICs: C48/C48 mx500" + - "ICs: C96/C48" + - "ICs: C96 mx100" + - "ICs: C384/C192" + - "ICs: C384 mx025" + - "ICs: C768 mx025" + - "ICs: C1152 mx025" + - "ICs: *NEW RESOLUTION*" + - "Exp Data: GOCART emissions" + - "Exp Data: JEDI obs" + - "Exp Data: *NEW*" + multiple: true + validations: + required: true + + - type: checkboxes + attributes: + label: Type of change + options: + - label: Update existing files (Version update needed) + - label: Remove files (Version update needed) + - label: Add new files to existing directory + - label: Add new directory (Version update needed if fix) + validations: + required: true + + - type: dropdown + attributes: + label: Any workflow changes needed? + description: | + ANY fix or IC change other than adding files will require at least a new version. + options: + - No change needed + - Fix/IC version update only + - Fix/IC version update + additional changes + - Non-version changes only + validations: + required: true + + - type: textarea + attributes: + label: | + Please list all related issues. If this request requires a workflow update (including a fix version update), please open a *separate* global-workflow issue to track the change. + description: Please link any related issues + placeholder: NOAA-EMC/repo#1234 + validations: + required: false + + - type: input + attributes: + label: Pickup location + description: | + Please provide a path to the data on either Hera or WCOSS. + + If this requires a version update, please provide a CLEAN copy of the *entire* directory, including the structure and unchanged files from current directory. + + If this is just adding additional files, you may provide just the new files. The directory structure should still match the target. + placeholder: '/path/to/new/data' + validations: + required: true + + - type: input + attributes: + label: Size + description: How much data needs to be copied (size of pickup location in MB)? + placeholder: '0' + validations: + required: true + + - type: textarea + attributes: + label: Additional information + description: | + Please provide any additional information needed for this request. If this is a new directory, please provide a short description and a point of contact. + validations: + required: false + + - type: markdown + attributes: + value: | + # Tasks + - [ ] Discuss needs with global-workflow developer assigned to request + - [ ] Stage a CLEAN copy of data on Hera or WCOSS in the location provided above + - [ ] [global-workflow CM] Update data in "glopara"-managed space on supported platforms + - [ ] [Fix only] Update "Fix File Management" [spreadsheet](https://docs.google.com/spreadsheets/d/1BeIvcz6TO3If4YCqkUK-oz_kGS9q2wTjwLS-BBemSEY/edit?usp=sharing) + - [ ] Make related workflow/component updates (if any) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 3f8fe65065..9a1d61eb30 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,19 +1,28 @@ - - # Description - - + + # Type of change - -- Bug fix (fixes something broken) -- New feature (adds functionality) -- Maintenance (code refactor, clean-up, new CI test, etc.) +- [ ] Bug fix (fixes something broken) +- [ ] New feature (adds functionality) +- [ ] Maintenance (code refactor, clean-up, new CI test, etc.) # Change characteristics + - Is this a breaking change (a change in existing functionality)? YES/NO - Does this change require a documentation update? YES/NO - Does this change require an update to any of the following submodules? YES/NO (If YES, please add a link to any PRs that are pending.) - - [ ] EMC verif-global - - [ ] GDAS - - [ ] GFS-utils - - [ ] GSI - - [ ] GSI-monitor - - [ ] GSI-utils - - [ ] UFS-utils - - [ ] UFS-weather-model - - [ ] wxflow - + - [ ] EMC verif-global + - [ ] GDAS + - [ ] GFS-utils + - [ ] GSI + - [ ] GSI-monitor + - [ ] GSI-utils + - [ ] UFS-utils + - [ ] UFS-weather-model + - [ ] wxflow # How has this been tested?