Skip to content

Commit

Permalink
Bring recent updates from develop and configure reforecast (#3015)
Browse files Browse the repository at this point in the history
<!--
  *** PLEASE READ ***

  Any PRs not following this template will be closed.

  Please delete all these comments before submitting the PR.

Please use a short (<60 char), descriptive title for the PR title above.
It should complete the sentence "If merged, this PR will _____".
Capitalize the first word and do not end with a period.

  No content should appear above the "Description" header.

If this PR is not merge-ready (e.g. it depends on other PRs not yet
merged), please mark it as draft until it is ready.

  PRs should meet these guidelines:
  - Each PR should address ONE topic and have an associated issue.
  - No hard-coded paths or personal directories.
  - No temporary or backup files should be committed (including logs).
- Any code that you disabled by being commented out should be removed or
reenabled.
-->
# Description
<!-- This description will become the commit message for the PR. -->
<!--
  Solely pointing to an issue is not an adequate description!

  Please use this format for your description:

Describe your changes. Focus on the *what* and *why*. The *how* will be
evident from the changes. In particular, be sure to note any interface
changes, such as command line syntax, that will need to be communicated
to users.

At the end of your description, please be sure to add the issue this PR
solves using the word "Resolves". If there are any issues that are
related but not yet resolved (including in other repos), you may use
"Refs".

  Resolves #1234
  Refs #4321
  Refs NOAA-EMC/repo#5678
-->

This PR brings recent changes from the develop branch to the GEFS
reforecast branch. This PR updates the GEFS reforecast branch to develop
hash ac3cde5 (10/11/2024). This version
of global-workflow uses the ufs-weather-model hash
[6a4e09e](https://github.com/ufs-community/ufs-weather-model/tree/6a4e09e94773ffa39ce7ab6a54a885efada91f21)
(9/9/2024).

Furthermore, this PR ensures the following adjustments for the
reforecast:

- [x] Speed up rocoto by grouping post job
- [x] Optimize PE configuration
- [x]  Remove duplicate OCNSPPT and EPBL settings
- [x] Set restart_interval to fhmax
- [x] Turn off SHUM in config.efcs
- [x] Set FHMIN_WAV to 3 in config.base 
- [x] Turn off ATM history file output
- [x] Change  HMS=${cyc}0000 to HMS=030000 in Wavepostpnt script (#2788)
- [x] Include YYYYMMDDHH (PDY) in job name
- [x] Change CA seed based on case and cyc for control member and
perturbed members
- [x] Fix post ensemble info
- [x] Add tob to ocean products (#2995 )
- [x] Move PEVPR from b group to a group for atmos products (#2995)
- [x] Add option to download initial condition from HPSS
- [x] Add ability to download and stage replay analysis from AWS, which
is needed for the repair_replay task
- [x] Add capability to run forecasts in 7-day intervals  (#2928)
- [x] Update defaults.yaml so that many of the reforecast-specific
settings can be used by default

<!-- For more on writing good commit messages, see
https://cbea.ms/git-commit/ -->

# Type of change
- [ ] Bug fix (fixes something broken)
- [ ] New feature (adds functionality)
- [x] Maintenance (code refactor, clean-up, new CI test, etc.)

# Change characteristics
<!-- Choose YES or NO from each of the following and delete the other
-->
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO
- Does this change require an update to any of the following submodules?
NO
  - [ ] EMC verif-global <!-- NOAA-EMC/EMC_verif-global#1234 -->
  - [ ] GDAS <!-- NOAA-EMC/GDASApp#1234 -->
  - [ ] GFS-utils <!-- NOAA-EMC/gfs-utils#1234 -->
  - [ ] GSI <!-- NOAA-EMC/GSI#1234 -->
  - [ ] GSI-monitor <!-- NOAA-EMC/GSI-Monitor#1234 -->
  - [ ] GSI-utils <!-- NOAA-EMC/GSI-Utils#1234 -->
  - [ ] UFS-utils <!-- ufs-community/UFS_UTILS#1234 -->
  - [ ] UFS-weather-model <!-- ufs-community/ufs-weather-model#1234 -->
  - [ ] wxflow <!-- NOAA-EMC/wxflow#1234 -->

# How has this been tested?
<!-- Please list any test you conducted, including the machine.

Example:
- Clone and build on WCOSS
- Cycled test on Orion
- Forecast-only on Hera
-->

This branch is being tested on WCOSS2. When testing has succeeded, this
PR will be marked as ready for review.

# Checklist
- [ ] Any dependent changes have been merged and published
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have documented my code, including function, input, and output
descriptions
- [ ] My changes generate no new warnings
- [ ] New and existing tests pass with my changes
- [ ] This change is covered by an existing CI test or a new one has
been added
- [ ] I have made corresponding changes to the system documentation if
necessary

---------

Co-authored-by: Wei Huang <[email protected]>
Co-authored-by: Kate Friedman <[email protected]>
Co-authored-by: Cory Martin <[email protected]>
Co-authored-by: Andrew.Tangborn <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Co-authored-by: AndrewEichmann-NOAA <[email protected]>
Co-authored-by: DavidBurrows-NCO <[email protected]>
Co-authored-by: AnningCheng-NOAA <[email protected]>
Co-authored-by: David Huber <[email protected]>
Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: AntonMFernando-NOAA <[email protected]>
Co-authored-by: BoCui-NOAA <[email protected]>
Co-authored-by: DavidNew-NOAA <[email protected]>
Co-authored-by: Jeffrey Whitaker <[email protected]>
Co-authored-by: mingshichen-noaa <[email protected]>
Co-authored-by: Jiarui Dong <[email protected]>
Co-authored-by: David Huber <[email protected]>
Co-authored-by: Guillaume Vernieres <[email protected]>
Co-authored-by: RussTreadon-NOAA <[email protected]>
Co-authored-by: Innocent Souopgui <[email protected]>
Co-authored-by: Neil Barton <[email protected]>
  • Loading branch information
1 parent abf7467 commit e7b07d5
Show file tree
Hide file tree
Showing 257 changed files with 6,999 additions and 2,508 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/NCO_bug_report.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: NCO Bug report
name: NCO Bug Report
description: Report something that is incorrect or broken
labels: ["nco-bug", "triage"]
assignees:
Expand Down
14 changes: 12 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Bug report
name: Bug Report
description: Report something that is incorrect or broken
labels: ["bug", "triage"]

Expand All @@ -9,6 +9,8 @@ body:
Your bug may already be reported!
Please search on the [Issue tracker](https://github.com/NOAA-EMC/global-workflow/issues) before creating one.
Is this actually a workflow bug? If not, please open an issue in the appropriate repository first.
- type: textarea
id: current_behavior
attributes:
Expand Down Expand Up @@ -46,6 +48,13 @@ body:
validations:
required: true

- type: input
id: hash
attributes:
label: What global-workflow hash are you using?
validations:
required: true

- type: textarea
id: reproduction_steps
attributes:
Expand All @@ -63,8 +72,9 @@ body:
attributes:
label: Additional information
description: Provide context or any additional information about the bug.
placeholder: Optional
validations:
required: true
required: false

- type: textarea
id: propsed_implementation
Expand Down
37 changes: 37 additions & 0 deletions .github/ISSUE_TEMPLATE/dump_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Global Observation Dump Request
description: Request additional dates be added to a machine's global dump archive (GDA) or introduce experimental dump data to the GDA
labels: ["Static Data Mgmt"]
assignees:
- KateFriedman-NOAA
- WalterKolczynski-NOAA

body:
- type: dropdown
attributes:
label: Machine
options:
- WCOSS2
- Hera/Ursa
- Orion/Hercules
- Jet
multiple: true
validations:
required: true

- type: input
attributes:
label: Start date
validations:
required: true

- type: input
attributes:
label: End date
validations:
required: true

- type: textarea
attributes:
label: Additional information
placeholder: |
Any additional information needed (experimental obs, etc.)
17 changes: 13 additions & 4 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Feature request
name: Feature Request
description: Request new capability
labels: ["feature", "triage"]

Expand All @@ -10,8 +10,7 @@ body:
- Something is wrong or broken
- It is a request from NCO
- It is initiating a production update
- Files need to be updated or added to fix
- Files need to be updated in a directory managed by workflow CMs under "glopara"
Please search on the [Issue tracker](https://github.com/NOAA-EMC/global-workflow/issues) to make sure the feature has not already been requested to avoid duplicates.
Expand All @@ -34,7 +33,17 @@ body:
id: criteria
attributes:
label: Acceptance Criteria
placeholder: What does it mean for this issue to be complete?
description: |
A checklist of criteria to be satisfied before this feature is considered complete.
Examples:
- Forecast with option X completes successfully
- File X produced
- Output unchanged
placeholder: |
- [ ] Criterion #1
- [ ] Criterion #2
- [ ] Criterion #3
validations:
required: true

Expand Down
97 changes: 0 additions & 97 deletions .github/ISSUE_TEMPLATE/fix_file.yml

This file was deleted.

29 changes: 15 additions & 14 deletions .github/ISSUE_TEMPLATE/production_update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,19 @@ body:
validations:
required: true

- type: markdown
- type: checkboxes
attributes:
value: |
** Tasks **
- [ ] Create release branch
- [ ] Make workflow changes for upgrade in release branch (add additional checklist items as needed)
- [ ] Create release notes
- [ ] Cut hand-off tag for CDF
- [ ] Submit CDF to NCO
- [ ] Implementation into operations complete
- [ ] Merge release branch into operational branch
- [ ] Cut version tag from operational branch
- [ ] Release new version tag
- [ ] Announce to users
- [ ] Update Read-The-Docs operations status version in develop
label: Tasks
description: List of tasks to complete update. For use after issue is created.
options:
- label: Create release branch
- label: Make workflow changes for upgrade in release branch (add additional checklist items as needed)
- label: Create release notes
- label: Cut hand-off tag for CDF
- label: Submit CDF to NCO
- label: Implementation into operations complete
- label: Merge release branch into operational branch
- label: Cut version tag from operational branch
- label: Release new version tag
- label: Announce to users
- label: Update Read-The-Docs operations status version in develop
134 changes: 134 additions & 0 deletions .github/ISSUE_TEMPLATE/static_data.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
name: Static Data Update
description: Request static data be added or updated
labels: ["Static Data Mgmt"]
assignees:
- KateFriedman-NOAA
- WalterKolczynski-NOAA

body:
- type: dropdown
attributes:
label: Type of static data
description: |
- Fix: large static files needed to run global-workflow
- Initial conditions: ICs needed to run one of global-workflow's included tests
- Experimental data: Temporary datasets needed for development that will be replaced by other sources for operations (e.g. data that will be in DCOM)
options:
- Fix
- Initial conditions
- Experimental data
validations:
required: true

- type: dropdown
attributes:
label: Target directory
options:
- "Fix: aer"
- "Fix: am"
- "Fix: chem"
- "Fix: cice"
- "Fix: cpl"
- "Fix: datm"
- "Fix: gdas/crtm"
- "Fix: gdas/fv3jedi"
- "Fix: gdas/gsibec"
- "Fix: gldas"
- "Fix: glwu"
- "Fix: gsi"
- "Fix: lut"
- "Fix: mom6"
- "Fix: orog"
- "Fix: raw"
- "Fix: reg2grb2"
- "Fix: sfc_climo"
- "Fix: ugwd"
- "Fix: verif"
- "Fix: wave"
- "Fix: *NEW*"
- "ICs: C48/C48 mx500"
- "ICs: C96/C48"
- "ICs: C96 mx100"
- "ICs: C384/C192"
- "ICs: C384 mx025"
- "ICs: C768 mx025"
- "ICs: C1152 mx025"
- "ICs: *NEW RESOLUTION*"
- "Exp Data: GOCART emissions"
- "Exp Data: JEDI obs"
- "Exp Data: *NEW*"
multiple: true
validations:
required: true

- type: checkboxes
attributes:
label: Type of change
options:
- label: Update existing files (Version update needed)
- label: Remove files (Version update needed)
- label: Add new files to existing directory
- label: Add new directory (Version update needed if fix)
validations:
required: true

- type: dropdown
attributes:
label: Any workflow changes needed?
description: |
ANY fix or IC change other than adding files will require at least a new version.
options:
- No change needed
- Fix/IC version update only
- Fix/IC version update + additional changes
- Non-version changes only
validations:
required: true

- type: textarea
attributes:
label: |
Please list all related issues. If this request requires a workflow update (including a fix version update), please open a *separate* global-workflow issue to track the change.
description: Please link any related issues
placeholder: NOAA-EMC/repo#1234
validations:
required: false

- type: input
attributes:
label: Pickup location
description: |
Please provide a path to the data on either Hera or WCOSS.
If this requires a version update, please provide a CLEAN copy of the *entire* directory, including the structure and unchanged files from current directory.
If this is just adding additional files, you may provide just the new files. The directory structure should still match the target.
placeholder: '/path/to/new/data'
validations:
required: true

- type: input
attributes:
label: Size
description: How much data needs to be copied (size of pickup location in MB)?
placeholder: '0'
validations:
required: true

- type: textarea
attributes:
label: Additional information
description: |
Please provide any additional information needed for this request. If this is a new directory, please provide a short description and a point of contact.
validations:
required: false

- type: markdown
attributes:
value: |
# Tasks
- [ ] Discuss needs with global-workflow developer assigned to request
- [ ] Stage a CLEAN copy of data on Hera or WCOSS in the location provided above
- [ ] [global-workflow CM] Update data in "glopara"-managed space on supported platforms
- [ ] [Fix only] Update "Fix File Management" [spreadsheet](https://docs.google.com/spreadsheets/d/1BeIvcz6TO3If4YCqkUK-oz_kGS9q2wTjwLS-BBemSEY/edit?usp=sharing)
- [ ] Make related workflow/component updates (if any)
Loading

0 comments on commit e7b07d5

Please sign in to comment.