Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update snow obs processing job #3055

Open
wants to merge 32 commits into
base: develop
Choose a base branch
from
Open

Update snow obs processing job #3055

wants to merge 32 commits into from

Conversation

jiaruidong2017
Copy link
Contributor

@jiaruidong2017 jiaruidong2017 commented Nov 1, 2024 •
edited
Loading

Description

This PR removes the prepsnowobs job, and add the IMS snow obs processing to the snow analysis job and it will only process the snow cover obs if the cycle is 00z.

Resolves (#2902)

Type of change

  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • Cycled test on Hera

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

jiaruidong2017 and others added 30 commits September 9, 2024 10:58
@jiaruidong2017
Remove the GTS BUFR2IODA part of the snow obs prep job and
4098f10 
Replace it with a direct read from BUFR at runtime.
@jiaruidong2017
Change HOMEgfs to PARMgfs.
8d1bc87
@jiaruidong2017
Merge branch 'develop' into feature/snow_gtsobs
6ff65bc
@CoryMartin-NOAA
Merge branch 'develop' into feature/snow_gtsobs
2295cab
@jiaruidong2017
Made changes to stage the static GTS mapping files.
5074d11
@jiaruidong2017
Change the name to be more descriptive.
f122edf
@jiaruidong2017
Merge branch 'develop' into feature/snow_gtsobs
73b4ec0
@jiaruidong2017
Update IMS preprocessing job.
086158d
@jiaruidong2017
Rename IMS snow preprocessing jobs/names to be snowcover
2df23be
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
bb8775a
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
5ddf552
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
0bfbff7
@jiaruidong2017
Address reviewer's comments.
3c08705
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
850f672
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
6f2fc66
@jiaruidong2017
Merge branch 'feature/ims_snowcover' of https://github.com/jiaruidong…
30c19d4 
…2017/global-workflow into feature/ims_snowcover
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
08bb8d8
@jiaruidong2017
Merge branch 'develop' into feature/ims_snowcover
8a4a743
@jiaruidong2017
Update to use <taskvalid> for managing the rocoto dependency.
b12a375
@jiaruidong2017
Fix the pynorms error.
679e57e
@jiaruidong2017
Fix an error.
d97480a
@jiaruidong2017
Update the name of dependency from prepsnowcover for ens recentering.
ed78b8c
@jiaruidong2017
Update to run gdas_prepsnowcover job on every 00Z cycle.
eac7e1f
@jiaruidong2017
Made changes as reviewer suggested.
96254c4
@jiaruidong2017
Fix pynorm error.
a00f9e8
@DavidHuber-NOAA
Merge branch 'develop' into feature/ims_snowcover
20b3b9c
@jiaruidong2017
Update the dependency for the enkfgdas_esnowrecen job.
db3a912
@jiaruidong2017
Merge branch 'feature/ims_snowcover' of https://github.com/jiaruidong…
2e319a7 
…2017/global-workflow into feature/ims_snowcover
@jiaruidong2017
Update the esnowrecen job dependency.
3d8c9d3
@jiaruidong2017
Update the gfs workflow for prepsnowcover job.
7148c4e
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Nov 1, 2024
@emcbot
Copy link

emcbot commented Nov 2, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 1 with error logs:

/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/enkfgdas_earc01.log

Follow link here to view the contents of the above file(s): (link)

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Nov 2, 2024
@emcbot
Copy link

emcbot commented Nov 2, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/EXPDIR/C96C48_hybatmaerosnowDA_278c5bb4

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Nov 2, 2024
@emcbot
Copy link

emcbot commented Nov 2, 2024

CI Failed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/3055


Experiment C48_ATM_278c5bb4 Completed 2 Cycles: *SUCCESS* at Sat Nov  2 02:30:19 UTC 2024
Experiment C96_S2SWA_gefs_replay_ics_278c5bb4 Completed 1 Cycles: *SUCCESS* at Sat Nov  2 02:42:36 UTC 2024
Experiment C48mx500_3DVarAOWCDA_278c5bb4 Completed 2 Cycles: *SUCCESS* at Sat Nov  2 02:54:46 UTC 2024
Experiment C96C48_hybatmaerosnowDA_278c5bb4 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Nov  2 03:00:55 UTC 2024
Experiment C96C48_hybatmaerosnowDA_278c5bb4 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/enkfgdas_earc01.log
Experiment C48_S2SWA_gefs_278c5bb4 Completed 1 Cycles: *SUCCESS* at Sat Nov  2 04:02:13 UTC 2024
Experiment C96_atm3DVar_278c5bb4 Completed 3 Cycles: *SUCCESS* at Sat Nov  2 04:26:20 UTC 2024
Experiment C96C48_hybatmDA_278c5bb4 Completed 3 Cycles: *SUCCESS* at Sat Nov  2 04:26:20 UTC 2024
Experiment C48_S2SW_278c5bb4 Completed 2 Cycles: *SUCCESS* at Sat Nov  2 04:32:25 UTC 2024
Experiment C96C48_ufs_hybatmDA_278c5bb4 Completed 3 Cycles: *SUCCESS* at Sat Nov  2 04:50:46 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Nov 2, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Nov 2, 2024
@emcbot
Copy link

emcbot commented Nov 2, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 2 with error logs:

/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f000.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f003.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f006.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f009.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f012.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f015.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f018.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f021.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f024.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f027.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f030.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f033.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f036.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f039.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f042.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f045.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f048.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f051.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f054.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f057.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f060.log

Follow link here to view the contents of the above file(s): (link)

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Nov 2, 2024
@emcbot
Copy link

emcbot commented Nov 2, 2024

Experiment C96C48_hybatmaerosnowDA FAILED on Hera in Build# 2 in
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/EXPDIR/C96C48_hybatmaerosnowDA_278c5bb4

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Nov 2, 2024
@emcbot
Copy link

emcbot commented Nov 2, 2024

CI Failed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/3055


Experiment C96_S2SWA_gefs_replay_ics_278c5bb4 Completed 1 Cycles: *SUCCESS* at Sat Nov  2 10:34:43 UTC 2024
Experiment C48_ATM_278c5bb4 Completed 2 Cycles: *SUCCESS* at Sat Nov  2 10:40:53 UTC 2024
Experiment C48mx500_3DVarAOWCDA_278c5bb4 Completed 2 Cycles: *SUCCESS* at Sat Nov  2 10:47:02 UTC 2024
Experiment C96C48_hybatmaerosnowDA_278c5bb4 Terminated with 0
FAIL
FAIL tasks failed and 21 dead at Sat Nov  2 10:53:35 UTC 2024
Experiment C96C48_hybatmaerosnowDA_278c5bb4 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f000.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f003.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f006.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f009.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f012.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f015.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f018.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f021.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f024.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f027.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f030.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f033.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f036.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f039.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f042.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f045.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f048.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f051.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f054.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f057.log
/scratch1/NCEPDEV/global/CI/3055/RUNTESTS/COMROOT/C96C48_hybatmaerosnowDA_278c5bb4/logs/2021122018/gfs_atmos_prod_f060.log
Experiment C48_S2SWA_gefs_278c5bb4 Completed 1 Cycles: *SUCCESS* at Sat Nov  2 11:43:16 UTC 2024
Experiment C96_atm3DVar_278c5bb4 Completed 3 Cycles: *SUCCESS* at Sat Nov  2 12:06:57 UTC 2024
Experiment C96C48_hybatmDA_278c5bb4 Completed 3 Cycles: *SUCCESS* at Sat Nov  2 12:06:58 UTC 2024
Experiment C48_S2SW_278c5bb4 Completed 2 Cycles: *SUCCESS* at Sat Nov  2 12:25:36 UTC 2024
Experiment C96C48_ufs_hybatmDA_278c5bb4 Completed 3 Cycles: *SUCCESS* at Sat Nov  2 12:43:56 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules labels Nov 4, 2024
@jiaruidong2017
Copy link
Contributor Author

@WalterKolczynski-NOAA It seems to me that the failure is unrelated to this PR. Any suggestions? Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DavidHuber-NOAA DavidHuber-NOAA DavidHuber-NOAA approved these changes

@KateFriedman-NOAA KateFriedman-NOAA KateFriedman-NOAA approved these changes

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

@CoryMartin-NOAA CoryMartin-NOAA CoryMartin-NOAA approved these changes

@aerorahul aerorahul Awaiting requested review from aerorahul aerorahul is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jiaruidong2017 @emcbot @CoryMartin-NOAA @WalterKolczynski-NOAA @KateFriedman-NOAA @DavidHuber-NOAA