Add writing out of ensemble mean for soil increments.

[ClaraDraper-NOAA]

Description

The GFSV17 soil analysis requires the ensemble mean increment to be added to the deterministic member. It's simplest to write it out from the EnKF, as the associated snow mask is available. These updates write out the ensemble mean soil increment when a soil update is calculated.

Addresses issue #801

Type of change

Please delete options that are not relevant.

• [ x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Tested that it correctly write out the ensemble mean soil increment for paranc = T and F.

I've run the global regression tests on hera. All passed except:

The runtime for hafs_4denvar_glbens_hiproc_updat is 364.487057 seconds. This has exceeded maximum allowable threshold time of 324.213506 seconds,
resulting in Failure of timethresh2 the regression test.

The above test passed all the reproducability tests.

The regional EnKF test also passed on hera.

Checklist

• [x ] My code follows the style guidelines of this project
• [x ] I have performed a self-review of my own code
• [x ] I have commented my code, particularly in hard-to-understand areas
• [x ] New and existing tests pass with my changes
• [x ] Any dependent changes have been merged and published

[RussTreadon-NOAA]

@ClaraDraper-NOAA , who would you like me to assign as the two peer reviewers for this PR?

[ClaraDraper-NOAA]

@ClaraDraper-NOAA , who would you like me to assign as the two peer reviewers for this PR?

@jswhit and @tsga have both worked on this section of the code. Or Yanjun Gan if one of them can't.

[ClaraDraper-NOAA]

@CoryMartin-NOAA could also review this (@jswhit2 is on travel this week).

[RussTreadon-NOAA]

Step 3 of GSI wiki page GSI: How to Make Changes requires that ctests be run with results reported in either the originating issue or PR.

I can run ctests on the development WCOSS2 once it returns to service.

Can someone run ctests on at least Hera? Running on Orion and Hercules is a good idea since code can behave differently on different machines.

[ClaraDraper-NOAA]

I've run the regression tests on hera. All passed except:

The runtime for hafs_4denvar_glbens_hiproc_updat is 364.487057 seconds. This has exceeded maximum allowable threshold time of 324.213506 seconds, resulting in Failure of timethresh2 the regression test.

The above test passed all the reproducability tests.

@RussTreadon-NOAA I've run the global tests on hera (see the PR description - please let me know if I should report that differently). From the webpage you just linked to it looks like I need to run the RRFS EnKF test as well. I'll do that today. Unfortunately, I don't have access to orion or hercules at the moment.

[RussTreadon-NOAA]

Thank you @ClaraDraper-NOAA . I did not catch the Hera ctest results. My mistake. I'll run on Hercules and Orion.

[CoryMartin-NOAA]

Just a clarifying question, @ClaraDraper-NOAA , so in operations, the ensemble mean increment file that is written out will be a full set of variables, this case where the 3D variables is skipped is only in certain situations? I ask because it seems like this would create a file with a bunch of variables that are all 0s which seems like a waste of disk space.

[ClaraDraper-NOAA]

Thank you @ClaraDraper-NOAA . I did not catch the Hera ctest results. My mistake. I'll run on Hercules and Orion.

the regional test passed. I'll amend the statement above.

[RussTreadon-NOAA]

Hercules ctest

Install ClaraDraper-NOAA:feature/sfc_ensmean at 85d0af2 along with develop at b0e3cba on Hercules. Run ctests with the following results

Test project /work/noaa/da/rtreadon/git/gsi/pr802/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/1 Test #2: rtma .............................   Passed  1568.01 sec
2/6 Test #6: global_enkf ......................***Failed  5053.41 sec
3/6 Test #3: rrfs_3denvar_rdasens .............   Passed  5068.36 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  6978.26 sec
5/6 Test #1: global_4denvar ...................   Passed  7144.54 sec
6/6 Test #4: hafs_4denvar_glbens ..............   Passed  8536.59 sec

83% tests passed, 1 tests failed out of 6

Total Test time (real) = 8536.62 sec

The following tests FAILED:
          6 - global_enkf (Failed)

The global_enkf failure is due to

The runtime for global_enkf_loproc_updat is 134.283243 seconds.  This has exceeded maximum allowable threshold time of 133.025119 seconds, resulting in Failure timethresh of the regression test.

This is not a fatal failure. A check of the enkf.x wall times does not show any anomalous behavior

global_enkf_hiproc_contrl/stdout:The total amount of wall time                        = 111.411896
global_enkf_hiproc_updat/stdout:The total amount of wall time                        = 111.500811
global_enkf_loproc_contrl/stdout:The total amount of wall time                        = 121.588227
global_enkf_loproc_updat/stdout:The total amount of wall time                        = 122.041134

A rerun of global_enkf passed

Test project /work/noaa/da/rtreadon/git/gsi/pr802/build
    Start 6: global_enkf
1/1 Test #6: global_enkf ......................   Passed  3736.79 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 3737.00 sec