Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature #2924 parse_config PR 2 #2975

Merged
merged 27 commits into from
Sep 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
285eeb2
Per #2924, remove GenEnsProd config file comment about parsing desc s…
JohnHalleyGotway Sep 3, 2024
e9ce34f
Per #2924, update list of needed config entry names
JohnHalleyGotway Sep 5, 2024
c1c4f9b
Per #2924, remove const from the parent() member function so that we …
JohnHalleyGotway Sep 5, 2024
d0ac39a
Per #2924, update the signature for and logic of the utility function…
JohnHalleyGotway Sep 5, 2024
5435d74
Per #2924, update all calls to the climatology utility functions base…
JohnHalleyGotway Sep 5, 2024
f8ddfcc
Merge remote-tracking branch 'origin/develop' into feature_2924_parse…
JohnHalleyGotway Sep 5, 2024
0e21fa6
Per #2924, update the parsing logic for the climatology regrid dictio…
Sep 5, 2024
f256940
Per #2924, forgot to add an else to print an error
Sep 5, 2024
347c644
Per #2924, remove extraneous semicolon
Sep 5, 2024
343f9b3
Per #2924, move 'fcst.regrid' into 'fcst.climo_mean.regrid'. Defining…
JohnHalleyGotway Sep 5, 2024
9cc932b
Merge remote-tracking branch 'origin/develop' into feature_2924_parse…
JohnHalleyGotway Sep 6, 2024
7849d5d
Per #2924, based on PR feedback from @georgemccabe, add the Upper_Lef…
JohnHalleyGotway Sep 12, 2024
6edd245
Merge remote-tracking branch 'origin/develop' into feature_2924_parse…
JohnHalleyGotway Sep 12, 2024
1563d97
Per #2924, update the logic of parse_conf_regrid() to (hopefully) mak…
JohnHalleyGotway Sep 12, 2024
364ec6d
Per #2924, need to check for non-null pointer before using it
Sep 12, 2024
c71c72e
Per #2924, revise the climo_name dictionary lookup logic when parsing…
JohnHalleyGotway Sep 13, 2024
fa719a1
Per #2924, update logic for handling RegridInfo
JohnHalleyGotway Sep 16, 2024
2a68a35
Merge remote-tracking branch 'origin/develop' into feature_2924_parse…
JohnHalleyGotway Sep 17, 2024
d47d4f7
Per #2924, remove the default regridding information from the 'Search…
JohnHalleyGotway Sep 17, 2024
37d3ca5
Merge remote-tracking branch 'origin/develop' into feature_2924_parse…
Sep 18, 2024
372016e
Per #2924, escape sequences, like \n, cannot be used inside R-string …
Sep 18, 2024
ea0c70a
Per #2924, update the logic of check_climo_n_vx()
Sep 18, 2024
227a552
Per #2924, revise logic in read_climo_data_plane_array(). Check the n…
Sep 18, 2024
ecbb7b1
Per #2924, update Series-Analysis to set both i_fcst and i_obs when l…
Sep 18, 2024
cc9c78d
Per #2924, no real change. Just whitespace.
Sep 18, 2024
394fb8d
Unrelated to #2924, superficial changes to formatting of method_name …
Sep 19, 2024
0f722ee
Per #2924, add a new series_analysis test that ERRORS OUT prior to th…
JohnHalleyGotway Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions internal/test_unit/config/SeriesAnalysisConfig_const_climo
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
////////////////////////////////////////////////////////////////////////////////
//
// Series-Analysis configuration file.
//
// For additional information, please see the MET User's Guide.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
model = "GFS";

//
// Output description to be written
//
desc = "NA";

//
// Output observation type to be written
//
obtype = "GFSANL";

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
//
regrid = {
to_grid = NONE;
method = NEAREST;
width = 1;
vld_thresh = 0.5;
}

////////////////////////////////////////////////////////////////////////////////

censor_thresh = [];
censor_val = [];
cat_thresh = [];
cnt_thresh = [ NA ];
cnt_logic = UNION;

//
// Forecast and observation fields to be verified
//
fcst = {
field = [
{ name = "TMP"; level = "P850"; valid_time = "20120409_12"; },
{ name = "TMP"; level = "P850"; valid_time = "20120410_00"; },
{ name = "TMP"; level = "P850"; valid_time = "20120410_12"; }
];
}
obs = {
field = [
{ name = "TMP"; level = "P850"; valid_time = "20120409_12"; },
{ name = "TMP"; level = "P850"; valid_time = "20120410_00"; },
{ name = "TMP"; level = "P850"; valid_time = "20120410_12"; }
];
}

////////////////////////////////////////////////////////////////////////////////

//
// Climatology data
//
climo_mean = fcst;
climo_mean = {

file_name = [ ${CLIMO_MEAN_FILE_LIST} ];

field = [
{ name = "TMP"; level = "P850"; valid_time = "19590409_00"; }
];

regrid = {
method = BILIN;
width = 2;
vld_thresh = 0.5;
}

time_interp_method = NEAREST;
day_interval = NA;
hour_interval = NA;
}

climo_stdev = climo_mean;
climo_stdev = {
file_name = [ ${CLIMO_STDEV_FILE_LIST} ];
}

climo_cdf = {
cdf_bins = 1;
center_bins = FALSE;
direct_prob = FALSE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
//
ci_alpha = [ 0.05 ];

boot = {
interval = PCTILE;
rep_prop = 1.0;
n_rep = 0;
rng = "mt19937";
seed = "1";
}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
//
mask = {
grid = "";
poly = "";
}

//
// Number of grid points to be processed concurrently. Set smaller to use less
// memory but increase the number of passes through the data. If set <= 0, all
// grid points are processed concurrently.
//
block_size = 0;

//
// Ratio of valid matched pairs to compute statistics for a grid point
//
vld_thresh = 0.5;

////////////////////////////////////////////////////////////////////////////////

//
// Statistical output types
//
output_stats = {
fho = [ ];
ctc = [ ];
cts = [ ];
mctc = [ ];
mcts = [ ];
cnt = [ "TOTAL", "RMSE", "ANOM_CORR" ];
sl1l2 = [ ];
sal1l2 = [ ];
pct = [ ];
pstd = [ ];
pjc = [ ];
prc = [ ];
}

////////////////////////////////////////////////////////////////////////////////

hss_ec_value = NA;
rank_corr_flag = FALSE;
tmp_dir = "/tmp";
version = "V12.0.0";

////////////////////////////////////////////////////////////////////////////////
28 changes: 28 additions & 0 deletions internal/test_unit/xml/unit_climatology_1.0deg.xml
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,34 @@
</output>
</test>

<test name="climatology_SERIES_ANALYSIS_1.0DEG_CONST_CLIMO">
<exec>&MET_BIN;/series_analysis</exec>
<env>
<pair><name>CLIMO_MEAN_FILE_LIST</name>
<value>"&DATA_DIR_CLIMO;/NCEP_NCAR_40YR_1.0deg/cmean_1d.19590409"
</value>
</pair>
<pair><name>CLIMO_STDEV_FILE_LIST</name>
<value>"&DATA_DIR_CLIMO;/NCEP_NCAR_40YR_1.0deg/cstdv_1d.19590409"
</value>
</pair>
</env>
<param> \
-fcst &DATA_DIR_MODEL;/grib2/gfs/gfs_2012040900_F012.grib2 \
&DATA_DIR_MODEL;/grib2/gfs/gfs_2012040900_F024.grib2 \
&DATA_DIR_MODEL;/grib2/gfs/gfs_2012040900_F036.grib2 \
-obs &DATA_DIR_MODEL;/grib2/gfsanl/gfsanl_4_20120409_1200_000.grb2 \
&DATA_DIR_MODEL;/grib2/gfsanl/gfsanl_4_20120410_0000_000.grb2 \
&DATA_DIR_MODEL;/grib2/gfsanl/gfsanl_4_20120410_1200_000.grb2 \
-out &OUTPUT_DIR;/climatology_1.0deg/series_analysis_GFS_CLIMO_1.0DEG_CONST_CLIMO.nc \
-config &CONFIG_DIR;/SeriesAnalysisConfig_const_climo \
-v 3
</param>
<output>
<grid_nc>&OUTPUT_DIR;/climatology_1.0deg/series_analysis_GFS_CLIMO_1.0DEG_CONST_CLIMO.nc</grid_nc>
</output>
</test>

<test name="climatology_SERIES_ANALYSIS_1.0DEG_AGGR">
<exec>&MET_BIN;/series_analysis</exec>
<env>
Expand Down
4 changes: 2 additions & 2 deletions src/basic/vx_cal/is_leap_year.cc
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ void adjuste_day_for_month_year_units(int &day, int &month, int &year, double mo
// Compute remaining days from the month fraction
bool day_adjusted = false;
const int day_offset = (int)(month_fraction * DAYS_PER_MONTH + 0.5);
const char *method_name = "adjuste_day() --> ";
const char *method_name = "adjuste_day_for_month_year_units() -> ";

day += day_offset;
if (day == 1 && abs(month_fraction-0.5) < DAY_EPSILON) {
Expand Down Expand Up @@ -162,7 +162,7 @@ unixtime add_to_unixtime(unixtime base_unixtime, int sec_per_unit,
unixtime ut;
auto time_value_ut = (unixtime)time_value;
double time_fraction = time_value - (double)time_value_ut;
const char *method_name = "add_to_unixtime() -->";
const char *method_name = "add_to_unixtime() -> ";

if (sec_per_unit == SEC_MONTH || sec_per_unit == SEC_YEAR) {
if (time_value < 0) {
Expand Down
24 changes: 12 additions & 12 deletions src/basic/vx_config/config_util.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2622,28 +2622,28 @@ void check_mask_names(const StringArray &sa) {

///////////////////////////////////////////////////////////////////////////////

void check_climo_n_vx(Dictionary *dict, const int n_vx) {
int n;
void check_climo_n_vx(Dictionary *dict, const int n_input) {
int n_climo;

// Check for a valid number of climatology mean fields
n = parse_conf_n_vx(dict->lookup_array(conf_key_climo_mean_field, false));
if(n != 0 && n != n_vx) {
n_climo = parse_conf_n_vx(dict->lookup_array(conf_key_climo_mean_field, false));
if(n_climo != 0 && n_climo != 1 && n_climo != n_input) {
mlog << Error << "\ncheck_climo_n_vx() -> "
<< "The number of climatology mean fields in \""
<< conf_key_climo_mean_field
<< "\" must be zero or match the number (" << n_vx
<< ") in \"" << conf_key_fcst_field << "\".\n\n";
<< conf_key_climo_mean_field << "\" (" << n_climo
<< ") must be 0, 1, or match the number of input fields ("
<< n_input << ").\n\n";
exit(1);
}

// Check for a valid number of climatology standard deviation fields
n = parse_conf_n_vx(dict->lookup_array(conf_key_climo_stdev_field, false));
if(n != 0 && n != n_vx) {
n_climo = parse_conf_n_vx(dict->lookup_array(conf_key_climo_stdev_field, false));
if(n_climo != 0 && n_climo != 1 && n_climo != n_input) {
mlog << Error << "\ncheck_climo_n_vx() -> "
<< "The number of climatology standard deviation fields in \""
<< conf_key_climo_stdev_field
<< "\" must be zero or match the number ("
<< n_vx << ") in \"" << conf_key_fcst_field << "\".\n\n";
<< conf_key_climo_stdev_field << "\" (" << n_climo
<< ") must be 0, 1, or match the number of input fields ("
<< n_input << ").\n\n";
exit(1);
}

Expand Down
2 changes: 1 addition & 1 deletion src/basic/vx_config/threshold.cc
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ if ( !match &&

mlog << Debug(2) << R"(Please replace the deprecated "SCP" and "CDP" )"
<< R"(threshold types with "SOCP" and "OCDP", respectively, in the ")"
<< str << R"(" threshold string.\n)";
<< str << R"(" threshold string.)" << "\n";

print_climo_perc_thresh_log_message = false;

Expand Down
2 changes: 1 addition & 1 deletion src/libcode/vx_data2d_nc_cf/nc_cf_file.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2299,7 +2299,7 @@ void NcCfFile::get_grid_mapping_polar_stereographic(const NcVar *grid_mapping_va
{
double x_coord_to_m_cf = 1.0;
double y_coord_to_m_cf = 1.0;
static const string method_name = "NcCfFile::get_grid_mapping_polar_stereographic() --> ";
static const string method_name = "NcCfFile::get_grid_mapping_polar_stereographic() -> ";

// Get projection attributes
// proj_origin_lat: either 90.0 or -90.0, to decide the northern/southern hemisphere
Expand Down
10 changes: 8 additions & 2 deletions src/libcode/vx_statistics/read_climo.cc
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,14 @@ DataPlaneArray read_climo_data_plane_array(Dictionary *dict,
cs << cs_erase << climo_name << "." << conf_key_field;
Dictionary *field_dict = dict->lookup_array(cs.c_str(), false);

// Get the i-th array entry
Dictionary i_dict = parse_conf_i_vx_dict(field_dict, i_vx);
// Determine which climo array entry to use
int i_climo_field = bad_data_int;
if(field_dict->n_entries() == 0) return dpa;
else if(field_dict->n_entries() == 1) i_climo_field = 0;
else i_climo_field = i_vx;

// Parse the climo dictionary
Dictionary i_dict = parse_conf_i_vx_dict(field_dict, i_climo_field);

// Parse the "regrid" dictionary from the top-level
// config file context (e.g. "config.climo_mean.regrid")
Expand Down
13 changes: 8 additions & 5 deletions src/tools/core/series_analysis/series_analysis.cc
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ void process_command_line(int argc, char **argv) {
if(fcst_files.n() != n_series_pair) {
mlog << Error << "\nprocess_command_line() -> "
<< R"(when using the "-paired" command line option, the )"
<< "the file list length (" << fcst_files.n()
<< "file list length (" << fcst_files.n()
<< ") and series length (" << n_series_pair
<< ") must match.\n\n";
usage();
Expand Down Expand Up @@ -861,11 +861,14 @@ void process_scores() {
// Loop over the series variable
for(int i_series=0; i_series<n_series_pair; i_series++) {

// Get the index for the forecast and climo VarInfo objects
// Get the index for the VarInfo objects
int i_fcst = (conf_info.get_n_fcst() > 1 ? i_series : 0);
int i_obs = (conf_info.get_n_obs() > 1 ? i_series : 0);

// Store the current VarInfo objects
fcst_info = conf_info.fcst_info[i_fcst];
fcst_info = (conf_info.get_n_fcst() > 1 ?
conf_info.fcst_info[i_series] :
conf_info.fcst_info[0]);
obs_info = (conf_info.get_n_obs() > 1 ?
conf_info.obs_info[i_series] :
conf_info.obs_info[0]);
Expand Down Expand Up @@ -914,12 +917,12 @@ void process_scores() {
ocmn_dp = read_climo_data_plane(
conf_info.conf.lookup_dictionary(conf_key_obs),
conf_key_climo_mean,
i_fcst, fcst_dp.valid(), grid,
i_obs, fcst_dp.valid(), grid,
"observation climatology mean");
ocsd_dp = read_climo_data_plane(
conf_info.conf.lookup_dictionary(conf_key_obs),
conf_key_climo_stdev,
i_fcst, fcst_dp.valid(), grid,
i_obs, fcst_dp.valid(), grid,
"observation climatology standard deviation");

bool fcmn_flag = !fcmn_dp.is_empty();
Expand Down
Loading