From 542b4ba9a1db69d445ecb3eb023d4c60cad291ea Mon Sep 17 00:00:00 2001 From: Dan Adriaansen Date: Tue, 7 May 2024 08:42:48 -0600 Subject: [PATCH] Feature 2748 document ugrid (#2869) * Initial documentation of the UGRID capability. * Fixes error in references, adds appendix to index, and adds sub-section for configuration entries and a table for metadata map items. * Corrects LFRic, rewords section on UGRID conventions, updates description of using GridStat, and removes mention of nodes. * Forgot one more mention of UGRID conventions. * Incorporates more suggestions from @willmayfield. * Switches to numerical table reference. --- docs/Users_Guide/appendixH.rst | 87 +++++++++++++++++++++++++++++ docs/Users_Guide/config_options.rst | 2 + docs/Users_Guide/grid-stat.rst | 11 ++-- docs/Users_Guide/index.rst | 1 + docs/Users_Guide/point-stat.rst | 15 +++-- docs/Users_Guide/refs.rst | 16 ++++++ 6 files changed, 122 insertions(+), 10 deletions(-) create mode 100644 docs/Users_Guide/appendixH.rst diff --git a/docs/Users_Guide/appendixH.rst b/docs/Users_Guide/appendixH.rst new file mode 100644 index 0000000000..3fff53218a --- /dev/null +++ b/docs/Users_Guide/appendixH.rst @@ -0,0 +1,87 @@ +.. _appendixH: + +***************************** +Appendix H Unstructured Grids +***************************** + +Introduction +============ + +METv12.0.0+ includes limited support for using datasets on unstructured grids (UGRIDs) in the PointStat and GridStat tools. This support includes verifying UGRID forecasts against point observations (PointStat), and verifying UGRID forecasts regridded to a structured grid against gridded observations on a structured grid (GridStat). The implementation of UGRID support in METv12.0.0+ provides a mechanism for a user to describe the topology of their unstructured grid. Thus far, development to support datasets on unstructured grids has been driven by the LFRic (“elfrick”) NWP model (:ref:`Adams et al. 2019 `), and the Model for Prediction Across Scales (MPAS) NWP model (:ref:`Skamarock et al., 2012 `). However, the support for unstructured grids was implemented in such a way that additional unstructured grids can be utilized, with the appropriate mapping of the user's topology to the elements that MET requires. + +Unstructured Grid Files +======================= + +Support for UGRID files in NetCDF format is provided. MET will attempt to auto-detect UGRID files. If the file is CF-compliant and the “Conventions” global attribute is “UGRID” or “MPAS”, or if the global attribute “mesh_spec” is present, MET recognizes the file as a UGRID file. If the user is not using a supported UGRID, or a UGRID file that does not contain the required global attributes for autodetection, the user should set the **file_type** configuration entry to **NETCDF_UGRID**. In some cases, the UGRID topology is provided in an ancillary file separate from forecast variables. This is controlled via the **ugrid_coordinates_file** configuration entry. This file is checked first, and then the UGRID data file. If the same elements are found in both files, the information from the data file takes precedence, and thus it is recommended to leave **ugrid_coordinates_file** unset if all required elements are found in the data file. If both a data and coordinates file are provided, the shape of the latitude and longitude dimensions are cross-referenced as a limited enforcement measure that the coordinates file and data file belong to the same UGRID. + +Required Unstructured Grid Metadata +=================================== + +To correctly parse the UGRID topology, MET needs the following information that must be contained within the UGRID coordinates file or the UGRID data file. It is often the case that the variable names for these elements differ between UGRIDs. If the user is not using one of the supported UGRIDs, they will need to define the **ugrid_metadata_map** in a custom configuration file that they provide on the command line using the **-ugrid_config** command line argument, which tells MET the variable names that correspond to the following elements: + +.. _table_ugrid_metadata: + +.. list-table:: Required UGRID Metadata + :widths: auto + :header-rows: 1 + + * - Metadata Map Key + - Description + - Required + * - dim_face + - Dimension name for UGRID cells + - Required + * - lat_face + - Coordinate variable name for the latitude of each UGRID cell + - Required + * - lon_face + - Coordinate variable name for the longitude of each UGRID cell + - Required + * - dim_time + - Dimension name for the time coordinate + - Required + * - time + - Coordinate variable for time + - Required + * - dim_vert + - Dimension name for the vertical coordinate, if present + - Optional + * - vert_face + - Coordinate variable name for the vertical coordinate + - Optional + +Unstructured Grid Configuration Entries +======================================= + +In the PointStat and GridStat config files, a user can control aspects of the UGRID capability. Since the UGRID support in MET is optional, these items are not included in the default MET config files. If the user requires modification for any of the following, they must add them to their MET config file. + +ugrid_dataset +------------- + +If the UGRID dataset is supported by MET, then this item is set to the string identifying the UGRID. Currently supported ugrid_dataset options include “lfric”, and “mpas”. If this item is not set, the user must provide a separate UGRID configuration file on the command line using the **-ugrid_config** command line argument. + +ugrid_max_distance_km +--------------------- + +For PointStat, this is the distance from each UGRID cell center that PointStat will search outward to collect observations to use for verification. The default is 0 km, which by default will use the closest observation to the UGRID cell for verification. For GridStat, this is the distance from each forecast grid point to search for observation grid cells to include when interpolating to the forecast grid point locations. Currently, only the nearest point observation or gridded observation cell is used. See `Unstructured Grid Limitations`_ for additional details about interpolation when using GridStat. + +ugrid_coordinates_file +---------------------- + +The absolute path to the ancillary NetCDF file that describes the UGRID topology. This file is checked for the required UGRID metadata first, and the data file is checked second and takes precedence over this file if the same elements are found in both files. + +ugrid_metadata_map +------------------ + +The mapping dictionary which allows a user to specify the variable names of the required UGRID topology elements that are required by MET. For "lfric" and "mpas", **ugrid_metadata_map** comes pre-configured with MET and a user can simply set **ugrid_dataset**. Otherwise, the user must create a separate UGRID configuration file containing the **ugrid_metadata_map** containing the elements from :numref:`table_ugrid_metadata` and provide it on the command line using the **-ugrid_config** command line argument. + +Unstructured Grid Limitations +============================= + +We anticipate expanding the UGRID capabilities in MET in the near future. Until then, users are encouraged to be mindful of the following limitations: + +1. In GridStat, there is no support for verifying directly on a UGRID. The UGRID (either forecast, obs, or both) must be regridded to a structured verification grid prior to verification being performed. In most cases, gridded observations are on a structured grid and so the users are encouraged to regrid their UGRID forecast to the structured observation grid. If the user wishes to verify a UGRID observation against a UGRID forecast (for example, a UGRID model analysis), the user must define a custom verification grid tailored to their UGRID characteristics, being mindful that the regridding can be slow for large verification grids. The user is referred to :ref:`appendixB` for guidance on defining a structured verification grid. + +2. Data at cell edges are currently not supported, only those variables which have data at the cell centers are supported. Users should note in particular that wind components that are typically derived using data at cell edges are currently unsupported. + +3. No aggregation methods of point observations within the **ugrid_max_distance_km** are supported except NEAREST, and no aggregation methods of gridded observations within the **ugrid_max_distance_km** are supported except NEAREST. diff --git a/docs/Users_Guide/config_options.rst b/docs/Users_Guide/config_options.rst index ec76ecf7d5..fb6e4f47eb 100644 --- a/docs/Users_Guide/config_options.rst +++ b/docs/Users_Guide/config_options.rst @@ -1047,6 +1047,8 @@ to be verified. This dictionary may include the following entries: single model level. file_type = NETCDF_NCCF; NetCDF following the Climate Forecast (CF) convention. + file_type = NETCDF_UGRID; NetCDF containing data on an + unstructured grid. file_type = PYTHON_NUMPY; Run a Python script to load data into a NumPy array. file_type = PYTHON_XARRAY; Run a Python script to load data into diff --git a/docs/Users_Guide/grid-stat.rst b/docs/Users_Guide/grid-stat.rst index 22fe740a55..c09df58d54 100644 --- a/docs/Users_Guide/grid-stat.rst +++ b/docs/Users_Guide/grid-stat.rst @@ -172,6 +172,7 @@ The usage statement for the Grid-Stat tool is listed below: fcst_file obs_file config_file + [-ugrid_config config_file] [-outdir path] [-log file] [-v level] @@ -191,13 +192,15 @@ Required Arguments for grid_stat Optional Arguments for grid_stat ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -4. The **-outdir path** indicates the directory where output files should be written. +4. The **-ugrid_config** option provides a way for a user to provide a separate config file with metadata about their UGRID. -5. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file. +5. The **-outdir path** indicates the directory where output files should be written. -6. The **-v level** option indicates the desired level of verbosity. The contents of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity above 1 will increase the amount of logging. +6. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file. -7. The **-compress level** option indicates the desired level of compression (deflate level) for NetCDF variables. The valid level is between 0 and 9. The value of "level" will override the default setting of 0 from the configuration file or the environment variable MET_NC_COMPRESS. Setting the compression level to 0 will make no compression for the NetCDF output. Lower number is for fast compression and higher number is for better compression. +7. The **-v level** option indicates the desired level of verbosity. The contents of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity above 1 will increase the amount of logging. + +8. The **-compress level** option indicates the desired level of compression (deflate level) for NetCDF variables. The valid level is between 0 and 9. The value of "level" will override the default setting of 0 from the configuration file or the environment variable MET_NC_COMPRESS. Setting the compression level to 0 will make no compression for the NetCDF output. Lower number is for fast compression and higher number is for better compression. An example of the grid_stat calling sequence is listed below: diff --git a/docs/Users_Guide/index.rst b/docs/Users_Guide/index.rst index ee3a5ff6dc..1df5c65e53 100644 --- a/docs/Users_Guide/index.rst +++ b/docs/Users_Guide/index.rst @@ -81,6 +81,7 @@ The National Center for Atmospheric Research (NCAR) is sponsored by NSF. The DTC appendixE appendixF appendixG + appendixH .. only:: html diff --git a/docs/Users_Guide/point-stat.rst b/docs/Users_Guide/point-stat.rst index d5f895a8aa..6c32537146 100644 --- a/docs/Users_Guide/point-stat.rst +++ b/docs/Users_Guide/point-stat.rst @@ -277,6 +277,7 @@ The usage statement for the Point-Stat tool is shown below: fcst_file obs_file config_file + [-ugrid_config config_file] [-point_obs file] [-obs_valid_beg time] [-obs_valid_end time] @@ -298,17 +299,19 @@ Required Arguments for point_stat Optional Arguments for point_stat ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -4. The **-point_obs** file may be used to pass additional NetCDF point observation files to be used in the verification. Python embedding for point observations is also supported, as described in :numref:`pyembed-point-obs-data`. +5. The **-ugrid_config** option provides a way for a user to provide a separate config file with metadata about their UGRID. -5. The **-obs_valid_beg** time option in YYYYMMDD[_HH[MMSS]] format sets the beginning of the observation matching time window, overriding the configuration file setting. +5. The **-point_obs** file may be used to pass additional NetCDF point observation files to be used in the verification. Python embedding for point observations is also supported, as described in :numref:`pyembed-point-obs-data`. -6. The **-obs_valid_end** time option in YYYYMMDD[_HH[MMSS]] format sets the end of the observation matching time window, overriding the configuration file setting. +6. The **-obs_valid_beg** time option in YYYYMMDD[_HH[MMSS]] format sets the beginning of the observation matching time window, overriding the configuration file setting. -7. The **-outdir path** indicates the directory where output files should be written. +7. The **-obs_valid_end** time option in YYYYMMDD[_HH[MMSS]] format sets the end of the observation matching time window, overriding the configuration file setting. -8. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file. +8. The **-outdir path** indicates the directory where output files should be written. -9. The **-v level** option indicates the desired level of verbosity. The value of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity will increase the amount of logging. +9. The **-log file** option directs output and errors to the specified log file. All messages will be written to that file as well as standard out and error. Thus, users can save the messages without having to redirect the output on the command line. The default behavior is no log file. + +10. The **-v level** option indicates the desired level of verbosity. The value of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity will increase the amount of logging. An example of the point_stat calling sequence is shown below: diff --git a/docs/Users_Guide/refs.rst b/docs/Users_Guide/refs.rst index 1c328014cb..00e682abad 100644 --- a/docs/Users_Guide/refs.rst +++ b/docs/Users_Guide/refs.rst @@ -10,6 +10,14 @@ References | Atlantic Basin. *Weather and Forecasting*, 13, 1005-1015. | +.. _Adams-2019: + +| Adams, S. V., R. W. Ford, M. Hambley, J.M. Hobson, I. Kavčič, C. M. Maynard, +| T. Melvin, E. H. Müller, S. Mullerworth, A. R. Porter, M. Rezny, B. J. Shipway, +| and R. Wong, 2019: LFRic: Meeting the challenges of scalability and performance +| portability in Weather and Climate models. *Journal of Parallel and Distributed Computing*, +| **132**, 383-396, doi: https://doi.org/10.1016/j.jpdc.2019.02.007. + .. _Ahijevych-2009: | Ahijevych, D., E. Gilleland, B.G. Brown, and E.E. Ebert, 2009: Application of @@ -333,6 +341,14 @@ References | and Recommendations. *Monthly Weather Review*, 145, 3397-3418. | +.. _Skamarock-2012: + +| Skamarock, W. C., J. B. Klemp, M. G. Duda, L. D. Fowler, S. Park, and +| T. Ringler, 2012: A Multiscale Nonhydrostatic Atmospheric Model Using +| Centroidal Voronoi Tesselations and C-Grid Staggering. *Mon. Wea. Rev.*, +| **140**, 3090-3105, doi: https://doi.org/10.1175/MWR-D-11-00215.1. +| + .. _Stephenson-2000: | Stephenson, D.B., 2000: Use of the "Odds Ratio" for diagnosing