Feature 2741 update use cases (#2795)

* updating template first pass * removing backslash * removing extra # symbol * removing spaces. Keep as is * removing spaces for consistency * adding changes to match template * fixing spacing * fixing spacing take 2 * Per #2741, correcting typo in OUPUT_BASE in template file * Per #2741, correcting typo in OUPUT_BASE * Per #2741, updating sections based on template and template description * Updated language in several sections, needs additional work on output and config file * updating for consistency * Updating file based on suggestions from George * Updating with suggestions from George * Updating file based on suggestions from George * reviewing with Julie * Per #2741 attempting to resolve errors * Per #2741, updating template to fix rendering problems * Per #2741, fixing formatting * Per #2741 attempting to resolve warnings * Per #2741, resolving error * Resolving formatting problems with template * Fixing formatting problems * updating for consistency * fixing formatting * removing extra spaces * Updating formatting * updating for consistency * adding METplus Workflow times * modifications * Update GridStat_fcstCESM_obsGFS_ConusTemp.py Added some information to the use case * Update MODE_fcstCESM_obsGPCP_AsianMonsoonPrecip.py Updates * Added an extra blank line * updating for consistency * fixing spacing * adding space * removing Update content * updating for consistency * adding a period * updating for consistency * updating for consistency * updating for consistency * updating for consistency * adding return * replacing underlines * replacing return? * fixing blank line * more attempts to fix warnings * updating for consistency * adding double colons and stuff * trying stuff * trying extra spacing * Changing Datasets Location info to match template * Per #2741, updating formatting and wording * Per #2741, updating after reviewing * Per #2741, attempting to resolve errors * updating for consistency * fixing brackets and section breaks * adding return * Per #2741, updating after reviewing * updating for consistency * fixing spacing * adding a pound * Updating wording for User Scripting * Updating the text for User Scripting * Updating User Script section * Updating text for User Scripting * Updating text * adding location information * adding location information * updated first cloud file, will use for remaining cloud use cases * updated language in numerous use case doc files * updated final marine use case --------- Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: j-opatz <[email protected]>
dtcenter · Nov 18, 2024 · 847088d · 847088d
1 parent 52c63bc
commit 847088d
Show file tree

Hide file tree

Showing 14 changed files with 1,263 additions and 675 deletions.
diff --git a/docs/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.py b/docs/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.py
@@ -1,11 +1,17 @@
 """
+
 EnsembleStat: Using Python Embedding for Aerosol Optical Depth
-=============================================================================
+==============================================================
 
-model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS\
-_aod.conf
+model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf
 
 """
+##############################################################################
+# .. contents::
+#   :depth: 1
+#   :local:
+#   :backlinks: none
+
 ############################################################################
 # Scientific Objective
 # --------------------
@@ -14,40 +20,68 @@
 # observation data for aersol optical depth (AOD) to an ensemble forecast.
 # These values can be used to help correct ensemble member deviations from observed values.
 
+##############################################################################
+# Version Added
+# -------------
+#
+# METplus version 4.0
 
 ##############################################################################
 # Datasets
 # --------
 #
-# | **Forecast:** International Cooperative for Aerosol Prediction (ICAP) ensemble netCDF file, 7 members
-# | **Observation:** Aggregate netCDF file with MODIS observed AOD field
+# **Forecast:** International Cooperative for Aerosol Prediction (ICAP) ensemble netCDF file, 7 members
+#
+# **Observation:** Aggregate netCDF file with MODIS observed AOD field
 #
-# | **Location:** All of the input data required for this use case can be found in the met_test sample data tarball. Click here to the METplus releases page and download sample data for the appropriate release: https://github.com/dtcenter/METplus/releases
-# | The tarball should be unpacked into the directory that you will set the value of INPUT_BASE. See `Running METplus`_ section for more information.
-# |
+# **Climatology:** None
 #
+# **Location:** All of the input data required for this use case can be 
+# found in a sample data tarball. Each use case category will have 
+# one or more sample data tarballs. It is only necessary to download 
+# the tarball with the use case’s dataset and not the entire collection 
+# of sample data. Click here to access the METplus releases page and download sample data 
+# for the appropriate release: https://github.com/dtcenter/METplus/releases
+# This tarball should be unpacked into the directory that you will 
+# set the value of INPUT_BASE. See :ref:`running-metplus` section for more information.
 
 ##############################################################################
 # METplus Components
 # ------------------
 #
-# This use case utilizes the METplus EnsembleStat wrapper to read in files using Python Embedding
-#
+# This use case utilizes the METplus EnsembleStat wrapper to read in files using Python Embedding.
 
 ##############################################################################
 # METplus Workflow
 # ----------------
 #
-# EnsembleStat is the only tool called in this example. It processes a single run time with seven ensemble members. Three of the members do not have data for the AOD field, so EnsembleStat will only process four of the members for statistics.
+# **Beginning time (INIT_BEG):** 201608150000 
+#
+# **End time (INIT_END):** 201608150000
+#
+# **Increment between beginning and end times (INIT_INCREMENT):** 06H
 #
+# **Sequence of forecast leads to process (LEAD_SEQ):** 12H
+#
+# EnsembleStat is the only tool called in this example. It processes a single run time with seven ensemble members,
+# with each ensemble member receiving its own verification.
+# Preprocessing of the ensemble forecast data is completed with Python Embedding, which takes 4 inputs:
+# the full path to the forecast file, variable name, valid time of verification, and ensemble member number. The script passes 
+# back the variable field requested to EnsembleStat for verification. A similar process is completed
+# for the observation data, which is preprocessed by a separate Python Embedding script which takes 3 inputs:
+# the full path to the observation file, group name that contains the variable field, and variable name.
+# The script passes back the requested variable field and begins the verification process.
+# Three of the ensemble members do not have data for the AOD field, so EnsembleStat 
+# will only process four of the members for statistics.
+# After a successful run, EnsembleStat will create the requested output and its corresponding files.
 
 ##############################################################################
 # METplus Configuration
 # ---------------------
 #
-# METplus first loads all of the configuration files found in parm/metplus_config,
-# then it loads any configuration files passed to METplus via the command line
-# with the -c option, i.e. -c parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf
+# METplus first loads all of the configuration files found in parm/metplus_config, 
+# then it loads any configuration files passed to METplus via the command line,
+# i.e. parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf
 #
 # .. highlight:: bash
 # .. literalinclude:: ../../../../parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf
@@ -56,64 +90,74 @@
 # MET Configuration
 # ---------------------
 #
-# METplus sets environment variables based on user settings in the METplus configuration file. 
-# See :ref:`How METplus controls MET config file settings<metplus-control-met>` for more details. 
+# METplus sets environment variables based on user settings in the METplus
+# configuration file. See :ref:`How METplus controls MET config file settings<metplus-control-met>` for more details.
 #
 # **YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!**
 #
-# If there is a setting in the MET configuration file that is currently not supported by METplus you'd like to control, please refer to:
+# If there is a setting in the MET configuration file that is currently
+# not supported by METplus you’d like to control, please refer to:
 # :ref:`Overriding Unsupported MET config file settings<met-config-overrides>`
 #
-# .. note:: See the :ref:`EnsembleStat MET Configuration<ens-stat-met-conf>` section of the User's Guide for more information on the environment variables used in the file below:
+# .. dropdown:: EnsembleStatConfig_wrapped
 #
-# .. highlight:: bash
-# .. literalinclude:: ../../../../parm/met_config/EnsembleStatConfig_wrapped
+#   .. highlight:: bash
+#   .. literalinclude:: ../../../../parm/met_config/EnsembleStatConfig_wrapped
 
 ##############################################################################
 # Python Embedding
 # ----------------
 #
-# This use case uses two Python embedding scripts to read input data
+# This use case uses two Python embedding scripts to read input data: one for
+# the forecast ensemble data, and one for the observation data. The script processing
+# the ensemble data receives four input arguments: the full path to the forecast file, 
+# variable name, valid time of verification, and ensemble member number. Since seven ensemble
+# members are being verified, this script will run seven times. The processing is very simple,
+# with the script grabbing the initialization time from the file name, calculating the lead
+# by finding the difference between the valid time argument and the initialization time, grabbing
+# the variable name and index corresponding to the ensemble member input value, and then masking bad data
+# (anything less than -800) to the expected METplus bad data value of -9999. The latitude and longitude
+# variables are also extracted, and all of the information is returned to METplus for
+# verification via array and accompanying attribute dictionary.
+#
+# The second script for observational data behaves very similarly to the ensemble data
+# script. The script receives three inputs at runtime: 
+# the full path to the observation file, group name that contains the variable field, and variable name.
+# The requested variable field is extracted from the group name provided at runtime, bad data is
+# deemed to be any value less than -800 and reset to METplus' bad value of -9999, and the data
+# array is inverted to properly align with METplus' expected orientation. The latitude and longitude
+# variables are also extracted, and all of the information is returned to METplus for
+# verification via array and accompanying attribute dictionary.
 #
-# parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/forecast_embedded.py
+# For more information on the basic requirements to utilize Python Embedding in METplus, 
+# please refer to the MET User’s Guide section on `Python embedding <https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#appendix-f-python-embedding>`_ 
 #
-# .. highlight:: python
-# .. literalinclude:: ../../../../parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/forecast_embedded.py
 #
-# parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/analysis_embedded.py
+# .. dropdown:: parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/forecast_embedded.py
 #
-# .. highlight:: python
-# .. literalinclude:: ../../../../parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/analysis_embedded.py
+#   .. highlight:: python
+#   .. literalinclude:: ../../../../parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/forecast_embedded.py
 #
+# .. dropdown:: parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/analysis_embedded.py
+#
+#   .. highlight:: python
+#   .. literalinclude:: ../../../../parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod/analysis_embedded.py
 
+##############################################################################
+# User Scripting
+# --------------
+# User Scripting is not used in this use case.
 
 ##############################################################################
 # Running METplus
 # ---------------
 #
-# It is recommended to run this use case by:
-#
-# Passing in EnsembleStat_python_embedding.conf then a user-specific system configuration file::
-#
-#   run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf -c /path/to/user_system.conf
-#
-# The following METplus configuration variables must be set correctly to run this example.:
-#
-# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs).
-# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions
-# * **MET_INSTALL_DIR** - Path to location where MET is installed locally
+# Pass the use case configuration file to the run_metplus.py script along 
+# with any user-specific system configuration files if desired::
 #
-# Example User Configuration File::
+#   run_metplus.py /path/to/METplus/parm/use_cases/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf /path/to/user_system.conf
 #
-#   [dir]
-#   INPUT_BASE = /path/to/sample/input/data
-#   OUTPUT_BASE = /path/to/output/dir
-#   MET_INSTALL_DIR = /path/to/met-X.Y 
-#
-# **NOTE:** All of these items must be found under the [dir] section.
-#
-
-
+# See :ref:`running-metplus` for more information.
 
 ##############################################################################
 # Expected Output
@@ -123,9 +167,10 @@
 #
 #   INFO: METplus has successfully finished running.
 #
-# Refer to the value set for **OUTPUT_BASE** to find where the output data was generated.
-# Output for this use case will be found in model_applications/air_quality/AOD  (relative to **OUTPUT_BASE**)
-# and will contain the following files:
+# Refer to the value set for **OUTPUT_BASE** to find where the output data was generated. 
+# Output for this use case will be found in 
+# {OUTPUT_BASE}/model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod 
+# and will contain the following files::
 #
 # * ensemble_stat_aod_20160815_120000V_ecnt.txt
 # * ensemble_stat_aod_20160815_120000V_ens.nc
@@ -135,7 +180,6 @@
 # * ensemble_stat_aod_20160815_120000V_rhist.txt
 # * ensemble_stat_aod_20160815_120000V_ssvar.txt
 # * ensemble_stat_aod_20160815_120000V.stat
-#
 
 ##############################################################################
 # Keywords
@@ -153,4 +197,3 @@
 #
 #
 # sphinx_gallery_thumbnail_path = '_static/air_quality_and_comp-EnsembleStat_fcstICAP_obsMODIS_aod.png'
-#