Skip to content
This repository has been archived by the owner on Jun 2, 2023. It is now read-only.

Metadata cleanup #12

Merged
merged 9 commits into from
Nov 3, 2020
Merged

Metadata cleanup #12

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3e23792
oops, hadn't rebuilt xml
limnoliver Nov 2, 2020
d10f7a8
test 01_spatial, add min/max
limnoliver Nov 3, 2020
35c31b2
The google tells me that point data don't have a resolution?
limnoliver Nov 3, 2020
30f1410
Figured out resolution.
limnoliver Nov 3, 2020
e686bca
edit metadata after running through checker
limnoliver Nov 3, 2020
8e71ecc
updates to 3_inputs after running through xml validation
limnoliver Nov 3, 2020
3ecf6d5
fix predicted discharge date range
limnoliver Nov 3, 2020
b6af3ff
updates after running through validation
limnoliver Nov 3, 2020
3258c96
moved observed discharge to inputs, changed file name to match predic…
limnoliver Nov 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion 3_inputs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ targets:
out_data/weather_drivers.zip:
command: zip_this(out_file = target_name, weather_drivers)


out_data/pred_discharge.csv:
command: subset_pred_discharge(
out_file = target_name,
Expand Down
16 changes: 10 additions & 6 deletions in_text/text_01_spatial.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,23 +38,27 @@ entities:
attr-def: >-
Latitude of the site location.
attr-defs: NA
data-min: NA
data-max: NA
data-units: NA
data-min: 30.14549
data-max: 48.90596
data-units: decimal degrees
-
attr-label: long
attr-def: >-
Longitude of the site location.
attr-defs: NA
data-min: NA
data-max: NA
data-units: NA
data-min: -123.3299
data-max: -70.97964
data-units: decimal degrees


data-name: GIS points of sites used in this study.
data-description: Location of USGS river gages used in this study.

file-format: Shapefile Data Set
process-date: 20201028
indirect-spatial: U.S.A.
latitude-res: 0.00001
longitude-res: 0.00001

build-environment: >-
This dataset was generated using XX.
9 changes: 8 additions & 1 deletion in_text/text_02_observations.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,4 +69,11 @@ entities:
data-max: NA
data-units: degrees Celsius

file-format: comma-delimited files
process-date: XX
indirect-spatial: U.S.A.
latitude-res: 0.00001
longitude-res: 0.00001
data-name: Flow and water temperature observations
data-description: >-
Flow and water temperature observations used to train and validate models described in Rahmani et al. 2020.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move flow observations to the Inputs item, too? (we already did this for flow predictions from the Ouyang model)

Copy link
Member Author

@limnoliver limnoliver Nov 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, I did not move these. Will do.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, done!

file-format: comma seperated file format (csv)
8 changes: 4 additions & 4 deletions in_text/text_03_inputs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,13 +306,13 @@ entities:
data-max: NA
data-units: cubic feet per second



build-environment: Multiple computer systems were used to generate these data, including XX. The open source languages R and Python was used on all systems, as well as XX.

process-date: !expr format(Sys.time(),'%Y%m%d')
indirect-spatial: U.S.A.
latitude-res: 0.1
longitude-res: 0.1
data-name: weather data, river catchment metadata

data-name: Model driver data
data-description: >-
Inputs (drivers) for the temperature models described in Rahmani et al. 2020, including weather drivers, river basin attributes, and simulated river discharge.
file-format: comma seperated file format (csv)
19 changes: 14 additions & 5 deletions in_text/text_05_predictions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ title: >-
Exploring the exceptional performance of a deep learning stream temperature model and the value of streamflow data: 5 model prediction data

abstract: >-
A deep learning model framework was used to make water temperature predictions in 118 river catchments across the U.S. All four model (LR, noQ, obsQ, and simQ) predictions are included. Additionally, a deep learning model was used to simulate discharge, which was used as inputs to the water temperature model.
A deep learning model framework was used to make water temperature predictions in 118 river catchments across the U.S. All four model (LR, noQ, obsQ, and simQ) predictions are included.

cross-cites:
-
authors: ['XX']
authors: ['DP Feng', 'K Fang', "CP Shen"]
title: >-
Cross cite code base?
pubdate: XX
link: XX
Enhancing streamflow forecast and extracting insights using continental-scale long-short term memory networks with data integration
pubdate: 2020
link: https://doi.org/10.1029/2019WR026793

build-environment: >-
We used XX open source XX; Any supercomputing resources used? XX
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this one, let's plan to (1) get more info from farshid about the compute environment (i added a bullet to "Text chunks we hope Farshid can fill in" in the "ERL data release plan" doc) and (2) add a reference to the environment.yml. I will make an issue for this so I can plan to do it - i have most of the info in hand already but will need a few minutes to put it together.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(We could also just write one cover-everything text chunk that we use for all metadata files)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added notes to #7

Expand Down Expand Up @@ -46,3 +46,12 @@ entities:
data-min: NA
data-max: NA
data-units: degrees Celsius

process-date: !expr format(Sys.time(),'%Y%m%d')
indirect-spatial: U.S.A.
latitude-res: 0.00001
longitude-res: 0.00001
data-name: Model predictions
data-description: >-
Stream water temperature predictions from each model described in Rahmani et al. 2020.
file-format: comma seperated file format (csv)
9 changes: 9 additions & 0 deletions in_text/text_06_evaluation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,12 @@ entities:

build-environment: >-
We used XX open source XX.

process-date: !expr format(Sys.time(),'%Y%m%d')
indirect-spatial: U.S.A.
latitude-res: 0.00001
longitude-res: 0.00001
data-name: Model evaluation metrics
data-description: >-
Evaluation metrics used to compare stream temperature models in Rahmani et al. 2020.
file-format: comma seperated file format (csv)
6 changes: 6 additions & 0 deletions in_text/text_SHARED.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,13 @@ funding-credits: >-
process-description: >-
At the core of the modeling framework is a deep learning model that uses inputs of XX.

process-date: 20201028
latitude-res: 0.1
longitude-res: 0.1
distro-person: Samantha K. Oliver

build-environment: Multiple computer systems were used to generate these data, including linux, OSX. The open source languages R and Python were used on all systems. XX

liability-statement: >-
Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected.
Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS),
Expand Down
142 changes: 105 additions & 37 deletions out_xml/00_parent.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,12 @@
<idinfo>
<citation>
<citeinfo>
<origin>Samantha K. Oliver, XX</origin>
<origin>Farshid Rahmani</origin>
<origin>Kathryn Lawson</origin>
<origin>Wenyu Ouyang</origin>
<origin>Alison Appling</origin>
<origin>Samantha Oliver</origin>
<origin>Chaopeng Shen</origin>
<pubdate>2020</pubdate>
<title>Data release: Exploring the exceptional performance of a deep learning stream temperature model and the value of streamflow data</title>
<geoform>parent data item for released data products</geoform>
Expand All @@ -14,16 +19,21 @@
<onlink>XX</onlink>
<lworkcit>
<citeinfo>
<origin>TBD XX</origin>
<origin>Farshid Rahmani</origin>
<origin>Kathryn Lawson</origin>
<origin>Wenyu Ouyang</origin>
<origin>Alison Appling</origin>
<origin>Samantha Oliver</origin>
<origin>Chaopeng Shen</origin>
<pubdate>2020</pubdate>
<title>TBD XX</title>
<title>Exploring the exceptional performance of a deep learning stream temperature model and the value of streamflow data.</title>
</citeinfo>
</lworkcit>
</citeinfo>
</citation>
<descript>
<abstract>&lt;p&gt;This data release provides all data and code used in Rahmani et al. 2020. Briefly, this project used the CAMELS dataset as a test case for temperature prediction using deep learning methods. XX &lt;br/&gt;The data are organized into these items:&lt;/p&gt; &lt;ol&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f908db182ce720ee2d0fef9"&gt;Spatial Information&lt;/a&gt; - Basin polygons and pour points for 118 river basins used in this study&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f986594d34e198cb77ff084"&gt;Observations&lt;/a&gt; - Water temperature observations and flow observations for the 118 river reaches used in this study&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865abd34e198cb77ff086"&gt;Model Inputs&lt;/a&gt; - Model inputs, including basin characteristics and weather drivers. &lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865cfd34e198cb77ff088"&gt;Models&lt;/a&gt; - Code and configuration for stream temperature models.&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865e5d34e198cb77ff08a"&gt;Model Predictions&lt;/a&gt; - Predictions of water temperature&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865fbd34e198cb77ff08c"&gt;Model Evaluation&lt;/a&gt; - Performance of models&lt;/li&gt; &lt;br/&gt; &lt;p&gt;This research was funded by the USGS, XX.&lt;/p&gt;</abstract>
<purpose>Decision support, limnological research, and fish habitat.</purpose>
<abstract>&lt;p&gt;This data release provides all data and code used in Rahmani et al. 2020. Briefly, this project used the Gages II dataset as a test case for temperature prediction using deep learning methods. XX &lt;br/&gt;The data are organized into these items:&lt;/p&gt; &lt;ol&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f908db182ce720ee2d0fef9"&gt;Spatial Information&lt;/a&gt; - Pour points for 118 river basins used in this study&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f986594d34e198cb77ff084"&gt;Observations&lt;/a&gt; - Water temperature observations and flow observations for the 118 river reaches used in this study&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865abd34e198cb77ff086"&gt;Model Inputs&lt;/a&gt; - Model inputs, including basin characteristics and weather drivers. &lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865cfd34e198cb77ff088"&gt;Models&lt;/a&gt; - Code and configuration for stream temperature models.&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865e5d34e198cb77ff08a"&gt;Model Predictions&lt;/a&gt; - Predictions of stream water temperature and discharge&lt;/li&gt; &lt;li&gt;&lt;a href="https://www.sciencebase.gov/catalog/item/5f9865fbd34e198cb77ff08c"&gt;Model Evaluation&lt;/a&gt; - Performance metrics of each stream temperature model&lt;/li&gt; &lt;br/&gt; &lt;p&gt;This research was funded by the Integrated Water Prediction Program at the US Geological Survey.&lt;/p&gt;</abstract>
<purpose>Decision support, water quality research</purpose>
</descript>
<timeperd>
<timeinfo>
Expand All @@ -39,25 +49,23 @@
<update>none planned</update>
</status>
<spdom>
<descgeog>River reach polylines as defined by the XX.</descgeog>
<descgeog>Locations of sites used in the study, which are a subset of the Gages II dataset.</descgeog>
<bounding>
<westbc/>
<eastbc/>
<northbc/>
<southbc/>
<westbc>-123.32988684</westbc>
<eastbc>-70.97964444</eastbc>
<northbc>48.90595739</northbc>
<southbc>30.1454932</southbc>
</bounding>
</spdom>
<keywords>
<theme>
<themekt>none</themekt>
<themekey>machine learning</themekey>
<themekey>deep learning</themekey>
<themekey>hybrid modeling</themekey>
<themekey>water</themekey>
<themekey>temperature</themekey>
<themekey>reservoirs</themekey>
<themekey>streams</themekey>
<themekey>modeling</themekey>
<themekey>XX</themekey>
</theme>
<theme>
<themekt>ISO 19115 Topic Category</themekt>
Expand All @@ -73,30 +81,90 @@
</place>
<place>
<placekt>U.S. Department of Commerce, 1987, Codes for the identification of the States, the District of Columbia and the outlying areas of the United States, and associated areas (Federal Information Processing Standard 5-2): Washington, D. C., NIST</placekt>
<placekey>Alabama</placekey>
<placekey>AL</placekey>
<placekey>Delaware</placekey>
<placekey>DE</placekey>
<placekey>Georgia</placekey>
<placekey>GA</placekey>
<placekey>Idaho</placekey>
<placekey>ID</placekey>
<placekey>Iowa</placekey>
<placekey>IA</placekey>
<placekey>Kansas</placekey>
<placekey>KS</placekey>
<placekey>Maine</placekey>
<placekey>ME</placekey>
<placekey>Maryland</placekey>
<placekey>MD</placekey>
<placekey>Massachusetts</placekey>
<placekey>MA</placekey>
<placekey>Michigan</placekey>
<placekey>MI</placekey>
<placekey>Mississippi</placekey>
<placekey>MS</placekey>
<placekey>Nevada</placekey>
<placekey>NV</placekey>
<placekey>New Jersey</placekey>
<placekey>NJ</placekey>
<placekey>New Mexico</placekey>
<placekey>NM</placekey>
<placekey>New York</placekey>
<placekey>NY</placekey>
<placekey>North Carolina</placekey>
<placekey>NC</placekey>
<placekey>Ohio</placekey>
<placekey>OH</placekey>
<placekey>Oklahoma</placekey>
<placekey>OK</placekey>
<placekey>Oregon</placekey>
<placekey>OR</placekey>
<placekey>Pennsylvania</placekey>
<placekey>PA</placekey>
<placekey>Rhode Island</placekey>
<placekey>RI</placekey>
<placekey>South Carolina</placekey>
<placekey>SC</placekey>
<placekey>Tennessee</placekey>
<placekey>TN</placekey>
<placekey>Texas</placekey>
<placekey>TX</placekey>
<placekey>Utah</placekey>
<placekey>UT</placekey>
<placekey>Virginia</placekey>
<placekey>VA</placekey>
<placekey>Washington</placekey>
<placekey>WA</placekey>
<placekey>West Virginia</placekey>
<placekey>WV</placekey>
<placekey>Wisconsin</placekey>
<placekey>WI</placekey>
<placekey>Wyoming</placekey>
<placekey>WY</placekey>
</place>
</keywords>
<accconst>none</accconst>
<useconst>These data are subject to change and are not citable until reviewed and approved for official publication by the USGS</useconst>
<ptcontac>
<cntinfo>
<cntperp>
<cntper>Samantha K. Oliver</cntper>
<cntper>Farshid Rahmani</cntper>
<cntorg>U.S. Geological Survey</cntorg>
</cntperp>
<cntpos>Hydrologist</cntpos>
<cntpos>Graduate Research Assistant</cntpos>
<cntaddr>
<addrtype>Mailing and Physical</addrtype>
<address>8505 Research Way</address>
<city>Middleton</city>
<state>WI</state>
<postal>53562</postal>
<address>Sackett Building, Pennsylvania State University</address>
<city>State College</city>
<state>PA</state>
<postal>16801</postal>
<country>U.S.A.</country>
</cntaddr>
<cntvoice>608-821-3824</cntvoice>
<cntemail>[email protected]</cntemail>
<cntvoice>NA</cntvoice>
<cntemail>[email protected]</cntemail>
</cntinfo>
</ptcontac>
<datacred>This study was funded by the Department of the Interior Northeast Climate Adaptation Science Center, the United States Geological Survey National Climate This research used resources of the Core Science Analytics and Synthesis Advanced Research Computing program at the U.S. Geological Survey.</datacred>
<datacred>This study was funded by the Integrated Water Prediction Program at the U.S. Geological Survey. XX.</datacred>
<native>Multiple computer systems were used to generate these data, including linux, OSX. The open source languages R and Python were used on all systems. XX</native>
</idinfo>
<dataqual>
Expand All @@ -115,18 +183,18 @@
</posacc>
<lineage>
<procstep>
<procdesc>At the core of the modeling framework is a coupled hydrologic-thermodynamic model that uses inputs of reach-specific properties and local meteorology to estimate flow and water temperature. Our chosen model is the open source, Precipitation Runoff Modeling System (PRMS) with the coupled Stream Network Temperature Model (SNTemp) version XX. PRMS-SNTemp is a ...XX. We used PRMS-SNTemp to simulate stream flow and temperature for the period of record...XX.</procdesc>
<procdate>20201027</procdate>
<procdesc>At the core of the modeling framework is a deep learning model that uses inputs of XX.</procdesc>
<procdate>20201103</procdate>
</procstep>
</lineage>
</dataqual>
<spdoinfo>
<indspref>U.S.A.</indspref>
<direct/>
<direct>Point</direct>
<ptvctinf>
<sdtsterm>
<sdtstype/>
<ptvctcnt/>
<sdtstype>Point</sdtstype>
<ptvctcnt>118</ptvctcnt>
</sdtsterm>
</ptvctinf>
</spdoinfo>
Expand Down Expand Up @@ -186,25 +254,25 @@
</stdorder>
</distinfo>
<metainfo>
<metd>20201027</metd>
<metd>20201103</metd>
<metc>
<cntinfo>
<cntperp>
<cntper>Samantha K. Oliver</cntper>
<cntper>Farshid Rahmani</cntper>
<cntorg>U.S. Geological Survey</cntorg>
</cntperp>
<cntpos>Hydrologist</cntpos>
<cntpos>Graduate Research Assistant</cntpos>
<cntaddr>
<addrtype>Mailing and Physical</addrtype>
<address>8505 Research Way</address>
<city>Middleton</city>
<state>WI</state>
<postal>53562</postal>
<address>Sackett Building, Pennsylvania State University</address>
<city>State College</city>
<state>PA</state>
<postal>16801</postal>
<country>U.S.A.</country>
</cntaddr>
<cntvoice>608-821-3824</cntvoice>
<cntfax>608-821-3817</cntfax>
<cntemail>[email protected]</cntemail>
<cntvoice>NA</cntvoice>
<cntfax>NA</cntfax>
<cntemail>[email protected]</cntemail>
</cntinfo>
</metc>
<metstdn>FGDC Content Standard for Digital Geospatial Metadata</metstdn>
Expand Down
Loading