Add unit tests for production YAML config file elements related to workers #117
Comments
douglatornell
added a commit
that referenced
this issue
Nov 8, 2022
Test suite maintenance. re: issue #117
douglatornell
added a commit
that referenced
this issue
Nov 8, 2022
Test suite maintenance. re: issue #117
douglatornell
added a commit
that referenced
this issue
Feb 8, 2024
Add unit tests for production YAML config file elements related to worker; re: issue #117.
douglatornell
added a commit
that referenced
this issue
Feb 29, 2024
… v3 (#234) * Modernize test_get_onc_ctd Replace unittest.mock.patch decorator with pytest.fixture for mock worker. Add unit tests for production YAML config file elements related to worker; re: issue #117. Replace unittest.mock.patch decorator with pytest caplog fixture for tests of logging; re: issue #82. * Add unit test for YAML config file elements Added unit test for production YAML configuration file ctd data observations elements used in after_* () functions. * Improve get_onc_ctd.main() function Update docstring re: issue #121. Return worker so that modernized unit tests for main() work; re issue #81. * Update get_onc_ctd worker to use ONC API v3 Modified the get_onc_ctd worker to use ONC API v3 'getByLocation' method. The new method uses a location code instead of a station name. Also, these changes include adding a 'dateTo' parameter to limit the data to a specific time range for the day. Additionally, small changes to variable and parameter names were made to match the new method requirements. * Remove test skipping due to resolved issue #174 The pytest skip marker for "_resample_nav_coord()" function in "tests/workers/test_get_onc_ferry.py" file has been removed. This has happened following the successful resolution of issue number #174, making it unnecessary to skip these specific unit tests anymore. Fixes issue #174 * Add unit tests for YAML config file elements Add unit tests for production YAML config file elements related to worker; re: issue #117. * Update YAML config with Tsawwassen - Duke Point ferry route The 'nowcast.yaml' configuration file has been updated to include the Tsawwassen - Duke Point ferry route details including devices, sensors, and other relevant data. * Update dataset ID for TWDP-ferry The dataset ID for TWDP-ferry was updated in the 'nowcast.yaml' configuration file and in respective test. The change is reflected accurately to match the new details for the Tsawwassen - Duke Point ferry route. * Update get_onc_ferry worker to use ONC API v3 Modified the get_onc_ferry worker to use ONC API v3 'getByLocation' method. The new method uses a location code instead of a station name. Also, these changes include adding a 'dateTo' parameter to limit the data to a specific time range for the day. Server-side averaging into 1-second bins is used to ensure that an entire day's observations from each sensor can be obtained by a single API request; there is a limit of 100,000 "rows" per sensor. Additionally, small changes to variable and parameter names were made to match the new method requirements. * Update ONC_data_product_url attr in ferry datasets Updated the ONC_data_product_url attribute in the datasets produced by the get_onc_ferry worker to reflect changes in the data source's domain name and API query parameter names. Code changes include updating the ONC data API domain name, adding the 'locationCode' query parameter, and changing the 'deviceCategory' query parameter name to 'deviceCategoryCode'. All of those changes are for compatibility with the ONC data API v3. * Correct representation of relative humidity attribute Changed the naming representation of the relative humidity attribute in the 'get_onc_ferry' worker. The attribute was initially named "REL_HUMIDITY" and it was renamed to "rel_humidity", matching the common low-caps format of the other attributes for consistency of naming convention. * Update QA/QC filter criteria in get_onc_ferry worker Adjust the QA/QC filter settings in the 'get_onc_ferry' worker to now include data where the 'qaqcFlag' attribute is less than or equal to 1 or greater than or equal to 7. This update is necessary because of the change to server-side averaging in the request. * Improve empty data array handling ONC API v3 has more/different ways of returning empty responses to data requests that we have to handle. It's not as clean as I would like, but it is good enough for the purpose.
douglatornell
added a commit
that referenced
this issue
Mar 29, 2024
Add unit tests for production YAML config file elements related to worker; re: issue #117.
douglatornell
added a commit
that referenced
this issue
Mar 30, 2024
* Change NowcastWorker mock to pytest fixture Test suite maintenance. re: issue #81 * Update make_plots worker main() function docstring Removed not informative "Set up and run the worker." line at the beginning. re: issue #121 * Change logging mocks to pytest caplog fixture Replace unittest.mock.patch decorator with pytest caplog fixture for tests of logging. Test suite maintenance re: issue #82. * Add unit tests for YAML config file elements Add unit tests for production YAML config file elements related to worker; re: issue #117. * Remove unused ferry_data_dir configuration retrieval The ferry_data_dir configuration was retrieved but not used in the make_plots.py worker script. This change removes the unneeded line to tidy up the code and avoid potential confusion in the future. * Update to V21-11 dataset URLs in config & tests The URLs for '3d tracer fields' and '3d biology fields' have been updated in the test_make_plots.py and nowcast.yaml files. * Rename physics dataset keys to '3d physics fields' This commit changes the '3d tracer fields' key in the nowcast.yaml configuration file, the make_plots.py worker and test_make_plots.py tests to '3d physics fields'. This change better reflects in the relevant data source URLs. * Update zooplankton field var names in make_plots Update zooplankton field variable names for the time series plot function in the make_plots worker. Specifically, change "mesozooplankton" and "microzooplankton" to "z1_zooplankton" and "z2_zooplankton". This update ensures consistency with the V21-11 model output variable names. * Replace Mesodinium rubrum w/ Diatoms in time series plots In the 'make_plots' worker, the field variable 'mesodinium' was changed to 'diatoms' for the time series plots. The 'diatoms_flagellates_timeseries' dictionary key is adjusted accordingly. This is necessary due to the removal of the Mesodinium rubrum variable from the V21-11 model calculations and output. * Add z1 & z2 zooplankton to color dict in website_theme Two new types of zooplankton, 'z1_zooplankton' and 'z2_zooplankton', have been added to the color dictionary of nowcast/figures/website_theme.py file. This change would allow the correct color to be displayed for these new types in the corresponding plots.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add
TestConfigclass in the following modules:test_download_wwatch3_results.pytest_get_onc_ctd.pytest_make_feeds.pytest_make_plots.pytest_make_turbidity_file.pytest_run_NEMO.pytest_run_NEMO_agrif.pytest_watch_NEMO_agrif.pytest_watch_NEMO.pytest_watch_ww3.pyThe text was updated successfully, but these errors were encountered: