[Issue #333] Use just one request to download a date range #347
Conversation
flake8 Merge pull request dbekaert#343 from the-garlic-os/add-dev-dep-autopep8 Add dev dependency "autopep8" Merge pull request dbekaert#346 from the-garlic-os/lint flake8 Merge pull request dbekaert#343 from the-garlic-os/add-dev-dep-autopep8 Add dev dependency "autopep8" Merge pull request dbekaert#349 from the-garlic-os/flake8 More flake8
Raise better error for missing subdatasets Housekeeping Housekeeping Use "_" for unused parameter Housekeeping Removing used variable assignments, adding comments, adding type annotations, etc. as I go along Type annotate main program arguments A mostly complete model of the arguments passed to tropo_delay() as I gathered from analyzing the code. May not be correct for all cases. Still need to figure out what Arguments["Heights"][1] is. Comment out unused function Searching for the function name through the whole project with vscode search, I don't see it called anywhere, but I probably shouldn't outright delete it before asking about it.
garlic-os
changed the title
|
This is the progress I've made so far. Crashes on an error happening in WeatherModel.self._uniform_in_z() involving three arrays not having the same dimensions. Successfully fetches a date range of ERA5 data in one request, but I still need to ensure the data is processed properly afterward. |
- Removed or commented out unused names found by vulture - Refactored some variable names to use snake_case - Importing only from typing and not typing_extensions - Replaced uses of `except BaseException` with just `except` along with a TODO comment to fill in the missing exception type -
It looks like the otherwise unused `cpu_num` is meant to be passed to np.Pool().
A considerable number of function signatures changed as collateral damage, but this should allow RAiDER to calculate delays through ERA5 with only one CDS request for any range of dates.
Use super() for calling superclass constructor
|
Working now in testing. It splits the downloaded date range into separate files for the rest of the program to digest as usual. From the testing I've done, the results seem to be the same as the were with multiple requests. But I need to write some tests so I can examine the change's validity beyond eyeballing the PDFs 😁 |
Following changes to _tropo_delay, it is no longer necessary as a processing layer to tropo_delay.
Fix dictionary formatting
Housekeeping Housekeeping
Fix missing output files
|
Added tests to ensure this change doesn't affect the outputted data. |
|
Fixed merge conflicts |
Remove commented-out code
Data that the tests use to compare to live data.
This also fixes the failing enu2ecef and ecef2enu series of tests.
Rename datetime module to dt to remove ambiguity with datetime and datetime.datetime Housekeeping Housekeeping Improve dict formatting
import datetime as dt
|
Weird. It works on my machine. Is CircleCI running the tests from a different working directory? |
|
Struggling to reproduce the error happening on CircleCI. Are the tests passing for anyone else? @jlmaurer |
|
While meeting with @jlmaurer this seems to have uncovered an issue with heights not being properly interpolated, which has also been causing issues elsewhere. Not able to properly test the changes in this issue until this other problem is fixed. Putting this PR on hold for now. |
|
@the-garlic-os I just updated the dev branch with the fix for ERA-5 API. Can you merge that with this and see if that solves the CI failures? |
|
@jlmaurer Some errors in tools.RAiDER.dem that I think you should look at: |
|
(Oops, I guess renaming a branch closes all its pull requests) |
autopep8 Housekeeping Remove download_only path - Per Jeremy this logic is no longer used. stash dep changes update environment deps correct name of dem_stitcher conda package Merge pull request dbekaert#339 from jlmaurer/fix_stitchdem update dem.py with new dem-stitcher api Merge branch 'dev' into update_deps remove uneeded deps Update ERA5 API Remove unneeded deps
|
@the-garlic-os yes those are bugs that I've fixed in #332 . Let's pause on this until that one gets merged. |
Remove unused import Update .gitignore Remove unused argument Ignore .pytest_cache/
No really for real this time
Description
Implements the feature described in the third issue in Issue #333.
How Has This Been Tested?
Added tests (
tests/test_issue_333.py) to ensure the processed data is the same with the new method of downloading the data and the old one.Type of change
Checklist: