-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't construct lat/long arrays before scattering. #59
base: main
Are you sure you want to change the base?
Conversation
Thanks for these optimizations! I'll review them soon and do timing/regression tests. I have some other changes that need to go in ASAP, so this may be a few days, and need to be rebased, but I'll take care of that. |
Should the assumption be made that latitude and longitude have the same dimensions? |
No, we can't assume that. For example, in GFS, the latitude dimension is 1536 and the longitude is 3072. |
by dimensions, I meant number of dimensions. sorry. |
I still think that assumption isn't safe; in some input files the latitude and longitude fields are 2D, and in some there is no time dimension on the climate fields. |
The reason I asked is because that assumption was being made implicitly in the code. I added some checks that hopefully can catch odd inputs. |
@rcabell are the regression tests publicly available? |
No, the tests are currently a bit ad-hoc and not yet available publicly. |
This should reduce the overhead of scattering the arrays. Using broadcasting also reduces memory usage.