Size of zarr store vs grib is 10x #9659
Unanswered
kingkong-cmd
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I'm donwloading wind forecast data from GFS in GRIB2 format. I'm only getting two variables (u and v 10m), and one GRIB file for each forecast time, which means I am downloading apprx. 150 files. Each file is apprx. 1.5 MB, so in total its around 200 MB.
I tried converting into a zarr store by reading each file into xarray and then write to the zarr store like this:
ds.to_zarr('test.zarr', append_dim = 'time')
, but noticed that the zarr store is 2+ GB on disk, meaning more than 10x size of original files. Am I doing something wrong here or are GRIB files just really well compressed? Is there anything I can do to reduce size?For the record, this is the xarray representation of a single GRIB files:
And this is the xarray representation when loading the whole zarr store into xarray:
Beta Was this translation helpful? Give feedback.
All reactions