Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: quantize_mode="GranularBitRound" replaces nans by "-0". #1314

Open
garciampred opened this issue Mar 20, 2024 · 3 comments
Open

Bug: quantize_mode="GranularBitRound" replaces nans by "-0". #1314

garciampred opened this issue Mar 20, 2024 · 3 comments

Comments

@garciampred
Copy link

garciampred commented Mar 20, 2024

This is likely an upstream bug, but I don't know how to reproduce it with netcdf-c. BitRound and BitGroom are not affected.

import numpy as np
import netCDF4 as ncdf
data = np.array([5.3, 6.2, 7.3, np.nan])
nc = ncdf.Dataset("test.nc", "w")
nc = ncdf.Dataset("test.nc", "w")
nc.createDimension("x", 4)
v = nc.createVariable("tas", "float32", "x", shuffle=True, compression="zlib", significant_digits=4, quantize_mode="GranularBitRound")
v[:] = data
nc.close

Then ncdump returns

netcdf test {
dimensions:
	x = 4 ;
variables:
	float tas(x) ;
		tas:_QuantizeGranularBitRoundNumberOfSignificantDigits = 4 ;
		tas:_Storage = "chunked" ;
		tas:_ChunkSizes = 4 ;
		tas:_DeflateLevel = 4 ;
		tas:_Shuffle = "true" ;
		tas:_Endianness = "little" ;

// global attributes:
		:_NCProperties = "version=2,netcdf=4.9.2,hdf5=1.14.3" ;
		:_SuperblockVersion = 2 ;
		:_IsNetcdf4 = 1 ;
		:_Format = "netCDF-4" ;
data:

 tas = 5.299805, 6.200195, 7.299805, -0 ;
}
``

I am using the following versions 

hdf5 1.14.3 nompi_h4f84152_100 conda-forge
libnetcdf 4.9.2 nompi_h9612171_113 conda-forge
netcdf4 1.6.5 nompi_py312h26027e0_100 conda-forge

@jswhit
Copy link
Collaborator

jswhit commented Mar 21, 2024

not sure you should expect the quantization to leave nans alone, unless the _FillValue or missing_value is set to nan

@jswhit2
Copy link
Contributor

jswhit2 commented Mar 22, 2024

it turns out that even if the _FillValue is set to nan, the same problem occurs.

however, you can use

data = np.ma.masked_invalid(np.array([5.3, 6.2, 7.3, np.nan]))

and you end up with

netcdf test {
dimensions:
	x = 4 ;
variables:
	float tas(x) ;
		tas:_QuantizeGranularBitRoundNumberOfSignificantDigits = 4 ;
data:

 tas = 5.299805, 6.200195, 7.299805, _ ;
}

@jswhit
Copy link
Collaborator

jswhit commented Apr 10, 2024

did you report this upstream?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants