Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] add decimal floating-point number support #689

Open
matthewdale opened this issue Jun 5, 2023 · 2 comments
Open

[feature] add decimal floating-point number support #689

matthewdale opened this issue Jun 5, 2023 · 2 comments

Comments

@matthewdale
Copy link
Contributor

The BSON and Avro formats support encoding decimal floating-point numbers which cannot be exactly represented by a float64. Currently, neither the bson or avro decoder packages actually support decoding the respective decimal number value, only displaying the values as binary. There are two main things that stand in the way of decoding BSON and Avro decimal number values:

  • The decode.D type doesn't support decoding decimal floating-point numbers.
  • The wader/gojq library doesn't support arithmetic on decimal floating-point numbers.

Add support for decoding various decimal floating-point binary formats. Consider using the shopspring/decimal library for the decoded value representation because it can represent almost any decimal value (except NaN, +Inf, -Inf).

@wader
Copy link
Owner

wader commented Jun 6, 2023

I wonder if we should ask the gojq author what he thinks of decimal support. He has already extended from original jq to support arbitrarily precision so maybe he has some ideas. Would be nice to not have too much difference compared to upstream.

btw any idea if big.Rat from standard library would be ok also?

@matthewdale
Copy link
Contributor Author

Good idea, I opened an issue on gojq requesting support for arbitrary precision non-integer decimal values: itchyny/gojq#216

I believe big.Rat can represent all numbers that shopspring/decimal can represent and seems to support the required arithmetic operations (except modulus, which would probably be accomplished by truncating the big.Rat to a big.Int and using that to calculate modulus). The main difficulty with big.Rat in the context of fq is that to get a decimal-with-radix representation (i.e. "123.4" instead of "1234/10") you use FloatString, which requires you to specify (and keep track of) the required number of digits of precision yourself. It's worth an experiment to validate if it's possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants