You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Aug 30, 2022. It is now read-only.
Currently the ParquetFileWriter cannot handle table with parquet columns. Due to the following reasons
Parquet has a Interval logical type which has the same meaning of Hillview Duration, but its format is too convoluted. It's essentially 3 int32 numbers representing months, days, and milliseconds. If we only use the milliseconds field there might be a precision loss.
If we save the Duration column as Double. Then currently we cannot guarantee reading back a saved table yields the same format as the original table.
One possible solution might be save as Double but also save a schema file.
The text was updated successfully, but these errors were encountered:
Currently the
ParquetFileWriter
cannot handle table with parquet columns. Due to the following reasonsInterval
logical type which has the same meaning of HillviewDuration
, but its format is too convoluted. It's essentially 3 int32 numbers representing months, days, and milliseconds. If we only use the milliseconds field there might be a precision loss.Duration
column asDouble
. Then currently we cannot guarantee reading back a saved table yields the same format as the original table.One possible solution might be save as
Double
but also save a schema file.The text was updated successfully, but these errors were encountered: