Compares different compression algorithms on floating point numbers.
Tested algorithms:
- Facebook's gorilla
- q_compress - alternative codec with delta-encoding. This ahieves a higher compression rate against gzip, snappy, and zstd. Post here
- Facebook's zstd
- snappy
- zfp
- lz4
To Explore:
- compressed_vec compresses data and allows processing directly on the compressed representation
- fpzip - memory-efficient algorithm from 2006
- gibbon
- fpzip
To determine the (approximately) best approach to encode floating point numbers, we record the following evaluation measures:
- Compression ratio (old file size / new file size)
- Encoding speed
- Decoding speed