Evaluate different data format for speeding up reprocessing #59
Labels
enhancement
New feature request or improvement to existing functionality
funder/drl2022-2024
priority/low
Nice to have
Reprocessing is currently not so fast, given that it's feeding from JSONL files. We should consider using something that's more performant like avro, msgpack or similar.
For historical purposes, here are some benchmarks I ran sometime ago while building OONI Data:
The text was updated successfully, but these errors were encountered: