Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
snapshot: delegate reading to arrow readers
We were previously copying all snapshot part bytes into a slice that was then passed to arrow readers. This was an unnecessary indirection and resulted in extra allocations on startup. Note that parquet bytes are still fully copied since we cannot read those lazily (underlying file is closed after reading from snapshot). Benchmark results on recovery from snapshot only: ``` goos: darwin goarch: arm64 pkg: github.com/polarsignals/frostdb │ benchmain │ benchnew │ │ sec/op │ sec/op vs base │ Replay-12 1.681 ± 5% 2.545 ± 4% +51.41% (p=0.002 n=6) │ benchmain │ benchnew │ │ B/op │ B/op vs base │ Replay-12 2.327Gi ± 0% 2.278Gi ± 0% -2.11% (p=0.002 n=6) │ benchmain │ benchnew │ │ allocs/op │ allocs/op vs base │ Replay-12 20.81M ± 0% 20.79M ± 0% -0.10% (p=0.002 n=6) ```
- Loading branch information