Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
# Description Addresses #4393. This PR adds a Binary Fuse Filter library and switches BucketListDB to use the new filter. The Binary Fuse Filter library is based on [this](https://github.com/FastFilter/xor_singleheader) library, but I ended up essentially rewriting it in C++. The original library had a bug where some filter bytes were non-deterministic. While this did not affect correctness or the false positive rate, we need determinism because State Archival will hash these filters into the ledger. Additionally, the library did not support 32 bit filters, which are required for State Archival. Currently a draft until the (non protocol breaking) associated XDR changes are merged: stellar/stellar-xdr#195 While Binary Fuse Filters are necessary for State Archival, they also significantly improve BucketListDB. Compared to the older bloom filter implementation, Bucket indexing time has decreased by 67% (which is blocking on startup in some cases), decreases index memory size by 20%, and reduces redundant disk reads by ~130x. # Checklist - [x] Reviewed the [contributing](https://github.com/stellar/stellar-core/blob/master/CONTRIBUTING.md#submitting-changes) document - [x] Rebased on top of master (no merge commits) - [x] Ran `clang-format` v8.0.0 (via `make format` or the Visual Studio extension) - [x] Compiles - [x] Ran all tests - [ ] If change impacts performance, include supporting evidence per the [performance document](https://github.com/stellar/stellar-core/blob/master/performance-eval/performance-eval.md)
- Loading branch information