From edfc54a2abd3bfba1ff9dadcecaec72de801ec9c Mon Sep 17 00:00:00 2001 From: King'ori Maina Date: Thu, 17 Feb 2022 11:07:29 +0200 Subject: [PATCH] Add project documentation and configuration --- CHANGELOG.md | 5 ++ CODEOWNERS | 1 + LICENSE | 21 +++++++++ README.md | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 152 insertions(+), 1 deletion(-) create mode 100644 CHANGELOG.md create mode 100644 CODEOWNERS create mode 100644 LICENSE diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..6f2ebfc --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,5 @@ +# Changelog + +## 20220217-0c4e97a + +* Package latest version of Redis Roaring into Docker image. diff --git a/CODEOWNERS b/CODEOWNERS new file mode 100644 index 0000000..f83760b --- /dev/null +++ b/CODEOWNERS @@ -0,0 +1 @@ +* @Intellection/SRE diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..bc67fc0 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2022 Zappi + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 45796b4..75c9654 100644 --- a/README.md +++ b/README.md @@ -1 +1,125 @@ -# docker-redis-roaring \ No newline at end of file +# Redis Roaring + +Roaring Bitmaps for Redis + +## Intro + +This project uses the [CRoaring](https://github.com/RoaringBitmap/CRoaring) library to implement roaring bitmap commands for Redis. +These commands can have the same performance as redis' native bitmaps for *O(1)* operations and be [up to 8x faster](#performance) for *O(N)* +calls, according to microbenchmarks, while consuming less memory than their uncompressed counterparts (benchmark pending). + +Pull requests are welcome. + +## Dependencies + +- CRoaring (bitmap compression library used by this redis module) +- cmake (build tools needed for compiling the source code) +- redis (server needed for integration tests) +- hiredis (redis client library needed for performance tests) + +### Known Limitations + +- This library only works with [32-bit integeres](https://github.com/RoaringBitmap/CRoaring/issues/1) (e.g. counting numbers up to 4294967296) + +## Getting started + +``` +git clone https://github.com/aviggiano/redis-roaring.git +cd redis-roaring/ +configure.sh +cd dist +./redis-server ./redis.conf +``` + +then you can open another terminal and use `./redis-cli` to connect to the redis server + +## Docker + +It is also possible to run this project as a docker container. + +```bash +docker run -p 6379:6379 aviggiano/redis-roaring:latest +``` + +## Tests + +Run the `test.sh` script for unit tests, integration tests and performance tests. +The performance tests can take a while, since they run on a real dataset of integer values. + +## API + +The following operations are supported + +- `R.SETBIT` (same as [SETBIT](https://redis.io/commands/setbit)) +- `R.GETBIT` (same as [GETBIT](https://redis.io/commands/getbit)) +- `R.BITOP` (same as [BITOP](https://redis.io/commands/bitop)) +- `R.BITCOUNT` (same as [BITCOUNT](https://redis.io/commands/bitcount) without `start` and `end` parameters) +- `R.BITPOS` (same as [BITPOS](https://redis.io/commands/bitpos) without `start` and `end` parameters) +- `R.SETINTARRAY` (create a roaring bitmap from an integer array) +- `R.GETINTARRAY` (get an integer array from a roaring bitmap) +- `R.SETBITARRAY` (create a roaring bitmap from a bit array string) +- `R.GETBITARRAY` (get a bit array string from a roaring bitmap) + +Additional commands + +- `R.APPENDINTARRAY` (append integers to a roaring bitmap) +- `R.RANGEINTARRAY` (get an integer array from a roaring bitmap with `start` and `end`, so can implements paging) +- `R.SETRANGE` (set or append integer range to a roaring bitmap) +- `R.SETFULL` (fill up a roaring bitmap in integer) +- `R.STAT` (get statistical information of a roaring bitmap) +- `R.OPTIMIZE` (optimize a roaring bitmap) +- `R.MIN` (get minimal integer from a roaring bitmap, if key is not exists or bitmap is empty, return -1) +- `R.MAX` (get maximal integer from a roaring bitmap, if key is not exists or bitmap is empty, return -1) +- `R.DIFF` (get difference between two bitmaps) + +Missing commands: + +- `R.BITFIELD` (same as [BITFIELD](https://redis.io/commands/bitfield)) + +## API Example + +``` +$ redis-cli +# create a roaring bitmap with numbers from 1 to 99 +127.0.0.1:6379> R.SETRANGE test 1 100 + +# get all the numbers as an integer array +127.0.0.1:6379> R.GETINTARRAY test + +# fill up the roaring bitmap +# because you need 2^32*4 bytes memory and a very long time +127.0.0.1:6379> R.SETFULL full + +# use `R.RANGEINTARRAY` to get numbers from 100 to 1000 +127.0.0.1:6379> R.RANGEINTARRAY full 100 1000 + +# append numbers to an existing roaring bitmap +127.0.0.1:6379> R.APPENDINTARRAY test 111 222 3333 456 999999999 9999990 +``` + +## Performance + +Tested using CRoaring's `census1881` dataset on the travis build [552223545](https://travis-ci.org/aviggiano/redis-roaring/builds/552223545): + +| OP | TIME/OP (us) | ST.DEV. (us) | +| ------------ | ------------ | ------------ | +| R.SETBIT | 31.83 | 71.85 | +| SETBIT | 30.52 | 74.83 | +| R.GETBIT | 30.29 | 46.99 | +| GETBIT | 29.30 | 38.39 | +| R.BITCOUNT | 30.38 | 0.04 | +| BITCOUNT | 169.46 | 0.95 | +| R.BITPOS | 30.62 | 0.08 | +| BITPOS | 55.06 | 0.77 | +| R.BITOP NOT | 103.90 | 1.71 | +| BITOP NOT | 328.14 | 5.81 | +| R.BITOP AND | 40.66 | 0.47 | +| BITOP AND | 433.52 | 7.98 | +| R.BITOP OR | 57.01 | 2.33 | +| BITOP OR | 425.10 | 7.68 | +| R.BITOP XOR | 60.50 | 2.77 | +| BITOP XOR | 415.21 | 7.51 | +| R.MIN | 27.16 | 0.08 | +| MIN | 24.57 | 0.18 | +| R.MAX | 24.62 | 0.04 | +| MAX | 25.85 | 0.03 |