You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current bao block size is = the chunk size = 1024 bytes.
That the size of the outboard will be 2 32 byte hashes per chunk, so 64 bytes per chunk, so 1/16 of the data size.
Unfortunately that means that it is not possible to keep the outboard in memory when sending large files from small devices. E.g. sending a 32 gb disk image would require a 2gb outboard, which is a bit much for e.g. a raspberry pi with 4 and impossible for a rpi with 1 gb of ram.
One option would be to persist the outboards on disk, but that is a can of worms. Also, 1024 byte chunks are less than optimal also for transfer speed.
The easiest thing to do would be to have a larger block size. E.g. 64k -> outboard = 1/1024 of data => no problem for the above use case.
There is an issue on the bao repo about larger chunk groups. Either get this merged (preferably), or fork bao and add it ourselves...
We are currently using a chunk group log of 4 (chunk group size 16kb) via the abao crate. We will continue to do so in the bao-tree crate #894 where this is runtime adjustable.
The current bao block size is = the chunk size = 1024 bytes.
That the size of the outboard will be 2 32 byte hashes per chunk, so 64 bytes per chunk, so 1/16 of the data size.
Unfortunately that means that it is not possible to keep the outboard in memory when sending large files from small devices. E.g. sending a 32 gb disk image would require a 2gb outboard, which is a bit much for e.g. a raspberry pi with 4 and impossible for a rpi with 1 gb of ram.
One option would be to persist the outboards on disk, but that is a can of worms. Also, 1024 byte chunks are less than optimal also for transfer speed.
The easiest thing to do would be to have a larger block size. E.g. 64k -> outboard = 1/1024 of data => no problem for the above use case.
There is an issue on the bao repo about larger chunk groups. Either get this merged (preferably), or fork bao and add it ourselves...
oconnor663/bao#34
The text was updated successfully, but these errors were encountered: