Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Large File Support #243

Open
ecton opened this issue Apr 8, 2022 · 0 comments
Open

Improve Large File Support #243

ecton opened this issue Apr 8, 2022 · 0 comments
Labels
enhancement New feature or request files Relating to the `bonsaidb-files` crate, which provides large file storage.
Milestone

Comments

@ecton
Copy link
Member

ecton commented Apr 8, 2022

There are design questions about how to tackle permissions. I want to expose "service" layer which checks permissions more alike what a user of an S3-like system would expect. Currently, the only permissions are handled by BonsaiDb directly, but that does not allow building permission rules unless you know the file's ID -- very cumbersome.

This service layer would take a privileged connection to run its "backend". This type could be used on the server implementation, and would support Api types for network access.

There would need to be a duplicate API surface that allows reading/writing files from this service layer instead of directly from collections. I'm sure this can be simplified nicely behind the scenes with traits.

The other two ideas I had don't deserve as much explanation:

  • For the buffered reader, would a version that always requests the next chunk from the database in a separate thread/task be helpful? I think it needs benchmarks, and the design was complex enough I just tabled the idea.
  • A way to "tail" a file would be awesome -- PubSub events for appending to a file, and a way to "update" a Contents with new blocks.
  • More examples.

Refs #222

@ecton ecton added enhancement New feature or request files Relating to the `bonsaidb-files` crate, which provides large file storage. labels Apr 8, 2022
@ecton ecton added this to the v1.0 milestone Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request files Relating to the `bonsaidb-files` crate, which provides large file storage.
Projects
Status: Todo
Development

No branches or pull requests

1 participant