Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support random access api in VFS #158

Open
oeway opened this issue Jul 24, 2021 · 1 comment
Open

Support random access api in VFS #158

oeway opened this issue Jul 24, 2021 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@oeway
Copy link

oeway commented Jul 24, 2021

Currently, the VFS has api such as readfile and writefile assumes that one would read and write files as a whole. However, that cannot work for really large files, it would be nice to support byte offset parameters which will then be translated into HTTP range request for remote file systems.

E.g. we can add position and length similar to the fs module in nodejs: https://nodejs.org/api/fs.html#fs_fs_read_fd_buffer_offset_length_position_callback

I would assume that it's easy to implement that for readfile, how about write file, can we also support appending content to a specific position of a file?

@andersevenrud andersevenrud added the enhancement New feature or request label Jul 24, 2021
@andersevenrud
Copy link
Member

It should be sufficient to send the standard HTTP range headers to the readfile endpoint.

On the server-end of things we're working with streams, and ranges is supported.

@andersevenrud andersevenrud added good first issue Good for newcomers help wanted Extra attention is needed labels Jul 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants