Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Configure compression level/speed #81

Open
technillogue opened this issue Mar 15, 2023 · 6 comments
Open

[FEA] Configure compression level/speed #81

technillogue opened this issue Mar 15, 2023 · 6 comments

Comments

@technillogue
Copy link

Is your feature request related to a problem? Please describe.
I'm using nvCOMP to load data from disk to GPU, and don't care about compression but do care about getting the best compression ratio possible.

Describe the solution you'd like
CompressionConfig has settings for each algorithm, for example gdeflate could max_hash_checks and lazy vs greedy matching.

Describe alternatives you've considered
If this is available through the low-level API, better documentation would be helpful.

@eschmidt-nvidia
Copy link
Collaborator

Hi technillogue,

In the case of GDeflate we do have a "high compression" mode. This is documented in include/nvcomp/gdeflate.h, and was designed for such use cases as yours where compression speed is less of a concern.

We're also considering adding similar modes to zSTD in an upcoming release.

-Eric

@github-actions
Copy link

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@technillogue
Copy link
Author

Setting anything but 0 for Gdeflate gives:

Invalid format_opts.algo value (high compression option (1) not currently supported)
Invalid format_opts.algo value (entropy only option (2) not currently supported)

@github-actions
Copy link

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@eschmidt-nvidia
Copy link
Collaborator

We changed the HLIF for 3.0 to call the LLIF. Now the other algorithms should work for the GDeflate HLIF.

Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants