[Feature Request] Restricting maximum memory usage in lgraph_import #83
hjk41
started this conversation in
Feature Request
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Currently the bucket_memory_limit option in lgraph_import is just a hint. If there is severe data skew (as in the case of twitter-2010), one bucket can be much larger than this limit, causing OOM errors. We should set a hard limit on the memory budget.
Beta Was this translation helpful? Give feedback.
All reactions