You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a follow-up for #15448, a corner case that was not addressed in that feature.
Assume there are many copies (fast copy versions) of a virtual map, and some copy is not released for some reason:
Since the copy is not released, it cannot be flushed or merged
The next copy after that can be merged, though, and the next after next, too
Copies are merged till at some point (some version) a copy is so large that its size exceeds flush threshold, and it can't be merged any longer
The next copy after that can still be merged
And so on
So in the end the list of copies in the virtual pipeline will look like this (newest to oldest):
mutable copy
immutable copy version N, contains changes from versions N - X + 1 to N
immutable copy version N - X, contains changes from version N - 2X + 1 to N - X
...
immutable copy version M, which is never released
The problem above is that all these intermediate copies may contain lots of obsolete mutations. Current in-memory mode implementation is that garbage collection is never run for these copies, but it should be.
This ticket is to improve in-memory mode for virtual maps:
Every copy is first checked if its size exceeds flush threshold
If so, garbage collection is run for this copy, otherwise no GC
Then the size of the copy is checked again. If it still exceeds the threshold, the copy is flushed
Otherwise it is merged to the next version
The text was updated successfully, but these errors were encountered:
This is a follow-up for #15448, a corner case that was not addressed in that feature.
Assume there are many copies (fast copy versions) of a virtual map, and some copy is not released for some reason:
So in the end the list of copies in the virtual pipeline will look like this (newest to oldest):
The problem above is that all these intermediate copies may contain lots of obsolete mutations. Current in-memory mode implementation is that garbage collection is never run for these copies, but it should be.
This ticket is to improve in-memory mode for virtual maps:
The text was updated successfully, but these errors were encountered: