[router] More efficient handling of batch get key count limit breach #1239
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FelixGV
force-pushed
the
efficiency_improvements_for_key_count_exceeded
branch
from
c5c112a
to
eeb81b2
Compare
| setPartitionKeys(this.routerKeyMap.keySet()); | ||
|
|
||
| int keyCount = getPartitionKeys().size(); | ||
| int keyCount = keys.size(); | ||
| if (keyCount > maxKeyCount) { |
There was a problem hiding this comment.
Curious, how does the request get pass the client key count check ?
There was a problem hiding this comment.
I think that check is only implemented in (recent versions of) the FC and not in the TC? Also, the check is based on some kind of periodically refreshed metadata that could be out of sync with the backend?
There was a problem hiding this comment.
Right, I see the check only in FC but not in TC. Should we add the check in TC as well ? it will save lots of resources for router and client as well for large requests.
There was a problem hiding this comment.
At first glance I would rather not... it seems to be "not very thin" 😅 ... but we can consider it.
...ice-router/src/main/java/com/linkedin/venice/router/api/RouterExceptionAndTrackingUtils.java
Outdated
Show resolved
Hide resolved
internal/venice-client-common/src/main/java/com/linkedin/venice/D2/D2ClientUtils.java
Show resolved
Hide resolved
- In VeniceMultiKeyPath::initialize, we check the key count sooner, which is possible thanks to an earlier refactoring to the Avro deserialization code which now returns a List rather than an Iterable. - In VenicePathParser::parseResourceUri, when the batch get key count is exceeded, we mark it as a 413, and then treat it as an "expected exception" in RouterExceptionAndTrackingUtils, which suppresses the stacktrace from the logs. - In RouterExceptionAndTrackingUtils, we further improve the handling of "expected exceptions" so that not only we avoid logging the stacktrace, but we also skip filling it in in the first place. Previously, the stacktrace would be filled in and then suppressed, and filling in stacktraces can be expensive. This change should help not only batch get key count limit breaches but also other forms of quota breaches, store not found, etc. Miscellaneous: - Tweaked the logging of the build which had a bookkeeping error in the time taken to run each test class. - Added a null check in D2ClientUtils::shutdownClient since it caused some integration tests to fail during their tear down.
of tests use just a ZkServerWrapper instead of a VeniceClusterWrapper.
FelixGV
force-pushed
the
efficiency_improvements_for_key_count_exceeded
branch
from
eeb81b2
to
48938b4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In VeniceMultiKeyPath::initialize, we check the key count sooner, which is possible thanks to an earlier refactoring to the Avro deserialization code which now returns a List rather than an Iterable.
In VenicePathParser::parseResourceUri, when the batch get key count is exceeded, we mark it as a 413, and then treat it as an "expected exception" in RouterExceptionAndTrackingUtils, which suppresses the stacktrace from the logs.
In RouterExceptionAndTrackingUtils, we further improve the handling of "expected exceptions" so that not only we avoid logging the stacktrace, but we also skip filling it in in the first place. Previously, the stacktrace would be filled in and then suppressed, and filling in stacktraces can be expensive. This change should help not only batch get key count limit breaches but also other forms of quota breaches, store not found, etc.
Miscellaneous:
Tweaked the logging of the build which had a bookkeeping error in the time taken to run each test class.
Added a null check in D2ClientUtils::shutdownClient since it caused some integration tests to fail during their tear down.
How was this PR tested?
GHCI.
Does this PR introduce any user-facing changes?
Clients will receive the 413 status code now instead of 400 if they breach the batch get key count limit.