-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[router] More efficient handling of batch get key count limit breach #1239
base: main
Are you sure you want to change the base?
[router] More efficient handling of batch get key count limit breach #1239
Conversation
c5c112a
to
eeb81b2
Compare
setPartitionKeys(this.routerKeyMap.keySet()); | ||
|
||
int keyCount = getPartitionKeys().size(); | ||
int keyCount = keys.size(); | ||
if (keyCount > maxKeyCount) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious, how does the request get pass the client key count check ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that check is only implemented in (recent versions of) the FC and not in the TC? Also, the check is based on some kind of periodically refreshed metadata that could be out of sync with the backend?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, I see the check only in FC but not in TC. Should we add the check in TC as well ? it will save lots of resources for router and client as well for large requests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At first glance I would rather not... it seems to be "not very thin" 😅 ... but we can consider it.
...ice-router/src/main/java/com/linkedin/venice/router/api/RouterExceptionAndTrackingUtils.java
Outdated
Show resolved
Hide resolved
internal/venice-client-common/src/main/java/com/linkedin/venice/D2/D2ClientUtils.java
Show resolved
Hide resolved
- In VeniceMultiKeyPath::initialize, we check the key count sooner, which is possible thanks to an earlier refactoring to the Avro deserialization code which now returns a List rather than an Iterable. - In VenicePathParser::parseResourceUri, when the batch get key count is exceeded, we mark it as a 413, and then treat it as an "expected exception" in RouterExceptionAndTrackingUtils, which suppresses the stacktrace from the logs. - In RouterExceptionAndTrackingUtils, we further improve the handling of "expected exceptions" so that not only we avoid logging the stacktrace, but we also skip filling it in in the first place. Previously, the stacktrace would be filled in and then suppressed, and filling in stacktraces can be expensive. This change should help not only batch get key count limit breaches but also other forms of quota breaches, store not found, etc. Miscellaneous: - Tweaked the logging of the build which had a bookkeeping error in the time taken to run each test class. - Added a null check in D2ClientUtils::shutdownClient since it caused some integration tests to fail during their tear down.
of tests use just a ZkServerWrapper instead of a VeniceClusterWrapper.
eeb81b2
to
48938b4
Compare
In VeniceMultiKeyPath::initialize, we check the key count sooner, which is possible thanks to an earlier refactoring to the Avro deserialization code which now returns a List rather than an Iterable.
In VenicePathParser::parseResourceUri, when the batch get key count is exceeded, we mark it as a 413, and then treat it as an "expected exception" in RouterExceptionAndTrackingUtils, which suppresses the stacktrace from the logs.
In RouterExceptionAndTrackingUtils, we further improve the handling of "expected exceptions" so that not only we avoid logging the stacktrace, but we also skip filling it in in the first place. Previously, the stacktrace would be filled in and then suppressed, and filling in stacktraces can be expensive. This change should help not only batch get key count limit breaches but also other forms of quota breaches, store not found, etc.
Miscellaneous:
Tweaked the logging of the build which had a bookkeeping error in the time taken to run each test class.
Added a null check in D2ClientUtils::shutdownClient since it caused some integration tests to fail during their tear down.
How was this PR tested?
GHCI.
Does this PR introduce any user-facing changes?
Clients will receive the 413 status code now instead of 400 if they breach the batch get key count limit.