SNOW-1357377 Add request Id in all streaming ingest APIs #759
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sfc-gh-rcheng
approved these changes
Jun 7, 2024
| .setHost(host) | ||
| .setPort(port) | ||
| .setPath(endPoint) | ||
| .setParameter(REQUEST_ID, UUID.randomUUID().toString()) |
There was a problem hiding this comment.
should we log this requestid? might be too many logs, but it will be useful to know where/when the request was created in the ingest sdk
There was a problem hiding this comment.
Valid point. added requestId in above debug log! we dont need everywhere IMO. this requestId is mainly for internal debugging.
sfc-gh-japatel
force-pushed
the
japatel-SNOW-1357377-requestid-qp
branch
from
0dee895
to
d47b5c1
Compare
| @@ -678,12 +678,23 @@ public HttpGet generateHistoryRangeRequest( | |||
| */ | |||
| public HttpPost generateStreamingIngestPostRequest( | |||
| String payload, String endPoint, String message) { | |||
| LOGGER.debug("Generate Snowpipe streaming request: endpoint={}, payload={}", endPoint, payload); | |||
| final String requestId = UUID.randomUUID().toString(); | |||
There was a problem hiding this comment.
My assumption is that the request id will remain the same with retry?
There was a problem hiding this comment.
thats true!
| @@ -493,7 +493,6 @@ ChannelsStatusResponse getChannelsStatus( | |||
| .collect(Collectors.toList()); | |||
| request.setChannels(requestDTOs); | |||
| request.setRole(this.role); | |||
| request.setRequestId(this.flushService.getClientPrefix() + "_" + counter.getAndIncrement()); | |||
There was a problem hiding this comment.
We should remove request_id in the request as well
There was a problem hiding this comment.
you mean in the POJO?
There was a problem hiding this comment.
Yes, in the request class
sfc-gh-japatel
force-pushed
the
japatel-SNOW-1357377-requestid-qp
branch
from
d47b5c1
to
168f3d3
Compare
sfc-gh-japatel
force-pushed
the
japatel-SNOW-1357377-requestid-qp
branch
from
5debf91
to
ba47eb2
Compare
sfc-gh-tzhang
approved these changes
Jun 12, 2024
| import static net.snowflake.ingest.utils.Constants.RESPONSE_SUCCESS; | ||
| import static net.snowflake.ingest.utils.Constants.ROLE; | ||
| import static net.snowflake.ingest.utils.Constants.USER; | ||
| import static net.snowflake.ingest.utils.Constants.*; |
sfc-gh-kgaputis
pushed a commit
to sfc-gh-kgaputis/snowflake-ingest-java
that referenced
this pull request
Sep 12, 2024
…#759) * SNOW-1357377 Add request Id in all streaming ingest APIs * Fix tests and log requestId * Format * Remove request_id and no star imports
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Along with this, our response should also include requestId but that can come later since they can still rely on offsetToken API. (Wont work for other APIs but there hasn't been any request)
This is for internal logging purposes.