Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow timeout for async DoFn #5201

Closed
wants to merge 4 commits into from
Closed

Allow timeout for async DoFn #5201

wants to merge 4 commits into from

Conversation

RustedBones
Copy link
Contributor

No description provided.

Copy link

codecov bot commented Jan 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9f4603e) 63.05% compared to head (862b054) 62.56%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5201      +/-   ##
==========================================
- Coverage   63.05%   62.56%   -0.50%     
==========================================
  Files         300      301       +1     
  Lines       10947    10807     -140     
  Branches      740      764      +24     
==========================================
- Hits         6903     6761     -142     
- Misses       4044     4046       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@kellen kellen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a speculated fix to the hanging pipelines?

@RustedBones
Copy link
Contributor Author

Yes. One user reported that setting the timeout will at least retry the bundle instead of hanging forever.
The root cause for the future that never completes is however still unknown.

It would be also nice to set the timeout as an option for the GrpcDoFn

@@ -87,7 +110,7 @@ public void onFailure(Throwable t) {
}
}
},
MoreExecutors.directExecutor());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MoreExecutors.directExecutor() doc says callback may not be executed if the exception terminates the executing thread

In other cases, no code will catch the exception, and it may terminate whichever thread happens to trigger the execution.

I've changed to use another executor that also completes the future when if it refuses to execute the callback. This is the strategy used by guava when transforming futures.

@RustedBones
Copy link
Contributor Author

Closing for now in favor of #5209 will reconsider if not solved

@RustedBones RustedBones closed this Feb 1, 2024
@RustedBones RustedBones deleted the async-do-fn-timeout branch June 10, 2024 11:46
@RustedBones RustedBones restored the async-do-fn-timeout branch December 11, 2024 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants