Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option for SMB joins to fall back to regular joins #5266

Open
clairemcginty opened this issue Feb 21, 2024 · 0 comments
Open

Add option for SMB joins to fall back to regular joins #5266

clairemcginty opened this issue Feb 21, 2024 · 0 comments
Labels
enhancement New feature or request SMB

Comments

@clairemcginty
Copy link
Contributor

Add an option, either pipeline option or API method param, to instruct Scio to fall back to a non-SMB join if it can't perform an SMB join (due to incompatible hash types, missing metadata, etc).

SMB does all this validation at job submission time -- after sc.run() but before the job is submitted to dataflow -- so we should be able to catch exceptions at that time.

@clairemcginty clairemcginty added enhancement New feature or request SMB labels Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request SMB
Projects
None yet
Development

No branches or pull requests

1 participant