-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BugFix] isLocalBucketShuffleJoin return wrong result #51954
Conversation
57aa058
to
bc90348
Compare
if (root instanceof JoinNode) { | ||
JoinNode joinNode = (JoinNode) root; | ||
if (joinNode.isLocalHashBucket()) { | ||
isRightOrFullBucketShuffle = joinNode.getJoinOp().isFullOuterJoin() || joinNode.getJoinOp().isRightJoin(); | ||
return true; | ||
hasBucketShuffle = joinNode.getJoinOp().isFullOuterJoin() || joinNode.getJoinOp().isRightJoin(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where willisRightOrFullBucketShuffle
be set to true, maybe as follows?
hasBucketShuffle = joinNode.getJoinOp().isFullOuterJoin() || joinNode.getJoinOp().isRightJoin(); | |
hasBucketShuffle = true; | |
isRightOrFullBucketShuffle ||= joinNode.getJoinOp().isFullOuterJoin() || joinNode.getJoinOp().isRightJoin(); |
bc90348
to
292714e
Compare
Signed-off-by: satanson <[email protected]>
292714e
to
6a91444
Compare
[FE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
Quality Gate passedIssues Measures |
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
@Mergifyio backport branch-3.3 |
@Mergifyio backport branch-3.2 |
@Mergifyio backport branch-3.1 |
@Mergifyio backport branch-3.0 |
@Mergifyio backport branch-2.5 |
✅ Backports have been created
|
✅ Backports have been created
|
✅ Backports have been created
|
✅ Backports have been created
|
✅ Backports have been created
|
Signed-off-by: satanson <[email protected]> (cherry picked from commit 1077a6d)
Signed-off-by: satanson <[email protected]> (cherry picked from commit 1077a6d)
Signed-off-by: satanson <[email protected]> (cherry picked from commit 1077a6d) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/qe/scheduler/dag/ExecutionFragment.java
Signed-off-by: satanson <[email protected]> (cherry picked from commit 1077a6d) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/qe/scheduler/dag/ExecutionFragment.java
Signed-off-by: satanson <[email protected]> (cherry picked from commit 1077a6d) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/qe/scheduler/dag/ExecutionFragment.java
… (#51977) Co-authored-by: satanson <[email protected]>
… (#51978) Co-authored-by: satanson <[email protected]>
Why I'm doing:
No data sent from the right side of Bucket Shuffle Right Join Build (ExchangeSink)
First, the Bucket seqs 3, 5, 7, 8, 9 of the left table in the Bucket shuffle join were pruned.
Generally, when bucket seqs are pruned, the instance id on the receiver side in the channel map is recorded as -1 to indicate no data is being sent. However, for right outer join, the NULL values from the build side need to be output, so for such cases, any valid instance id must be assigned to the pruned bucket seq.
2. The issue where NULL values on the HashJoin Build side need to be output, and assigning any instance id to the pruned bucketSeqs
Cases where NULL values on the build side need to be output:
a. Right join, full join, Right Anti JOIN
b. Null-safe-eq join:
<=>
. It seems this is not currently considered in the logic, further confirmation is required.Determine whether NULL values on the build side need to be output by
isRightOrFullBucketShuffleFragment
.isRightOrFullBucketShuffleFragment
is expected to betrue
, but is actuallyfalse
.Top-Down Determination of
isRightOrFullBucketShuffle
: depends on first visitedIn the erroneous plan, there are three JOINS, and the above bucket shuffle left join is the first visited, directly determining
isRightOrFullBucketShuffle = false
.What I'm doing:
Fix function isLocalBucketShuffleJoin to return correct result.
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: