Skip to content

Commit

Permalink
[SPARK-50904][SQL] Fix collation expression walker query execution
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
Changing when we collect results in `CollationExpressionWalkerSuite` on borders of changing session default collation.

### Why are the changes needed?
Because we are analyzing the query and then setting a different default collation, this leads to problems for runtime replaceable expressions which will only be resolved when we call `collect` and by then the session default collation is changed. This is blocking the change in #49576.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
This is a test-only change.

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #49586 from stefankandic/fixExpressionWalker.

Authored-by: Stefan Kandic <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
  • Loading branch information
stefankandic authored and MaxGekk committed Jan 21, 2025
1 parent 98f2767 commit dabd212
Showing 1 changed file with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -740,10 +740,10 @@ class CollationExpressionWalkerSuite extends SparkFunSuite with SharedSparkSessi
for (funInfo <- funInfos.filter(f => !toSkip.contains(f.getName))) {
for (query <- "> .*;".r.findAllIn(funInfo.getExamples).map(s => s.substring(2))) {
try {
val resultUTF8 = sql(query)
val resultUTF8 = sql(query).collect()
withSQLConf(SqlApiConf.DEFAULT_COLLATION -> "UTF8_LCASE") {
val resultUTF8Lcase = sql(query)
assert(resultUTF8.collect() === resultUTF8Lcase.collect())
val resultUTF8Lcase = sql(query).collect()
assert(resultUTF8 === resultUTF8Lcase)
}
} catch {
case e: SparkRuntimeException => assert(e.getCondition == "USER_RAISED_EXCEPTION")
Expand Down

0 comments on commit dabd212

Please sign in to comment.