Skip to content

Commit

Permalink
[SPARK-50654][SS][TESTS][FOLLOWUP] Fix compile issue of `RocksDBState…
Browse files Browse the repository at this point in the history
…StoreCheckpointFormatV2Suite`

### What changes were proposed in this pull request?
This PR replaces `metadata.stateUniqueIds(0)` with `metadata.stateUniqueIds.head(0)` to fix the compilation issue in `RocksDBStateStoreCheckpointFormatV2Suite`. The compilation error occurred because the type of `stateUniqueIds` was changed from `stateUniqueIds: Map[Long, Array[Array[String]]] = Map.empty` to `stateUniqueIds: Option[Map[Long, Array[Array[String]]]] = None` in #49278.

### Why are the changes needed?
Fix the compilation error in GitHub Actions:

- https://github.com/apache/spark/actions/runs/12801139497/job/35690169405
- https://github.com/apache/spark/actions/runs/12801160621/job/35690189293

![image](https://github.com/user-attachments/assets/a94ab85f-9f5d-49cd-9eb6-01258b367cf4)

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions
- manual check:

```
build/sbt "sql/testOnly *RocksDBStateStoreCheckpointFormatV2Suite"
```
```
[info] Run completed in 1 minute, 33 seconds.
[info] Total number of tests run: 17
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 17, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #49523 from LuciferYang/SPARK-50654-FOLLOWUP.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
LuciferYang authored and dongjoon-hyun committed Jan 16, 2025
1 parent b6685e3 commit 0ca9633
Showing 1 changed file with 2 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -622,8 +622,8 @@ class RocksDBStateStoreCheckpointFormatV2Suite extends StreamTest
val metadata = commitLog.get(Some(0), Some(1)).map(_._2)

val versionToUniqueIdFromCommitLog = metadata.zipWithIndex.map { case (metadata, idx) =>
// Use stateUniqueIds(0) because there is only one state operator
val res2 = metadata.stateUniqueIds(0).map { uniqueIds =>
// Use stateUniqueIds.head(0) because there is only one state operator
val res2 = metadata.stateUniqueIds.head(0).map { uniqueIds =>
uniqueIds(0)
}
idx -> res2
Expand Down

0 comments on commit 0ca9633

Please sign in to comment.