From 0ca96333763786ad1409b5ecd0206fe9564739ef Mon Sep 17 00:00:00 2001 From: yangjie01 Date: Wed, 15 Jan 2025 19:46:48 -0800 Subject: [PATCH] [SPARK-50654][SS][TESTS][FOLLOWUP] Fix compile issue of `RocksDBStateStoreCheckpointFormatV2Suite` ### What changes were proposed in this pull request? This PR replaces `metadata.stateUniqueIds(0)` with `metadata.stateUniqueIds.head(0)` to fix the compilation issue in `RocksDBStateStoreCheckpointFormatV2Suite`. The compilation error occurred because the type of `stateUniqueIds` was changed from `stateUniqueIds: Map[Long, Array[Array[String]]] = Map.empty` to `stateUniqueIds: Option[Map[Long, Array[Array[String]]]] = None` in https://github.com/apache/spark/pull/49278. ### Why are the changes needed? Fix the compilation error in GitHub Actions: - https://github.com/apache/spark/actions/runs/12801139497/job/35690169405 - https://github.com/apache/spark/actions/runs/12801160621/job/35690189293 ![image](https://github.com/user-attachments/assets/a94ab85f-9f5d-49cd-9eb6-01258b367cf4) ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions - manual check: ``` build/sbt "sql/testOnly *RocksDBStateStoreCheckpointFormatV2Suite" ``` ``` [info] Run completed in 1 minute, 33 seconds. [info] Total number of tests run: 17 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 17, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed. ``` ### Was this patch authored or co-authored using generative AI tooling? No Closes #49523 from LuciferYang/SPARK-50654-FOLLOWUP. Authored-by: yangjie01 Signed-off-by: Dongjoon Hyun --- .../state/RocksDBStateStoreCheckpointFormatV2Suite.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala index b5fc88130ef6b..d44e2742ad001 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala @@ -622,8 +622,8 @@ class RocksDBStateStoreCheckpointFormatV2Suite extends StreamTest val metadata = commitLog.get(Some(0), Some(1)).map(_._2) val versionToUniqueIdFromCommitLog = metadata.zipWithIndex.map { case (metadata, idx) => - // Use stateUniqueIds(0) because there is only one state operator - val res2 = metadata.stateUniqueIds(0).map { uniqueIds => + // Use stateUniqueIds.head(0) because there is only one state operator + val res2 = metadata.stateUniqueIds.head(0).map { uniqueIds => uniqueIds(0) } idx -> res2