About I use hibernate multithreaded queries simultaneously happened: "Illegal pop () with non - matching JdbcValuesSourceProcessingState" exception

[Sparkel-wsl]

Describe the bug

我有个任务是需要从数据库中一次性查询多张不同的表，由于数据量比较大这个任务比较耗时，因此我想到使用多线程来分同时查询不同的表以此来减少任务执行时间。
但是在执行任务的时候收到了一个错误："Illegal pop() with non-matching JdbcValuesSourceProcessingState".我了解到是因为并发查询使用了同一个Session导致的，
在 #32790 这个问题中提到过使用 Mutiny.SessionFactory.openSession()可以解决，但是这么做并不能为我组装分页数据。
希望能提供一个其它的解决办法。

I have a task that needs to query multiple different tables from the database at one time. Due to the large amount of data, this task is time-consuming, so I think of using multiple threads to query different tables simultaneously to reduce the task execution time.
But received an error when a mission: "Illegal pop () with non - matching JdbcValuesSourceProcessingState". I learned that it was caused by concurrent queries using the same Session,
At #32790 this problem mentioned in the use of Mutiny. SessionFactory. OpenSession () can be solved, but can do this with no assembly paging data for me.
Hope to provide an alternative solution.

Expected behavior

顺序完成查询任务
Complete the query tasks in sequence

Actual behavior

抛出了异常：Illegal pop() with non-matching JdbcValuesSourceProcessingState
An exception was receive:Illegal pop() with non-matching JdbcValuesSourceProcessingState

java.util.concurrent.CompletionException: java.lang.IllegalStateException: Illegal pop() with non-matching JdbcValuesSourceProcessingState

at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
at java.base/java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1498)
at java.base/java.util.concurrent.CompletableFuture$CoCompletion.tryFire(CompletableFuture.java:1219)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
at io.smallrye.context.CompletableFutureWrapper.lambda$new$0(CompletableFutureWrapper.java:37)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1810)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:581)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1512)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)

Caused by: java.lang.IllegalStateException: Illegal pop() with non-matching JdbcValuesSourceProcessingState
at org.hibernate.sql.results.spi.LoadContexts.deregister(LoadContexts.java:46)
at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:209)
at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:33)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.doExecuteQuery(JdbcSelectExecutorStandardImpl.java:361)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.executeQuery(JdbcSelectExecutorStandardImpl.java:168)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.list(JdbcSelectExecutorStandardImpl.java:93)
at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:31)
at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.lambda$new$0(ConcreteSqmSelectQueryPlan.java:110)
at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.withCacheableSqmInterpretation(ConcreteSqmSelectQueryPlan.java:303)
at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.performList(ConcreteSqmSelectQueryPlan.java:244)
at org.hibernate.query.sqm.internal.QuerySqmImpl.doList(QuerySqmImpl.java:518)
at org.hibernate.query.spi.AbstractSelectionQuery.list(AbstractSelectionQuery.java:367)
at org.hibernate.query.Query.getResultList(Query.java:119)
at io.quarkus.hibernate.orm.panache.common.runtime.CommonPanacheQueryImpl.list(CommonPanacheQueryImpl.java:280)
at io.quarkus.hibernate.orm.panache.runtime.PanacheQueryImpl.list(PanacheQueryImpl.java:149)
at com.dl.ExampleResourceTest.lambda$concurrentQuery$2(ExampleResourceTest.java:71)
at io.smallrye.context.impl.wrappers.SlowContextualRunnable.run(SlowContextualRunnable.java:19)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:581)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
... 4 more

How to Reproduce?

提供一个简单的示例，就模拟多线程并发查询的情况。

Provides a simple example to simulate the situation of multi-threaded concurrent queries.

CompletableFuture<Void> selectUser = managedExecutor.runAsync(() -> {
            int pageCount = 1;
            for (int i = 0; i < pageCount; i++) {
                PanacheQuery<UserEntity> pageInfo = userDao.find("clazz = 2").page(i,1);
                pageCount = pageInfo.pageCount();
                System.out.println(pageInfo.list());
            }
            System.out.println("查询完成");
        });
        CompletableFuture<Void> selectOrder = managedExecutor.runAsync(() -> {
            int pageCount = 1;
            for (int i = 0; i < pageCount; i++) {
                PanacheQuery<OrderEntity> pageInfo = orderDao.find("skuId = 2").page(i,1);
                pageCount = pageInfo.pageCount();
                System.out.println(pageInfo.list());
            }
            System.out.println("查询完成");
        });

        CompletableFuture.allOf(selectUser,selectOrder).join();
        System.out.println("所有查询已完成");

Output of uname -a or ver

Microsoft Windows [版本 10.0.19045.3570]

Output of java -version

openjdk version "17.0.6" 2023-01-17 OpenJDK Runtime Environment GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13) OpenJDK 64-Bit Server VM GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13, mixed mode, sharing)

Quarkus version or git rev

3.2.0.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c) Maven home: D:\apache-maven-3.9.2 Java version: 17.0.6, vendor: GraalVM Community, runtime: D:\graalvm-ce-java17-22.3.1 Default locale: zh_CN, platform encoding: GBK OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Additional information

No response

[quarkus-bot[bot]]

/cc @gsmet (hibernate-orm), @yrodiere (hibernate-orm)

[quarkus-bot[bot]]

/cc @gsmet (hibernate-orm), @yrodiere (hibernate-orm)

[yrodiere]

Please don't use bug reports to ask questions. This is what discussions are for.

You cannot use the same session from multiple threads, and there's no way around that.

Just open a separate transaction in each thread, and that will lead to using a separate session in each thread, and all will be fine.

With Hibernate ORM/JPA you'd do this (won't work for you):

CompletableFuture<Void> selectUser = managedExecutor.runAsync(() -> QuarkusTransaction.requiringNew().run(() -> {
            int pageCount = 1;
            for (int i = 0; i < pageCount; i++) {
                PanacheQuery<UserEntity> pageInfo = userDao.find("clazz = 2").page(i,1);
                pageCount = pageInfo.pageCount();
                System.out.println(pageInfo.list());
            }
            System.out.println("查询完成");
        }));
        CompletableFuture<Void> selectOrder = managedExecutor.runAsync(() -> QuarkusTransaction.requiringNew().run(() -> {
            int pageCount = 1;
            for (int i = 0; i < pageCount; i++) {
                PanacheQuery<OrderEntity> pageInfo = orderDao.find("skuId = 2").page(i,1);
                pageCount = pageInfo.pageCount();
                System.out.println(pageInfo.list());
            }
            System.out.println("查询完成");
        }));

        CompletableFuture.allOf(selectUser,selectOrder).join();
        System.out.println("所有查询已完成");

With Hibernate Reactive I'm sure there's a similar solution, but I don't personally know what it is. I think it has its own way to open transactions.

[yrodiere]

Also, unrelated to your problem but pagination doesn't seem like a good way to iterate over a large dataset; I'd personally use scrolling instead. But then Panache doesn't expose that, so you'd need to use plain Hibernate code. And I don't know how Hibernate Reactive exposes scrolling (if it even does).

[Sparkel-wsl]

Thank you very much for your help, I am not very familiar with the use of hibernate, I will try to use a scrolling way to process the data.