Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loader inventory fetching records is blocking pipeline #89

Open
ikus060 opened this issue Nov 2, 2021 · 1 comment
Open

Loader inventory fetching records is blocking pipeline #89

ikus060 opened this issue Nov 2, 2021 · 1 comment
Labels
bug Something isn't working

Comments

@ikus060
Copy link

ikus060 commented Nov 2, 2021

Logstash information:

docker image
bitnami/logstash:7.10.1-debian-10-r35

JVM (e.g. java -version):

N/A

OS version (uname -a if on a Unix-like system):

N/A

Description of the problem including expected versus actual behavior:

When using jdbc_static to fetch a reasonable list of records from MySQL database, the pipeline is blocked while recrods get fetched. This causes peak CPU usage as schedule intervale.

I would expect the pipeline to be blocked only during the derby database refresh. Not during the fetching.

Provide logs (if relevant):

When the following line of code get printed, the pipeline stop processing.

[2021-11-02T15:44:48,867][INFO ][logstash.filters.jdbc.readwritedatabase][main] loader inventory, fetched 54114 records in: 79.353 seconds
[2021-11-02T15:44:50,897][INFO ][logstash.filters.jdbc.readwritedatabase][main] loader inventory, saved fetched records to import file in: 2.028 seconds
[2021-11-02T15:44:53,203][INFO ][logstash.filters.jdbc.readwritedatabase][main] loader inventory, imported all fetched records in: 2.304 seconds

While attached to the logstash process using JMX. Here the relevent Thread dumps.
worker0 is block by "Ruby-0-Thread-21"

"[main]>worker0" - Thread t@34
   java.lang.Thread.State: WAITING
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        - waiting to lock <504ac9f5> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync) owned by "Ruby-0-Thread-21: :1" t@50
        at [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
        at [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:1009)
        at [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1324)
        at [email protected]/java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:738)
        at [email protected]/jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
        at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
        at app//org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:441)
        at app//org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:305)
        at app//org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:32)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007a4440.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a5440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.read_write_database.RUBY$method$fetch$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/read_write_database.rb:23)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007f6c40.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.lookup.RUBY$method$fetch$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/lookup.rb:144)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100807040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.lookup.RUBY$method$enhance$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/lookup.rb:87)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100807040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.lookup_processor.RUBY$block$enhance$1(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/lookup_processor.rb:47)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000101887440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at app//org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
        at app//org.jruby.runtime.Block.yield(Block.java:184)
        at app//org.jruby.RubyArray.collect(RubyArray.java:2563)
        at app//org.jruby.RubyArray.map19(RubyArray.java:2577)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007a2840.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007c8040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cf040.guardWithCatch(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cfc40.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.lookup_processor.RUBY$method$enhance$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/lookup_processor.rb:47)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100813c40.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc_static.RUBY$method$filter$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc_static.rb:155)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000101aa4040.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:267)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.logstash_minus_core.lib.logstash.filters.base.RUBY$method$do_filter$0(/opt/bitnami/logstash/logstash-core/lib/logstash/filters/base.rb:159)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x000000010195e040.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:267)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.logstash_minus_core.lib.logstash.filters.base.RUBY$block$multi_filter$1(/opt/bitnami/logstash/logstash-core/lib/logstash/filters/base.rb:178)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000101728440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at app//org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
        at app//org.jruby.runtime.Block.yield(Block.java:184)
        at app//org.jruby.RubyArray.each(RubyArray.java:1809)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007a2840.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007c8040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cf040.guardWithCatch(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cfc40.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.logstash_minus_core.lib.logstash.filters.base.RUBY$method$multi_filter$0(/opt/bitnami/logstash/logstash-core/lib/logstash/filters/base.rb:175)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000101d89040.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
        at app//org.logstash.config.ir.compiler.FilterDelegatorExt.doMultiFilter(FilterDelegatorExt.java:127)
        at app//org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multiFilter(AbstractFilterDelegatorExt.java:134)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset4.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset4.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset6.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset6.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset3.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset4.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset5.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset2.compute(Unknown Source)
        at org.logstash.generated.CompiledDataset3.compute(Unknown Source)
        at app//org.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.compute(CompiledPipeline.java:329)
        at app//org.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.compute(CompiledPipeline.java:323)
        at app//org.logstash.execution.WorkerLoop.run(WorkerLoop.java:83)
        at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
        at app//org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:426)
        at app//org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:293)
        at app//org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:24)
        at app//org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:86)
        at app//org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a4040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_workers$5(/opt/bitnami/logstash/logstash-core/lib/logstash/java_pipeline.rb:295)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cc840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
        at app//org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
        at app//org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
        at app//org.jruby.runtime.Block.call(Block.java:139)
        at app//org.jruby.RubyProc.call(RubyProc.java:318)
        at app//org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
        at [email protected]/java.lang.Thread.run(Thread.java:834)

   Locked ownable synchronizers:
        - None

"Ruby-0-Thread-21: :1" - Thread t@50
   java.lang.Thread.State: RUNNABLE
        at [email protected]/java.net.SocketInputStream.socketRead0(Native Method)
        at [email protected]/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
        at [email protected]/java.net.SocketInputStream.read(SocketInputStream.java:168)
        at [email protected]/java.net.SocketInputStream.read(SocketInputStream.java:140)
        at com.mysql.cj.protocol.ReadAheadInputStream.fill(ReadAheadInputStream.java:107)
        at com.mysql.cj.protocol.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:150)
        at com.mysql.cj.protocol.ReadAheadInputStream.read(ReadAheadInputStream.java:180)
        - locked <6c1d30a0> (a com.mysql.cj.protocol.ReadAheadInputStream)
        at [email protected]/java.io.FilterInputStream.read(FilterInputStream.java:133)
        at com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:64)
        at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:63)
        at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:45)
        at com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:52)
        at com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:41)
        at com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:54)
        at com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:44)
        at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:532)
        at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:702)
        at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:641)
        at com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:940)
        at com.mysql.cj.protocol.a.NativeProtocol.sendQueryString(NativeProtocol.java:886)
        at com.mysql.cj.NativeSession.execSQL(NativeSession.java:1073)
        at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1168)
        - locked <f06dd79> (a com.mysql.cj.jdbc.ConnectionImpl)
        at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
        at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
        at app//org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:456)
        at app//org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:317)
        at app//org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:42)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007be040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010802a440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077dc40.reinvoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.adapters.jdbc.RUBY$block$execute$4(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/adapters/jdbc.rb:268)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010183a440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at app//org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:66)
        at app//org.jruby.runtime.Block.yieldSpecific(Block.java:153)
        at app//org.jruby.ir.targets.YieldSite.yieldSpecific(YieldSite.java:156)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007a2840.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a3840.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.database.logging.RUBY$method$log_connection_yield$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/database/logging.rb:38)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001073c7040.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:116)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:175)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:301)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007d5c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d6040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007edc40.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.adapters.jdbc.RUBY$block$execute$3(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/adapters/jdbc.rb:268)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010183b840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at app//org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
        at app//org.jruby.runtime.Block.yield(Block.java:184)
        at app//org.jruby.ir.runtime.IRRuntimeHelpers.yield(IRRuntimeHelpers.java:499)
        at app//org.jruby.ir.targets.YieldSite.yield(YieldSite.java:114)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007d3440.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007f6840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.adapters.jdbc.RUBY$method$statement$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/adapters/jdbc.rb:704)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001073c4840.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.adapters.jdbc.RUBY$block$execute$2(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/adapters/jdbc.rb:263)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010183c040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at app//org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
        at app//org.jruby.runtime.Block.yield(Block.java:184)
        at app//org.jruby.ir.runtime.IRRuntimeHelpers.yield(IRRuntimeHelpers.java:499)
        at app//org.jruby.ir.targets.YieldSite.yield(YieldSite.java:114)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007d3440.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007f6840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.connection_pool.threaded.RUBY$method$hold$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/connection_pool/threaded.rb:92)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000105a6b040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.database.connecting.RUBY$method$synchronize$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/database/connecting.rb:269)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000107447440.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:103)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.adapters.jdbc.RUBY$method$execute$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/adapters/jdbc.rb:262)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001073c6440.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:116)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:175)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:301)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007d5c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d6040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007edc40.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$execute$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb:1093)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100f13440.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:103)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.adapters.jdbc.RUBY$method$fetch_rows$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/adapters/jdbc.rb:755)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x000000010aa07040.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$with_sql_each$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb:946)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100949840.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$with_sql_first$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb:954)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007b9c40.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$with_sql_single_value$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb:962)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x000000010079c440.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:106)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:278)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077a440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$single_value!$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb:734)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$single_value!$0$__VARARGS__(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100e41840.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:248)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:255)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007a4440.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100a94040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_38_dot_0.lib.sequel.dataset.actions.RUBY$method$count$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.38.0/lib/sequel/dataset/actions.rb:110)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100903c40.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:248)
        at app//org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:255)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x00000001007a4440.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100a94040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.read_only_database.RUBY$method$count$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/read_only_database.rb:13)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010cc35840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.loader.RUBY$method$fetch$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/loader.rb:23)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007ec840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.read_write_database.RUBY$method$fill_local_table$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/read_write_database.rb:68)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010cc35840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.read_write_database.RUBY$method$repopulate_all$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/read_write_database.rb:9)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010cc35840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.repeating_load_runner.RUBY$method$repeated_load$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/repeating_load_runner.rb:9)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007ec840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.single_load_runner.RUBY$method$call$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/single_load_runner.rb:30)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_0_dot_6.lib.logstash.filters.jdbc.single_load_runner.RUBY$method$call$0$__VARARGS__(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/filters/jdbc/single_load_runner.rb)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100e28440.invokeStatic(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000100751840.invokeExact_MT(LambdaForm$MH)
        at app//org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
        at app//org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
        at app//org.jruby.RubyMethod.call(RubyMethod.java:131)
        at app//org.jruby.RubyMethod$INVOKER$i$call.call(RubyMethod$INVOKER$i$call.gen)
        at app//org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:174)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007be040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.RUBY$method$do_call$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:234)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007f6c40.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d7c40.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007d5840.linkToCallSite(LambdaForm$MH)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.RUBY$method$do_trigger$0(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:258)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010cc35840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010077e040.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.RUBY$block$start_work_thread$3(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:300)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010092e040.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
        at app//org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
        at app//org.jruby.runtime.Block.call(Block.java:143)
        at app//org.jruby.RubyProc.call(RubyProc.java:275)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x000000010081c440.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.DelegatingMethodHandle$Holder.delegate(DelegatingMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007a6440.guard(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.RUBY$block$start_work_thread$2(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:299)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x0000000109781840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at app//org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:66)
        at app//org.jruby.runtime.Block.yieldSpecific(Block.java:153)
        at app//org.jruby.RubyKernel.loop(RubyKernel.java:1442)
        at app//org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(RubyKernel$INVOKER$s$0$0$loop.gen)
        at app//org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555)
        at app//org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:197)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/0x0000000100779c40.invokeVirtual(LambdaForm$DMH)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cd840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.linkToCallSite(Invokers$Holder)
        at opt.bitnami.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.RUBY$block$start_work_thread$1(/opt/bitnami/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:289)
        at [email protected]/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
        at [email protected]/java.lang.invoke.LambdaForm$MH/0x00000001007cc840.invoke(LambdaForm$MH)
        at [email protected]/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
        at app//org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
        at app//org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
        at app//org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
        at app//org.jruby.runtime.Block.call(Block.java:139)
        at app//org.jruby.RubyProc.call(RubyProc.java:318)
        at app//org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
        at [email protected]/java.lang.Thread.run(Thread.java:834)

   Locked ownable synchronizers:
        - locked <504ac9f5> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)

The problem seams to be related to Line 66: https://github.com/logstash-plugins/logstash-integration-jdbc/blob/master/lib/logstash/filters/jdbc/read_write_database.rb#L66

The write lock is aquired to early in the process. It should be aquired after fetching the records on Line 72
@ikus060 ikus060 added the bug Something isn't working label Nov 2, 2021
@ikus060
Copy link
Author

ikus060 commented Feb 28, 2022

The following patch fixes the issue:

--- read_write_database.rb.orig	2021-12-08 20:39:59.499527593 +0000
+++ read_write_database.rb	2021-12-08 20:42:32.960977605 +0000
@@ -63,12 +63,12 @@
 
     def fill_local_table(loader)
       begin
-        @rwlock.writeLock().lock()
         start = Time.now.to_f
         records = loader.fetch
         records_size = records.size
         return if records_size.zero?
         logger.info("loader #{loader.id}, fetched #{records_size} records in: #{(Time.now.to_f - start).round(3)} seconds")
+        @rwlock.writeLock().lock()
         start = Time.now.to_f
         import_file = ::File.join(loader.staging_directory, loader.table.to_s)
         ::File.open(import_file, "w") do |fd|

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ikus060