2023.04-pre: COSTORE_GW Segmentation Fault

[OPMUSER]

This case files using the latest master build.

Current Working Directory  /media/sf_D_DRIVE/Linux/OPM/OPM-TESTS-DBX/co2store
flow --parameter-file=CO2STORE_GW.param | tee CO2STORE_GW.LOG
Running /home/david/OPM/GitHub/opm-simulators/build/bin/flow
===============Saturation Functions Diagnostics===============

System:  Water-Gas system.
Relative permeability input format: Saturation Family II.
Number of saturation regions: 1

flow: /home/david/OPM/GitHub/opm-models/opm/models/blackoil/blackoilintensivequantities.hh:477: void Opm::BlackOilIntensiveQuantities<TypeTag>::update(const ElementContext&, unsigned int, unsigned int) [with TypeTag = Opm::Properties::TTag::EclFlowGasWaterDissolutionProblem; Opm::BlackOilIntensiveQuantities<TypeTag>::ElementContext = Opm::FvBaseElementContext<Opm::Properties::TTag::EclFlowGasWaterDissolutionProblem>]: Assertion `isfinite(fluidState_.density(phaseIdx))' failed.
[david-VirtualBox:05514] *** Process received signal ***
[david-VirtualBox:05514] Signal: Aborted (6)
[david-VirtualBox:05514] Signal code:  (-6)
[david-VirtualBox:05514] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fd921a42520]
[david-VirtualBox:05514] [ 1] /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7fd921a96a7c]
[david-VirtualBox:05514] [ 2] /lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7fd921a42476]
[david-VirtualBox:05514] [ 3] /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7fd921a287f3]
[david-VirtualBox:05514] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x2871b)[0x7fd921a2871b]
[david-VirtualBox:05514] [ 5] /lib/x86_64-linux-gnu/libc.so.6(+0x39e96)[0x7fd921a39e96]
[david-VirtualBox:05514] [ 6] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x165432f)[0x55dbe2a2832f]
[david-VirtualBox:05514] [ 7] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x15f235b)[0x55dbe29c635b]
[david-VirtualBox:05514] [ 8] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x15f2985)[0x55dbe29c6985]
[david-VirtualBox:05514] [ 9] /lib/x86_64-linux-gnu/libgomp.so.1(GOMP_parallel+0x46)[0x7fd922815a16]
[david-VirtualBox:05514] [10] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x160170e)[0x55dbe29d570e]
[david-VirtualBox:05514] [11] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x167b5d1)[0x55dbe2a4f5d1]
[david-VirtualBox:05514] [12] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x167be6b)[0x55dbe2a4fe6b]
[david-VirtualBox:05514] [13] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x15f4bc8)[0x55dbe29c8bc8]
[david-VirtualBox:05514] [14] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x2a4701)[0x55dbe1678701]
[david-VirtualBox:05514] [15] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fd921a29d90]
[david-VirtualBox:05514] [16] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fd921a29e40]
[david-VirtualBox:05514] [17] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x3252e5)[0x55dbe16f92e5]
[david-VirtualBox:05514] *** End of error message ***
Aborted (core dumped)
Process Complete (0) 

[bska]

Possibly resolved by OPM/opm-simulators#4580. Please retest if you have the time and inclination.

[totto82]

This model runs locally with master.

[OPMUSER]

Unfortunately, this still fails on my machine using the latest master build, with:

Flow: /home/david/OPM/GitHub/opm-models/opm/models/blackoil/blackoilintensivequantities.hh:477: void Opm::BlackOilIntensiveQuantities<TypeTag>::update(const ElementContext&, unsigned int, unsigned int) [with TypeTag = Opm::Properties::TTag::EclFlowGasWaterDissolutionProblem; Opm::BlackOilIntensiveQuantities<TypeTag>::ElementContext = Opm::FvBaseElementContext<Opm::Properties::TTag::EclFlowGasWaterDissolutionProblem>]: Assertion `isfinite(fluidState_.density(phaseIdx))' failed.
[david-VirtualBox:02067] *** Process received signal ***
[david-VirtualBox:02067] Signal: Aborted (6)
[david-VirtualBox:02067] Signal code:  (-6)
[david-VirtualBox:02067] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fb8a9442520]
[david-VirtualBox:02067] [ 1] /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7fb8a9496a7c]
[david-VirtualBox:02067] [ 2] /lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7fb8a9442476]
[david-VirtualBox:02067] [ 3] /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7fb8a94287f3]
[david-VirtualBox:02067] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x2871b)[0x7fb8a942871b]
[david-VirtualBox:02067] [ 5] /lib/x86_64-linux-gnu/libc.so.6(+0x39e96)[0x7fb8a9439e96]
[david-VirtualBox:02067] [ 6] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x1641b0f)[0x563295876b0f]
[david-VirtualBox:02067] [ 7] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x15df38b)[0x56329581438b]
[david-VirtualBox:02067] [ 8] /home/david/OPM/GitHub/opm-simulators/build/bin/flow(+0x15df9b5)[0x5632958149b5]
[david-VirtualBox:02067] [ 9] /lib/x86_64-linux-gnu/libgomp.so.1(+0x1db9e)[0x7fb8aa281b9e]
[david-VirtualBox:02067] [10] /lib/x86_64-linux-gnu/libc.so.6(+0x94b43)[0x7fb8a9494b43]
[david-VirtualBox:02067] [11] /lib/x86_64-linux-gnu/libc.so.6(+0x126a00)[0x7fb8a9526a00]
[david-VirtualBox:02067] *** End of error message ***
Aborted (core dumped)
Process Complete (0) 

Notice the error message is different than before.

[akva2]

runs fine for me with master as well. with or without asserts enabled, release and debug binaries. all is good. david please double-check you are up to date in all modules.

[OPMUSER]

@akva2 and @totto82 Well something is not right, as I rebuilt master this morning (Singapore time), and now I get:

Error: Solver failed to converge after cutting timestep 10 times.
Program threw an exception: [/home/david/OPM/GitHub/opm-simulators/opm/simulators/timestepping/AdaptiveTimeSteppingEbos.hpp:583] Solver failed to converge after cutting timestep 10 times.

[akva2]

what's in that param file of yours?

[OPMUSER]

All default values except for EnableOpmRstFile="true". See attached *.PRT file

CO2STORE_GW.zip

[OPMUSER]

@akva2 Well, the good news is the job runs with 2023-04-RC1. So I think I need to do a clean re-built and test again.

Will close when I have re-tested.

[OPMUSER]

I deleted all build directories and re-built from master today. And I'm using the default parameter set.

I still get:

================ Starting main simulation loop ===============


Report step  0/100 at day 0/100, date = 01-Jan-2019

Warning: Keyword 'BO' is unhandled for output to file.

Warning: Keyword 'KRO' is unhandled for output to file.

Warning: Keyword 'VOIL' is unhandled for output to file.

Starting time step 0, stepsize 1 days, at day 0/1, date = 01-Jan-2019
 well INJ gets SHUT during iteration
Restart file written for report step   0/100, date = 01-Jan-2019 00:00:00

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.330000 days


Starting time step 0, stepsize 0.33 days, at day 0/1, date = 01-Jan-2019
 well INJ gets SHUT during iteration

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.108900 days


Starting time step 0, stepsize 0.1089 days, at day 0/1, date = 01-Jan-2019
 well INJ gets SHUT during iteration

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.035937 days


Starting time step 0, stepsize 0.035937 days, at day 0/1, date = 01-Jan-2019
 well INJ gets SHUT during iteration

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.011859 days


Starting time step 0, stepsize 0.0118592 days, at day 0/1, date = 01-Jan-2019
 well INJ gets SHUT during iteration
Well INJ will be shut because it cannot get converged.

Problem:
Problematic well(s) were shut: INJ (retrying timestep)


Starting time step 0, stepsize 0.0118592 days, at day 0/1, date = 01-Jan-2019

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.003914 days


Starting time step 0, stepsize 0.00391354 days, at day 0/1, date = 01-Jan-2019

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.001291 days


Starting time step 0, stepsize 0.00129147 days, at day 0/1, date = 01-Jan-2019

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.000426 days


Starting time step 0, stepsize 0.000426184 days, at day 0/1, date = 01-Jan-2019

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.000141 days


Starting time step 0, stepsize 0.000140641 days, at day 0/1, date = 01-Jan-2019

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.000046 days


Starting time step 0, stepsize 4.64115e-05 days, at day 0/1, date = 01-Jan-2019

Problem: Solver convergence failure - Numerical problem encountered
Timestep chopped to 0.000015 days


Starting time step 0, stepsize 1.53158e-05 days, at day 0/1, date = 01-Jan-2019

Error: Solver failed to converge after cutting timestep 10 times.
Program threw an exception: [/home/david/OPM/GitHub/opm-simulators/opm/simulators/timestepping/AdaptiveTimeSteppingEbos.hpp:583] Solver failed to converge after cutting timestep 10 times.
Failed: 10842
OPM Flow Process 10842 Has Failed
Process Complete (0) 

What gives?

[daavid00]

I just locally build (clang) flow from master and the CO2STORE_GW.DATA deck runs fine.