Database Inconsistency on windows with orientdb 3.0.44

[sylvain53]

Hi,

After upgrading embedded orient-db from 3.0.38 to 3.0.44, the database often loses its consistency. Orient-DB throws various exceptions when it tries to open the database.

With one crashed database, we got the following exception :

Caused by: java.lang.IndexOutOfBoundsException
	at java.nio.Buffer.checkIndex(Buffer.java:693) ~[?:?]
	at java.nio.DirectByteBuffer.getInt(DirectByteBuffer.java:758) ~[?:?]
	at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurablePage.getIntValue(ODurablePage.java:138) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.index.sbtree.singlevalue.OSBTreeBucketSingleValue.getKey(OSBTreeBucketSingleValue.java:310) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.index.sbtree.singlevalue.OSBTreeBucketSingleValue.find(OSBTreeBucketSingleValue.java:103) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.index.sbtree.singlevalue.OCellBTreeSingleValue.findBucket(OCellBTreeSingleValue.java:1230) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.index.sbtree.singlevalue.OCellBTreeSingleValue.get(OCellBTreeSingleValue.java:164) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.config.OClusterBasedStorageConfiguration.readProperty(OClusterBasedStorageConfiguration.java:1619) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.config.OClusterBasedStorageConfiguration.readConfiguration(OClusterBasedStorageConfiguration.java:814) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.config.OClusterBasedStorageConfiguration.load(OClusterBasedStorageConfiguration.java:177) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.lambda$open$1(OAbstractPaginatedStorage.java:394) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideAtomicOperation(OAtomicOperationsManager.java:153) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:391) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.db.OrientDBEmbedded.open(OrientDBEmbedded.java:296) ~[orientdb-core-3.0.44.jar:3.0.44]
	... 17 more

With another crashed database, we got the following exception:

Caused by: java.lang.NullPointerException
	at com.orientechnologies.orient.core.storage.config.OClusterBasedStorageConfiguration.preloadClusters(OClusterBasedStorageConfiguration.java:1263) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.config.OClusterBasedStorageConfiguration.load(OClusterBasedStorageConfiguration.java:183) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.lambda$open$1(OAbstractPaginatedStorage.java:394) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideAtomicOperation(OAtomicOperationsManager.java:153) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:391) ~[orientdb-core-3.0.44.jar:3.0.44]
	at com.orientechnologies.orient.core.db.OrientDBEmbedded.open(OrientDBEmbedded.java:296) ~[orientdb-core-3.0.44.jar:3.0.44]

We do not know why the database lost its consistency. It appends only on Windows, not on Linux. We did not find a way to replicate the crash easily. However, we think that it’s happened when we shut down the Windows host without closing the database first.

I tried to repair the database by using the console:

OrientDB console v.3.2.16 (build eb52360cf5e90a23c6103aa4b2403a1c4e9b636e, branch UNKNOWN) https://www.orientdb.com
Type 'help' to display all the supported commands.
orientdb> connect plocal:/data/dbdefault admin admin
IMPORTANT! Using default password is unsafe, please change password for user 'admin' on database 'dbdefault'
Connecting to database [plocal:/data/dbdefault] with user 'admin'...
Error: com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database 'dbdefault'

Error: com.orientechnologies.orient.core.exception.OStorageException: Exception during execution of atomic operation inside of storage dbdefault

Error: java.lang.NullPointerException

I did not find any documentation to repair a database that crash during its opening process. Can you help me to recover the database and to pinpoint the crash causes ?

[tglman]

Hi,

This error looks quite strange, the database seems completely broken, we do suggest to restore from a backup, and if you can, plan a full upgrade to 3.2.x series that has some improvements in durability.

Regards