Migrating from 3.x to 4.x: Database file is corrupted

[sea-steve]

Migrating from 3.x to 4.x: Database file is corrupted

[anidotnet]

Not sure what causing this issue. You can take a look at the upgrade test code here -

nitrite-java/nitrite-mvstore-adapter/src/test/java/org/dizitart/no2/NitriteTest.java

Line 424 in 072909e

public void testReadCompatibility() throws IOException {

One thing I'll suggest, to use minimal store config, as there are config changes as well in H2 v1 vs v2.

I'll caution you against the upgrade functionality, all kind of v1 db might not be upgradable to v2. H2 itself does not support the upgradation (no idea why). Here, it is a mere effort to try upgrade on common use cases only.

[sea-steve]

Thanks for the suggestions. I'll give them a try.

Failing those, if the Nitrite upgrade logic is for common use-cases only, what do you suggest for people with (apparently) non-common use-cases?

[anidotnet]

In un-common use cases, unfortunately, there is no upgrade path yet. User have to destroy the old database and re-create the new one. That's what H2 is also suggesting.

[sea-steve]

Looks like H2 recommends export/import using their export/import tools. I'll take a look at that...

https://www.h2database.com/html/migration-to-v2.html

[sea-steve]

Also, the "Database file is corrupted" message is bogus: the open/upgrade code creates a new (empty) database file whose name is the old db file appended with "_new" (e.g., "my-nitrite.db" -> "my-nitrite.db_new"). The open/upgrade starts to populate this new file, but if the upgrade/open fails (as it did in my case), the "_new" file is not deleted and on subsequent attempts to open "my-nitrite.db", it tries to open the existing "my-nitrite.db_new" file and fails (with the exception/stacktraces I shared -- "Database is corrupted"). The "_new" file should really be deleted on failure (or the code should check for an existing "_new" file and, if it exists, create a new uniquely named file)...

If I delete the "_new" file and try to start my app, here is the actual error that's happening during the open/upgrade (I abbreviated a long list of numbers following "-84, -19, 0,..."):

Caused by: java.lang.IllegalArgumentException: Could not deserialize [-84, -19, 0,... [2.1.214/0]
	at org.dizitart.no2.mvstore.compat.v1.mvstore.DataUtils.newIllegalArgumentException(DataUtils.java:925)
	at org.dizitart.no2.mvstore.compat.v1.NitriteDataType.deserialize(NitriteDataType.java:363)
	at org.dizitart.no2.mvstore.compat.v1.NitriteDataType$SerializedObjectType.read(NitriteDataType.java:1774)
	at org.dizitart.no2.mvstore.compat.v1.NitriteDataType.read(NitriteDataType.java:525)
	at org.dizitart.no2.mvstore.compat.v1.NitriteDataType.read(NitriteDataType.java:408)
	at org.dizitart.no2.mvstore.compat.v1.mvstore.Page$Leaf.readPayLoad(Page.java:1483)
	at org.dizitart.no2.mvstore.compat.v1.mvstore.Page.read(Page.java:596)
	at org.dizitart.no2.mvstore.compat.v1.mvstore.Page.read(Page.java:232)
	at org.dizitart.no2.mvstore.compat.v1.mvstore.MVStore.readPage(MVStore.java:2200)
	... 61 more
Caused by: java.lang.ClassCastException: class org.dizitart.no2.mvstore.compat.v1.Compat$NitriteId cannot be cast to class java.lang.Comparable (org.dizitart.no2.mvstore.compat.v1.Compat$NitriteId is in unnamed module of loader 'app'; java.lang.Comparable is in module java.base of loader 'bootstrap')
	at java.base/java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:393)
	at java.base/java.util.concurrent.ConcurrentSkipListMap.readObject(ConcurrentSkipListMap.java:1255)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1101)
	at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2442)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2280)
	at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1760)
	at java.base/java.io.ObjectInputStream$FieldValues.<init>(ObjectInputStream.java:2625)
	at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2476)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2280)
	at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1760)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:538)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:496)
	at org.dizitart.no2.mvstore.compat.v1.NitriteDataType.deserialize(NitriteDataType.java:360)
	... 68 more

This is very similar to the errors I remember seeing the last time I tried to migrate from 3.x to 4.x. Any thoughts?

[anidotnet]

You are right. The unfinished new file needs to be cleaned up. Is it possible for you to share the data file? I could not reproduce it at my end.

[anidotnet]

Necessary clean up code has been added to 4.2.0-SNAPSHOT.