Sync: Desktop -> Android: All text after the first NUL character is lost

[personalizedrefrigerator]

Sync: Desktop -> Android: All text after the first NUL character is lost

[personalizedrefrigerator]

With the following diff applied,

diff

diff --git a/packages/lib/JoplinServerApi.ts b/packages/lib/JoplinServerApi.ts
index 5bf29e310..f70b66837 100644
--- a/packages/lib/JoplinServerApi.ts
+++ b/packages/lib/JoplinServerApi.ts
@@ -193,6 +193,7 @@ export default class JoplinServerApi {
 			}
 
 			const responseText = await response.text();
+			logger.debug('got text', responseText)
 
 			if (this.debugRequests_) {
 				logger.debug('Response', Date.now() - startTime, options.responseFormat, responseText);
@@ -216,6 +217,7 @@ export default class JoplinServerApi {
 				if (responseJson_) return responseJson_;
 				responseJson_ = JSON.parse(responseText);
 				if (!responseJson_) throw newError('Cannot parse JSON response', response.status);
+				logger.debug('got response JSON', JSON.stringify(responseJson_));
 				return responseJson_;
 			};
 
diff --git a/packages/lib/Synchronizer.ts b/packages/lib/Synchronizer.ts
index c8701c7a3..c0cbee974 100644
--- a/packages/lib/Synchronizer.ts
+++ b/packages/lib/Synchronizer.ts
@@ -937,6 +937,7 @@ export default class Synchronizer {
 								logger.warn(`Remote has been deleted between now and the delta() call? In that case it will be handled during the next sync: ${path}`);
 								continue;
 							}
+							logger.debug('UpdateLocal:', content);
 							content = ItemClass.filter(content);
 
 							// 2017-12-03: This was added because the new user_updated_time and user_created_time properties were added
@@ -984,6 +985,7 @@ export default class Synchronizer {
 									await MasterKey.save(content);
 								}
 							} else {
+								logger.debug('Saving...', ItemClass.name, '...', ItemClass);
 								await ItemClass.save(content, options);
 							}
 
diff --git a/packages/lib/models/Note.ts b/packages/lib/models/Note.ts
index 6c1416be1..c3b099a1d 100644
--- a/packages/lib/models/Note.ts
+++ b/packages/lib/models/Note.ts
@@ -717,6 +717,7 @@ export default class Note extends BaseItem {
 		const oldNote = !isNew && o.id ? await Note.load(o.id) : null;
 
 		syncDebugLog.info('Save Note: P:', oldNote);
+		this.logger().debug('Save note', { isNew, isProvisional }, o);
 
 		let beforeNoteJson = null;
 		if (oldNote && this.revisionService().isOldNote(o.id)) {
@@ -737,6 +738,7 @@ export default class Note extends BaseItem {
 		syncDebugLog.info('Save Note: N:', o);
 
 		const note = await super.save(o, options);
+		syncDebugLog.info('Saved content:', await Note.load(o.id));
 
 		void ItemChange.add(BaseModel.TYPE_NOTE, note.id, isNew ? ItemChange.TYPE_CREATE : ItemChange.TYPE_UPDATE, changeSource, beforeNoteJson);
 
diff --git a/packages/lib/services/synchronizer/syncDebugLog.ts b/packages/lib/services/synchronizer/syncDebugLog.ts
index 1d2c12312..ad37a1565 100644
--- a/packages/lib/services/synchronizer/syncDebugLog.ts
+++ b/packages/lib/services/synchronizer/syncDebugLog.ts
@@ -4,6 +4,6 @@
 
 import Logger from '@joplin/utils/Logger';
 
-const syncDebugLog = new Logger();
+const syncDebugLog = Logger.create('SyncDebugLog');//new Logger();
 
 export default syncDebugLog;

Note: For easier debugging, I disabled note history. The issue also happens with note history enabled.

I'm getting the following logs:

 INFO  19:03:52: SyncDebugLog: Save Note: N:: {"altitude": "0.0000", "application_data": "", "author": "", "body": "This is a test...

NUL:  After the NUL character.








Also after the NUL characters.
.......

````
  
````", "conflict_original_id": "", "created_time": 1707429220069, "encryption_applied": 0, "encryption_cipher_text": "", "id": "b19aa70cacec4506b17d91f6bfb855b8", "is_conflict": 0, "is_shared": 0, "is_todo": 0, "latitude": "123.00000000", "longitude": "456.00000000", "markup_language": 1, "master_key_id": "", "order": 0, "parent_id": "344352dba573486ba6910459713a5aff", "share_id": "", "source": "joplindev-desktop", "source_application": "net.cozic.joplindev-desktop", "source_url": "", "title": "NUL test", "todo_completed": 0, "todo_due": 0, "type_": 1, "updated_time": 1707523427342, "user_created_time": 1707429180000, "user_data": "", "user_updated_time": 1707523427342}



 INFO  19:03:52: SyncDebugLog: Saved content: {"altitude": "0.0000", "application_data": "", "author": "", "body": "This is a test...

NUL: ", "conflict_original_id": "", "created_time": 1707429220069, "encryption_applied": 0, "encryption_cipher_text": "", "id": "b19aa70cacec4506b17d91f6bfb855b8", "is_conflict": 0, "is_shared": 0, "is_todo": 0, "latitude": "123.00000000", "longitude": "456.00000000", "markup_language": 1, "master_key_id": "", "order": 0, "parent_id": "344352dba573486ba6910459713a5aff", "share_id": "", "source": "joplindev-desktop", "source_application": "net.cozic.joplindev-desktop", "source_url": "", "title": "NUL test", "todo_completed": 0, "todo_due": 0, "type_": 1, "updated_time": 1707523427342, "user_created_time": 1707429180000, "user_data": "", "user_updated_time": 1707523427342}

Notice that the Saved content log doesn't include the NUL characters, but the Save Note: N: log does.

The two most relevant log statements are included in Note.ts like this:

 		syncDebugLog.info('Save Note: N:', o);
 
 		const note = await super.save(o, options);
+		syncDebugLog.info('Saved content:', await Note.load(o.id));

Thus, BaseItem.save is truncating text at the first NUL character on mobile, but not on desktop.

[personalizedrefrigerator]

This seems to be related to andpor/react-native-sqlite-storage#107. Unfortunately, react-native-sqlite-storage seems to be unmaintained.

According to the linked issue, this SQLite library should have a fix, but it also seems to be unmaintained.

Edit: Note that the upstream issue suggests this is an Android-only issue.