"Not enough output buffer space" error during sync

[dstillman]

https://forums.zotero.org/discussion/118673/zotero-android-webdav-sync-issue

[Dima-Android]

From the crash logs it seems the application crashes when trying to set baseTitle of a DB Object Item. This happens during a sync. The crash is born from somewhere inside of the realm db library.
The text that is set to baseTitle is a string that:
1. Originates from note’s content.
2. Being sanitized from various characters.
3. Being cut down to 200 chars.

We haven’t acquired any note’s content from users for which this crash happens and for now we only have exactly 199 characters in the form of “0xd83d 0xdd24 0x524d” that appears after a crash message in firebase. When these codes are converted to UTF-8 we can see the original text. Lots of (if not all) of these strings end with 0xd83d. Which in UTF-8 is “=”. And this as I’ve found out a part of an emoji(smiley). Some emojis, as it turns out, consist of 2 characters - a high surrogate and low surrogate. And apparently that “0xd83d” is one of those.
I was confused for a while trying to figure out why the exception states “Not enough output buffer space; error_code=4” since we always limit the string we set to baseTitle to just 200 chars and googling it didn’t give me any useful insights. I think the issue we have is not really related to “not enough space”. What seems to happen is that internals of Realm lib are not happy with a string which contains a high surrogate without a low surrogate. 
I ran some tests where I fed a bunch of strings with a missing surrogate pair and was able to crash the app, but still wasn’t able to get the exact crash log that users are reporting. I was only able to get “Failure when converting to UTF-8; error_code = 6”, which corresponds to an error regarding missing surrogate pair.

The fix for now is that we check whether the last character is a surrogate without a pair, and if it is, we drop it from a string.
If this error still continues to appear then now we also try-catch that part of the code and log the note’s first 210 characters so that we can analyze them later. Also we avoid crashing the app and just set baseTitle to an empty string.