Java wrapper: Create multi-architecture JAR distributed via Maven Central

[terryburton]

Housekeeping to capture discussion that occurred in PR#4 as a separate issue to allow that PR to be closed...

2024/04/09: @terryburton

The major decision to be made is whether to publish a Java and/or Android library, and where to. My intent so far has been to release a Java library package consisting of a single multi-architecture JAR published on Maven Central. The reasoning is that this benefits all Java developers and if there is no Android app specific helper code in the library then there is little point in also creating and maintaining an Android library — but I'm happy to be educated if this analysis is lacking.

(Because the library is based on native C code, it must be compiled for each of the common target architectures, e.g. x64, x86, arm, arm64. Creating multi-architecture Java library packages is challenging, but there are projects that appear to do this successfully, e.g. https://github.com/OddSource/ifaddrs4u/tree/main/ifaddrs4j)

2024/04/18: @vinaygopinath

... I'm on board with your proposal of publishing a JAR accessible from Maven Central purely to parse the scanned string, leaving the task of scanning codes to the application developer.

2024/04/19: @terryburton

Thank you for taking the time to consider the approach that I outlined concerning the multi-architecture JAR. I conducted some brief testing, but I have had to set things aside for now because of the amount of effort involved. It may be prioritised in the future as a result of community feedback such as what you have provided.

Building the multi-architecture JAR:

• There is a project that provides cross-compiler toolchains in the form of Docker images that might be usable to create the Syntax Engine shared library and JNI for a variety of platforms: https://github.com/multiarch/crossbuild — hopefully all relevant platforms (for servers and mobile / embedded) are supported.
• To automate the generation of assets it would be necessary to create a CI workflow based on this which can assemble the final multi-architecture JAR file.

How the JAR would work:

• The Java library (in the JAR file) can include a custom class loader routine that can probe the OS and architecture at runtime in order to attempt to System.load the correct JNI library for the platform — hopefully all relevant platforms can be detected reliably so that a single JAR is all that is necessary.
• In order to System.load the native code it is necessary for the custom class loader to first copy it out of the JAR to temporary storage. This seems to work well for regular Java platforms, at least on Linux which I have tested.

Considerations for Android:

• The above approach does not appear to work for Android since the native library .so files are stripped out of the APK during the build, i.e. the JAR doesn't appear to be manifested in the same way once the app is assembled. I haven't looked at how and why this occurs.
• Assuming that you can avoid the above issue then there is still the issue that recent versions of Android introduce "W^X" enforcement, i.e. that they do not allow code to be loaded and executed from writable parts of the app storage, i.e. you cannot System.load what the application has written at runtime.
• So a "supplementary" approach might be required for Android: Project may include a separate Gradle task in the app build process that performs the native library from JAR this extraction immediately prior to assembling the APK, effectively placing the native code in the typical project lib/ location (e.g. lib/arm64) so that it gets built into the APK in that same way that a "vendored-in" shared library (or output of an NDK build step) would be, such that it is accessible to the app at runtime. The custom class loader would need to be programmed to not attempt library extraction when running on Android and when the library can be loaded from the default native library location for the current platform.
• So to use the packaged library for Android development, in addition to declaring an implementation dependency you would also need to add a task to the gradle.build, which sounds like it may be preferable to vendoring in the Syntax Engine code in the way shown by the current example Android app.

That's a lot of steps to put in order and there is still some risk of something critical not working out as intended, so the idea is shelved for now...

Of course if anyone would like to have a go at the above then this will be welcome.