More Integrated Python Inferface

[eriq-augustine]

Right now the Python interface just wraps the CLI.
We should look into the tradeoffs of integrating the interface more tightly (calling directly into Java code).
Some possible ways to do this are:

• py4j
• JPype
• pyjnius

[Thrameos]

All three are options to consider.

• Py4j approaches this using the concept of a Gateway which means the JVM and the Python process are separated from each other and live in different processes. Likely a good option if you are looking for strong separation, but it will definitely be a pretty heavy lift as it is a bit like working through RPC.

• JPype is focused much more on ease of use and literally being able to copy Java code into Python with minimal updates (remove type declarations, drop the new, and patch up some syntax differences). It is designed to support scientific computing with loads of fast type conversions and convenience functions. It can work if you are providing access to an app (using JPype as the backend) or attempting to expose a Java library as Python.

• pyjnius is oriented towards providing support of android api and is usable on desktops. It has some similarities to JPype, but is somewhat behind as it lacks the level of memory linkage. It can't represent certain things like Java backed arrays that may prove limiting.

Both JPype and pyjnius use a JNI linked model so you only get one JVM per process. Benchmarks show that they are pretty similar. Py4J is slower as it works over sockets and lacks direct memory transfer without effort such as setting up shared memory.

As far as I am aware all 3 projects are currently active. JPype is currently pushing to consolidate with pyjnius as they are rather similar in terms of functions and the key driver for pyjnius is android support which JPype is working to provide.

[eriq-augustine]

Interesting, thanks for the insights.
Great to hear from an expert.

Sounds like any would be a fair option.
There will not be a lot of back and forth communication in our main use case.
We just need to load up some data (in the order of 100K records for a medium/large model, 10-100M for a large model), invoke inference, then read out the results (typically ~10K records).

I'm leaning towards JPype given the apparent activeness of the community ;)

[Thrameos]

That seems like some pretty light duty so Py4J and JPype are likely options. PyJnius lack of memory integration may be a problem with the number of records if they are all going across the interface.

I use JPype for very integrated code where there are are nearly as many times that Java is calling Python. In your case if invoking the Java interface is going to call back to Python. It can be subtle, for example if reading the results needs to take a java.util.function.Predicate and you want to supply a Python lambda as the search criteria then there will be a huge amount of back and forth. Py4J can do this, but the costs in terms of overhead are much higher as it is an RPC. I use this to the hilt by placing audit hooks in my Java code so that I can have Java call the audit points during operation which launches Python to produce matplotlib results. But that is because my use case is actual development of scientific Java code rather than just use of Java though Python.

On the other hand if the Python code needs to start and stop the JVM then Py4J would be a better option. As much as I try I can't get over the inherent limits of JNI.