Java version

[jayvdb]

Currently jgo only looks for java on the PATH. Some jars only work with certain versions, so it would be nice to detect available Java versions and choose the right one.

Ideally the Java version can be stored in the shortcuts, so the right one can be selected for each shortcut.

e.g. Groovy atm fails on JRE 18

$ jgo org.codehaus.groovy:groovy-groovysh:org.codehaus.groovy.tools.shell.Main
INFO 2022-07-26 16:34:03,211: First time start-up may be slow. Downloaded dependencies will be cached for shorter start-up times in subsequent executions.
Exception in thread "main" java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release
        at java.base/java.lang.System.setSecurityManager(System.java:416)
        at java_lang_System$setSecurityManager$2.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
        at org.codehaus.groovy.tools.shell.Main.startGroovysh(Main.groovy:187)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:577)
        at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:193)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:73)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:148)
        at org.codehaus.groovy.tools.shell.Main.main(Main.groovy:156)

I know on Windows all the Java versions and their homes can be found in the registry, and Python on Windows has registry supprt. MacOS makes this discoverable also - subdirectories of a common path IIRC. Linux will have some common approach also, but might be a bit fragmented.

Happy to dig around to come up with a more concrete plan if this idea sounds desirable.

[ctrueden]

I know on Windows all the Java versions and their homes can be found in the registry, and Python on Windows has registry supprt. MacOS makes this discoverable also - subdirectories of a common path IIRC. Linux will have some common approach also, but might be a bit fragmented.

Right. So we could certainly use the Windows registry, and on macOS use /usr/libexec/java_home to list them, and on Linux look in /usr/lib/jvm (at least that's where Ubuntu puts system-wide openjdk package installs). But there is also Homebrew on macOS and Linux, and openjdk installs into conda environments, and local user installs, and maybe containerized package installs from systems like flatpak and snap... it starts to be a lot.

My colleague @marktsuchida recently created another Python-based tool called cjdk for downloading and caching OpenJDK installations of various flavors and versions. We had briefly discussed the possibility of cjdk and jgo working together somehow, but didn't settle on any immediate conclusions or directions.

Maybe we could develop some features along of the lines of what you describe with the help of cjdk. Trying to detect which OpenJDK installations are already present on various systems of different OSes becomes quite complicated and full of heuristics... whereas downloading and installing a requested JDK version on demand would be more straightforward.

Or, we could do both: look for system-wide installs, and also let users specify an installed-on-demand JDK by leaning on cjdk. Thoughts? Opinions?

Either way, I don't think it solves the issue of some Java programs requiring certain versions of Java to function, but us not knowing what they are for a particular program. Unfortunately, it's not part of the Maven standard for a program to declare which Java versions it needs. At least... not reliably enough. The Java standard has minimum Java version baked into every class, and we could extrapolate that easily enough to "program as a whole" by generating the environment, then scanning all the JAR files and taking the max of the minimum Java version over all classes. But it still wouldn't solve the problem you mention above @jayvdb:

Exception in thread "main" java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release

Which is Java breaking backwards compatibility with old versions—i.e. a maximum supported version. Everything in the Java world is designed around the assumption that Java will remain backwards compatible... but it isn't anymore. The introduction of JPMS in Java 9 broke backwards compatibility by requiring new command line flags to be passed for old code to continue to execute. And now the security manager changes in Java 18—even though they are being done for good reasons—still pose a challenge. For the moment, if I understand correctly, passing -Djava.security.manager=allow should avoid this error, and allow such programs to run on Java 18. But likely not forever.

A half-baked idea which occurs to me along these lines is that we could have some kind of "compatibility mode" heuristics in jgo for launching code targeting older versions on newer versions, depending what those two respective versions are. For example, code intended for Java 8 but being run on Java 18 should have -Djava.security.manager=allow as well as --illegal-access=permit and/or the slew of allow-opens to various base Java packages. It would be super hacky, though, and I don't really want to be stuck maintaining these heuristics, especially since Java will continue to move on and these flags will stop working over time. It is probably enough to just enable the caller to tell jgo which version of Java to use, install it via cjdk, and use it—and leave the responsibility of picking the right one to the caller, who needs to read the docs for the program they are using to see what the supported versions are.

[jayvdb]

Thanks for the long response - lots of details there I wasnt aware of.

then scanning all the JAR files and taking the max of the minimum Java version over all classes

I am not so sure that this wouldn't solve my problem. I guess that Groovy would have its minimum set to 17 or maybe even earlier, which would mean we could infer that it should be run on 17, not 18.

Fetching the most suitable jdk using cjdk looks like a great approach at least initially; less work than finding all the JVMs on the system already. This is not ideal for my use-case, which is CI where I already have a JVM installed due to other requirements, but a working solution is better than an ideal one which hasnt been implemented, so I'll take whatever is easiest to build.

[marktsuchida]

It would not be out of the question to add to cjdk the ability to opportunistically use system-installed (or otherwise locally configured) JDKs instead of downloading (something like cachedjdk/cjdk#6 is a prerequisite, and I haven't fully thought it through). The advantage will be small when the goal is to only ever use pre-installed JDKs, other than being able to easily switch to downloading when needed.

[ctrueden]

This is not ideal for my use-case, which is CI where I already have a JVM installed due to other requirements, but a working solution is better than an ideal one which hasnt been implemented, so I'll take whatever is easiest to build.

@jayvdb If you set the JAVA_HOME environment variable to point to the desired installed JVM, then the mvn command uses that Java installation for the environment building portion. We could change jgo to also prefer the path given by this environment variable when it invokes java. Or probably more robustly: invoke mvn -v and use the path to Java that Maven is using, so both are harmonious. Would that help?

[ctrueden]

I guess that Groovy would have its minimum set to 17 or maybe even earlier, which would mean we could infer that it should be run on 17, not 18.

Ah I see, so you're suggesting that we always prefer to launch using the minimum version of Java required for a project? And install that Java if it isn't already present? I like this approach! Run code with the version of Java it was designed for, always. 👍

I do think we'd need to provide a way for the user to override this behavior by specifying an explicit version of Java, though... because there may be some projects that e.g. support Java 8 in a "mostly" kind of way, but behave better when running under Java 11 or 17 when those APIs are detected to be present.

[jayvdb]

Using JAVA_HOME, if set, would be helpful.

But this issue is about jgo internally switching the JVM to use. An opt-in to detect minimum JVM would be a low risk way of introducing the detection logic, letting get road tested and mature before considering whether it should be default behaviour.

I guess a simple first step would be to have an explicit jgo --cjdk=<jdk version id> ... to enable use of cjdk to fetch and select specific jdks. Does that sound acceptable?

[jayvdb]

I've run into another one. https://github.com/fizzed/blaze requires Java 8, failing on 9 fizzed/blaze#14.