Linking broken due to unchecked trust of .a or .lib files

[credmond]

This method in AbstractTargetConfiguration:

private void extractNativeLibs(String classPath) throws IOException {
        Path libPath = paths.getGvmPath().resolve(Constants.LIB_PATH);
        if (Files.exists(libPath)) {
            FileOps.deleteDirectory(libPath);
        }
        Logger.logDebug("Extracting native libs to: " + libPath);

        List<String> jars = new ClassPath(classPath).filter(s -> s.endsWith(".jar") && !s.contains("javafx-"));
        for (String jar : jars) {
            FileOps.extractFilesFromJar("." + getStaticLibraryFileExtension(), Path.of(jar),
                    libPath, getTargetSpecificNativeLibsFilter());
        }
    }

... can copy files (.a files, for example) that were built for different architectures/operating systems.

Then GCC fails at the linking stage with errors like:

[Mon Nov 04 21:44:45 UTC 2024][INFO] [SUB] ld: multiple errors: unknown file type in '/Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib/libsnappyjava.a'; unknown file type in '/Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib/libjnidispatch.a'
[Mon Nov 04 21:44:45 UTC 2024][INFO] [SUB] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[Mon Nov 04 21:44:45 UTC 2024][FINE] Result for link: 1
[Mon Nov 04 21:44:45 UTC 2024][SEVERE] Process link failed with result: 1

You can see the two "bad" files added at near end of the GCC command. I determined they are indeed for other architectures:

[Mon Nov 04 21:44:45 UTC 2024][FINE] PB Command for link: gcc /Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/KafkaTopical/AppDelegate.o /Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/KafkaTopical/launcher.o /Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/tmp/SVM-1730755868416/com.certak.kafkatopical.cli.launcher.o -ljava -lnio -lzip -lnet -lprefs -lj2pkcs11 -lfdlibm -lextnet -ljvm -llibchelper -ldarwin -lpthread -lz -ldl -lstdc++ -mmacosx-version-min=10.12 -lobjc -Wl,-framework,Foundation -Wl,-framework,AppKit -Wl,-framework,ApplicationServices -Wl,-framework,OpenGL -Wl,-framework,QuartzCore -Wl,-framework,Security -Wl,-framework,Accelerate -Wl,-force_load,/Users/runner/.gluon/substrate/javafxStaticSdk/21-ea+11.2/darwin-x86_64/sdk/lib/libglass.a -Wl,-force_load,/Users/runner/.gluon/substrate/javafxStaticSdk/21-ea+11.2/darwin-x86_64/sdk/lib/libjavafx_font.a -Wl,-force_load,/Users/runner/.gluon/substrate/javafxStaticSdk/21-ea+11.2/darwin-x86_64/sdk/lib/libjavafx_iio.a -Wl,-force_load,/Users/runner/.gluon/substrate/javafxStaticSdk/21-ea+11.2/darwin-x86_64/sdk/lib/libprism_es2.a -Wl,-force_load,/Users/runner/.gluon/substrate/javafxStaticSdk/21-ea+11.2/darwin-x86_64/sdk/lib/libprism_sw.a -o /Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/KafkaTopical -L/Users/runner/.gluon/substrate/javafxStaticSdk/21-ea+11.2/darwin-x86_64/sdk/lib -L/Users/runner/hostedtoolcache/GraalVM/gluon-22.1.0-1-Final/x86_64/lib/svm/clibraries/darwin-amd64 -L/Users/runner/hostedtoolcache/GraalVM/gluon-22.1.0-1-Final/x86_64/lib/static/darwin-amd64 -L/Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib -Wl,-force_load,/Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib/libsnappyjava.a -Wl,-force_load,/Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib/libjnidispatch.a /Users/runner/hostedtoolcache/GraalVM/gluon-22.1.0-1-Final/x86_64/lib/static/darwin-amd64/libjaas.a

In complex projects, we cannot control or foresee what native files are going to exist in third-party library JARs -- many of which contain native libs for all OSs with no real naming or folder standard.

MacOSTargetConfiguration does not implement getTargetSpecificNativeLibsFilter(), which looks like was intended to prevent this problem -- but only implemented on Linux.

Note, this problem was simply logged at warning level on Github macos-12 runners (now removed unfortunately), but is causing a blocking error on macos-13+ -- I'm not sure why exactly.

[credmond]

Example of such files: https://github.com/xerial/snappy-java/tree/master/src/main/resources/org/xerial/snappy/native/AIX/ppc

[credmond]

Xcode default is 15.2 on macos-13, but was 14.2 on macos-12 runner. Switching back to 14.2 on macos-13, resolves the problem. This would do the trick:

env:
  DEVELOPER_DIR: /Applications/Xcode_14.2.0.app

Then instead of fatal errors, you just get these warnings:

[Tue Nov 05 01:13:49 UTC 2024][INFO] [SUB] ld: warning: ignoring file /Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib/libsnappyjava.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x01 0xDF 0x00 0x04 0x5A 0x1D 0xEB 0x76 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 )
[Tue Nov 05 01:13:49 UTC 2024][INFO] [SUB] ld: warning: ignoring file /Users/runner/work/ct-ui-kafkatopical/ct-ui-kafkatopical/target/gluonfx/x86_64-darwin/gvm/lib/libjnidispatch.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x01 0xDF 0x00 0x05 0x63 0x9A 0x0B 0xE0 0x00 0x05 0x7D 0xB0 0x00 0x00 0x34 0x6F )

Not really sure what changed or why in clang from 14 to 15. But anyway, this pulling in of "incorrect" native code, will be a problem again in future when forced to upgrade to later versions of XCode/clang...

Any thoughts?