Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force static linking for libLLVM #1027

Open
geofft opened this issue Jan 26, 2024 · 3 comments
Open

Force static linking for libLLVM #1027

geofft opened this issue Jan 26, 2024 · 3 comments
Labels

Comments

@geofft
Copy link

geofft commented Jan 26, 2024

We build LLVM and llvmlite from source, and our LLVM is configured to build both static and dynamic libraries. I noticed that llvmlite was getting dynamically linked against libLLVM-14.so. My sense is that llvmlite intends to be statically linked to libLLVM per https://llvmlite.readthedocs.io/en/latest/faqs.html , so I think the build system should enforce it.

For the make-based build, adding --link-static to the llvm-config --libs call fixes it, but for the CMake build, I'm not exactly sure what to do. The lib/cmake/llvm/ I get seems to list both static and dynamic libraries and that's how my libllvmlite.so is getting linked if I set LLVMLITE_USE_CMAKE=1, and I am not quite familiar enough with CMake to know why.

If you would like, I am happy to send you a pull request for the makefile change alone if you think that's useful by itself (but it's also a one-line change).

On the other hand, if both static and dynamic linking is supported but your wheels statically link LLVM because shipping the dynamic libraries in a wheel is both useless and painful, that makes perfect sense and I am happy to send you a PR to clarify the FAQ. Dynamic linking was working fine for us, and I think libLLVM is good enough at symbol versioning that the concern in that FAQ about multiple versions is not so relevant these days - you should be able to safely import both llvmlite and something else that uses e.g. libLLVM-15.so into the same process.

(The issue template wants me to confirm that this affects the latest release and to attach a reproducer. It does affect the latest release as well as main, but a reproducer is going to be a little complicated... let me know if you want me to write a shell script to build LLVM from source and demonstrate the problem :) )

@stuartarchibald
Copy link
Contributor

We build LLVM and llvmlite from source, and our LLVM is configured to build both static and dynamic libraries. I noticed that llvmlite was getting dynamically linked against libLLVM-14.so. My sense is that llvmlite intends to be statically linked to libLLVM per https://llvmlite.readthedocs.io/en/latest/faqs.html , so I think the build system should enforce it.

For the make-based build, adding --link-static to the llvm-config --libs call fixes it, but for the CMake build, I'm not exactly sure what to do. The lib/cmake/llvm/ I get seems to list both static and dynamic libraries and that's how my libllvmlite.so is getting linked if I set LLVMLITE_USE_CMAKE=1, and I am not quite familiar enough with CMake to know why.

If you would like, I am happy to send you a pull request for the makefile change alone if you think that's useful by itself (but it's also a one-line change).

On the other hand, if both static and dynamic linking is supported but your wheels statically link LLVM because shipping the dynamic libraries in a wheel is both useless and painful, that makes perfect sense and I am happy to send you a PR to clarify the FAQ. Dynamic linking was working fine for us, and I think libLLVM is good enough at symbol versioning that the concern in that FAQ about multiple versions is not so relevant these days - you should be able to safely import both llvmlite and something else that uses e.g. libLLVM-15.so into the same process.

Thanks for raising this @geofft, it will be discussed at the maintainer's meeting next week. Both static and dynamic linking are supported, though it would appear from the above that the CMake build may need some investigation/adjustment. Most "distributors" e.g. conda packagers, linux distributions etc. are using dynamic linking, whereas wheels use static (as noted above, it's hard to do dynamic linking in wheels etc). Numba conda channel builds use static linking too, this is so that the maintainers can be absolutely sure about the linkage chain and provision of symbols (making sure that the Numba stack is isolated from external influence where possible makes it easier to debug). Historically, collisions across multiple LLVM versions have been a problem and this could arise from e.g. very old LLVM versions in use in surprising places through to various build/configuration options, though it may indeed be less of an issue now.

(The issue template wants me to confirm that this affects the latest release and to attach a reproducer. It does affect the latest release as well as main, but a reproducer is going to be a little complicated... let me know if you want me to write a shell script to build LLVM from source and demonstrate the problem :) )

I think omitting this is entirely reasonable!

@geofft
Copy link
Author

geofft commented Jan 26, 2024

Oh, your mention of Linux distros reminded me: I see that your Conda build script for LLVM is applying a few patches, one of which appears to be generic (not platform-specific). We're linking against stock upstream LLVM. Is that fine? I assume that's what distros are doing too, right? I know in the past you had more extensive patches against LLVM.

@seibert
Copy link
Contributor

seibert commented Jan 26, 2024

The two major patches are:

  • A fix for an upstream linker issue in MCJIT that manifests on AArch64 platforms.
  • A larger patch for LLVM that enables autovectorization of more math functions using the SVML library from Intel. (Numba only enables this if SVML is detected in the environment, which is only available for x86-64.)

If you are on x86-64, then using stock LLVM should be fine, but some loops containing special math functions will no longer be autovectorizable.

wrvsrx added a commit to wrvsrx/nixpkgs that referenced this issue Sep 7, 2024
Llvmlite doc says that they link to llvm statically, but it's not. Check

- numba/llvmlite#1027.
- https://github.com/numba/llvmlite/blob/ca123c3ae2a6f7db865661ae509862277ec5d692/docs/source/faqs.rst#why-static-linking-to-llvm

This behavior causes problems when python uses another version of llvm,
for example, mesa llvmpipe. This PR force llvmlite statically link to llvm.

Besides, this PR also use llvm as buildInputs rather than nativeBuildInputs
since it's a library that llvmlite use rather than a part of build system.

This PR also cleans outdated patches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants