Replacing ldd output with elf-inspector

[stefan6419846]

As proposed by @pombredanne in stefan6419846/license_tools@71cea6c#r136933876, I just had a deeper look at elf-inspector to replace plain ldd calls. I have to admit that I am no ELF expert.

I used two examples to check how both approaches behave in comparison:

1. /usr/bin/bc:

   $ ldd /usr/bin/bc
       linux-vdso.so.1 (0x00007ffd973cb000)
       libreadline.so.7 => /lib64/libreadline.so.7 (0x00007fc22c800000)
       libc.so.6 => /lib64/libc.so.6 (0x00007fc22c609000)
       libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007fc22ceb3000)
       /lib64/ld-linux-x86-64.so.2 (0x00007fc22cf0b000)
   $ python -c "from elf_inspector import elf; print(list(elf.get_elf_dependencies(location='/usr/bin/bc')))"
   ['libreadline.so.7', 'libc.so.6']
   

2. Some old OpenCV build I found on my hard drive (https://files.pythonhosted.org/packages/67/50/665a503167396ad347957bea0bd8d5c08c865030b2d1565ff06eba613780/opencv_python-4.5.5.64-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl, running unzip beforehand):

   $ ldd /path/to/cv2/cv2.abi3.so
       linux-vdso.so.1 (0x00007fffc3f9c000)
       libavcodec-65fa80df.so.58.134.100 => /path/to/cv2/../opencv_python.libs/libavcodec-65fa80df.so.58.134.100 (0x00007f8bcf2a3000)
       libavformat-8ef5c7db.so.58.76.100 => /path/to/cv2/../opencv_python.libs/libavformat-8ef5c7db.so.58.76.100 (0x00007f8bcf024000)
       libavutil-9c768859.so.56.70.100 => /path/to/cv2/../opencv_python.libs/libavutil-9c768859.so.56.70.100 (0x00007f8bced77000)
       libswscale-e6451464.so.5.9.100 => /path/to/cv2/../opencv_python.libs/libswscale-e6451464.so.5.9.100 (0x00007f8bcece4000)
       libQt5Widgets-e69d94fb.so.5.15.0 => /path/to/cv2/../opencv_python.libs/libQt5Widgets-e69d94fb.so.5.15.0 (0x00007f8bce3bf000)
       libQt5Gui-ba0a2070.so.5.15.0 => /path/to/cv2/../opencv_python.libs/libQt5Gui-ba0a2070.so.5.15.0 (0x00007f8bcdab7000)
       libQt5Test-c38a5234.so.5.15.0 => /path/to/cv2/../opencv_python.libs/libQt5Test-c38a5234.so.5.15.0 (0x00007f8bcda4c000)
       libQt5Core-39545cc7.so.5.15.0 => /path/to/cv2/../opencv_python.libs/libQt5Core-39545cc7.so.5.15.0 (0x00007f8bcd349000)
       libz.so.1 => /lib64/libz.so.1 (0x00007f8bcd30a000)
       libopenblas-r0-f650aae0.3.3.so => /path/to/cv2/../opencv_python.libs/libopenblas-r0-f650aae0.3.3.so (0x00007f8bcac00000)
       libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8bcd2e4000)
       librt.so.1 => /lib64/librt.so.1 (0x00007f8bcd2da000)
       libdl.so.2 => /lib64/libdl.so.2 (0x00007f8bcd2d5000)
       libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f8bca9bb000)
       libm.so.6 => /lib64/libm.so.6 (0x00007f8bca86f000)
       libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8bcd2af000)
       libc.so.6 => /lib64/libc.so.6 (0x00007f8bca678000)
       /lib64/ld-linux-x86-64.so.2 (0x00007f8bd4173000)
       libswresample-99364a1c.so.3.9.100 => /path/to/cv2/../opencv_python.libs/libswresample-99364a1c.so.3.9.100 (0x00007f8bcd28e000)
       libvpx-1016051d.so.7.0.0 => /path/to/cv2/../opencv_python.libs/libvpx-1016051d.so.7.0.0 (0x00007f8bca37b000)
       libbz2-a273e504.so.1.0.6 => /path/to/cv2/../opencv_python.libs/libbz2-a273e504.so.1.0.6 (0x00007f8bca000000)
       libssl-b92f8066.so.1.1 => /path/to/cv2/../opencv_python.libs/libssl-b92f8066.so.1.1 (0x00007f8bcd1ed000)
       libcrypto-09fe7800.so.1.1 => /path/to/cv2/../opencv_python.libs/libcrypto-09fe7800.so.1.1 (0x00007f8bc9cfe000)
       libGL.so.1 => /usr/lib64/libGL.so.1 (0x00007f8bc9a00000)
       libpng15-ce838cd1.so.15.13.0 => /path/to/cv2/../opencv_python.libs/libpng15-ce838cd1.so.15.13.0 (0x00007f8bc9600000)
       libgthread-2.0.so.0 => /usr/lib64/libgthread-2.0.so.0 (0x00007f8bcd1ea000)
       libglib-2.0.so.0 => /usr/lib64/libglib-2.0.so.0 (0x00007f8bca246000)
       libgfortran-91cc3cb1.so.3.0.0 => /path/to/cv2/../opencv_python.libs/libgfortran-91cc3cb1.so.3.0.0 (0x00007f8bc9200000)
       libGLX.so.0 => /usr/lib64/libGLX.so.0 (0x00007f8bc8e00000)
       libGLdispatch.so.0 => /usr/lib64/libGLdispatch.so.0 (0x00007f8bc8a00000)
       libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x00007f8bc8600000)
       libquadmath-96973f99.so.0.0.0 => /path/to/cv2/../opencv_python.libs/libquadmath-96973f99.so.0.0.0 (0x00007f8bc8200000)
       libX11.so.6 => /usr/lib64/libX11.so.6 (0x00007f8bc98bf000)
       libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x00007f8bc7e00000)
       libXau.so.6 => /usr/lib64/libXau.so.6 (0x00007f8bc7a00000)
   $ python -c "from elf_inspector import elf; print(list(elf.get_elf_dependencies(location='/path/to/cv2/cv2.abi3.so')))"
   ['libavcodec-65fa80df.so.58.134.100', 'libavformat-8ef5c7db.so.58.76.100', 'libavutil-9c768859.so.56.70.100', 'libswscale-e6451464.so.5.9.100', 'libQt5Widgets-e69d94fb.so.5.15.0', 'libQt5Gui-ba0a2070.so.5.15.0', 'libQt5Test-c38a5234.so.5.15.0', 'libQt5Core-39545cc7.so.5.15.0', 'libz.so.1', 'libopenblas-r0-f650aae0.3.3.so', 'libpthread.so.0', 'librt.so.1', 'libdl.so.2', 'libstdc++.so.6', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
   

My observations:

• elf-inspector only supports returning absolute paths for DWARF files. For Manylinux wheels like OpenCV, it is not clear where each dependency is used from.
• elf-inspector lists the ld-linux-x86-64.so.2 and libc.so.6 dependencies for OpenCV, while it does not for bc, although ldd states that both are linking to them.
• bc links against libtinfo.so.6 according to ldd, but elf-inspector does not list it.

Is there an easy way to resolve these discrepancies and get rid of the limits of the current approach? At the moment, it seems like ldd is more reliable for this use case, but maybe I am just missing something important here.

[pombredanne]

@stefan6419846 Thanks for the detailed report. The code calls pyelftools at the moment and we are likely filtering out too much or not collecting the correct details. This would be the first place to look. Beyond this we could look into using lief as an alternative, but this is a bit more involved as this is native code.