alos2_to_nisar_l1.py can't convert alos2 ceos to h5

[zyxgis]

os : centos 7
python: 3.9
cxx: gcc11

---

i use alos2_to_nisar_l1.py to convert alos2 ceos to h5, but i get the follow error:

$ /python/bin/python3 /home/testUser/isce/share/nisar/examples/alos2_to_nisar_l1.py -l HH -i /home/testUser/test/ceos/0000471428_001001_ALOS2441290770-220725/ -o /home/testUser/test/ceos/output-hh2.h5
looking for available polarizations...
found polarization: HH
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
#000: H5O.c line 1175 in H5Ovisit_by_name3(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
#1: H5VLint.c line 1741 in H5VL_vol_object(): invalid identifier
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
#000: H5D.c line 132 in H5Dcreate2(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
Traceback (most recent call last):
File "/home/testUser/isce/share/nisar/examples/alos2_to_nisar_l1.py", line 720, in
process(args=args)
File "/home/testUser/isce/share/nisar/examples/alos2_to_nisar_l1.py", line 139, in process
orbit = construct_nisar_hdf5(args.outh5, leader)
File "/home/testUser/isce/share/nisar/examples/alos2_to_nisar_l1.py", line 253, in construct_nisar_hdf5
set_h5_orbit(orbit_group, orbit)
File "/home/testUser/isce/share/nisar/examples/alos_to_nisar_l0b.py", line 141, in set_h5_orbit
orbit.save_to_h5(group)
RuntimeError: Caught an unknown exception!

[bhawkins]

My guess is that your h5py and isce3 are linked against different versions of the HDF5 library. You might be able to confirm this with ldd. For example, figure out where your h5py is installed with

python3 -c "import h5py; print(h5py.__file__)"

and there should be a bunch of *.so files in the same directory. You can run ldd on one of those .so files and libisce3.so to see if they link to the same libhdf5.so file (or if it was statically linked then it won't show up). My usual workaround is to install h5py from source, manually making sure to link with the same HDF5 library I used to compile isce3.

The isce3 developers plan to refactor the code to avoid sharing HDF5 objects between the C++ and Python layers, but that effort will take some time.

[zyxgis]

python3 -c "import h5py; print(h5py.__file__)"
/python/lib/python3.9/site-packages/h5py/__init__.py

ldd /home/testUser/isce/lib64/libisce3.so
        linux-vdso.so.1 =>  (0x00007ffd03f41000)
        libgdal.so.31 => /home/testUser/gdal/lib/libgdal.so.31 (0x00007fb3d9492000)
        libpyre.so => /python/compile/lib64/libpyre.so (0x00007fb3db258000)
        libhdf5_cpp.so.200 => /home/testUser/gdal/lib/libhdf5_cpp.so.200 (0x00007fb3db1da000)
        libhdf5.so.200 => /home/testUser/gdal/lib/libhdf5.so.200 (0x00007fb3d9054000)
        libhdfs.so.0.0.0 => /home/testUser/gdal/lib/libhdfs.so.0.0.0 (0x00007fb3d8e41000)
        libcrypto.so.3 => /home/testUser/gdal/lib64/libcrypto.so.3 (0x00007fb3d89d9000)
        libcurl.so.4 => /home/testUser/gdal/lib/libcurl.so.4 (0x00007fb3db14d000)
        libsz.so.2 => /home/testUser/gdal/lib/libsz.so.2 (0x00007fb3db138000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fb3d87c3000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fb3d85bf000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fb3d82bd000)
        libjournal.so => /python/compile/lib64/libjournal.so (0x00007fb3db0a4000)
        libfftw3f.so.3 => /home/testUser/isce/lib/libfftw3f.so.3 (0x00007fb3d819d000)
        libfftw3f_threads.so.3 => /home/testUser/isce/lib/libfftw3f_threads.so.3 (0x00007fb3db09a000)
        libfftw3_threads.so.3 => /home/testUser/isce/lib/libfftw3_threads.so.3 (0x00007fb3db091000)
        libgomp.so.1 => /lib64/libgomp.so.1 (0x00007fb3d7f77000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb3d7d5b000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fb3d7be7000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb3d79d1000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fb3d7603000)
        libSFCGAL.so.1 => /home/testUser/gdal/lib64/libSFCGAL.so.1 (0x00007fb3d6b42000)
        libgeos_c.so.1 => /home/testUser/gdal/lib64/libgeos_c.so.1 (0x00007fb3d6afa000)
        libexpat.so.1 => /lib64/libexpat.so.1 (0x00007fb3d68cf000)
        libopenjp2.so.7 => /lib64/libopenjp2.so.7 (0x00007fb3d667c000)
        libnetcdf.so.19 => /home/testUser/gdal/lib/libnetcdf.so.19 (0x00007fb3d6427000)
        libpng15.so.15 => /lib64/libpng15.so.15 (0x00007fb3d61fc000)
        libpq.so.5 => /home/testUser/gdal/lib/libpq.so.5 (0x00007fb3d5fac000)
        liblz4.so.1 => /lib64/liblz4.so.1 (0x00007fb3d5d9d000)
        libzstd.so.1 => /home/testUser/gdal/lib/libzstd.so.1 (0x00007fb3d5cb5000)
        liblzma.so.5 => /home/testUser/gdal/lib/liblzma.so.5 (0x00007fb3d5c8d000)
        libproj.so.25 => /home/testUser/gdal/lib64/libproj.so.25 (0x00007fb3d588f000)
        libsqlite3.so.0 => /home/testUser/gdal/lib/libsqlite3.so.0 (0x00007fb3d574c000)
        libtiff.so.5 => /home/testUser/gdal/lib/libtiff.so.5 (0x00007fb3d56d3000)
        libjpeg.so.62 => /lib64/libjpeg.so.62 (0x00007fb3d547e000)
        libdeflate.so.0 => /home/testUser/gdal/lib/libdeflate.so.0 (0x00007fb3db077000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fb3d5276000)
        libpcre2-8.so.0 => /home/testUser/gdal/lib/libpcre2-8.so.0 (0x00007fb3d51da000)
        libiconv.so.2 => /home/testUser/gdal/lib/libiconv.so.2 (0x00007fb3d50f4000)
        libxml2.so.2 => /lib64/libxml2.so.2 (0x00007fb3d4d8a000)
        libjvm.so => /home/testUser/java/jdk1.8.0_211/jre/lib/amd64/server/libjvm.so (0x00007fb3d3da1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb3db04d000)
        libssl.so.3 => /home/testUser/gdal/lib64/libssl.so.3 (0x00007fb3d3cf8000)
        libfftw3.so.3 => /home/testUser/isce/lib/libfftw3.so.3 (0x00007fb3d3bd1000)
        libgmpxx.so.4 => /lib64/libgmpxx.so.4 (0x00007fb3d39ca000)
        libboost_thread.so.1.80.0 => /home/testUser/gdal/lib/libboost_thread.so.1.80.0 (0x00007fb3d39af000)
        libboost_system.so.1.80.0 => /home/testUser/gdal/lib/libboost_system.so.1.80.0 (0x00007fb3d39aa000)
        libboost_serialization.so.1.80.0 => /home/testUser/gdal/lib/libboost_serialization.so.1.80.0 (0x00007fb3d3954000)
        libmpfr.so.6 => /home/testUser/gdal/lib/libmpfr.so.6 (0x00007fb3d36aa000)
        libgmp.so.10 => /home/testUser/gdal/lib/libgmp.so.10 (0x00007fb3d3633000)
        libgeos.so.3.11.0 => /home/testUser/gdal/lib64/libgeos.so.3.11.0 (0x00007fb3d3391000)
        libhdf5_hl.so.200 => /home/testUser/gdal/lib/libhdf5_hl.so.200 (0x00007fb3d336c000)
        libssl.so.10 => /lib64/libssl.so.10 (0x00007fb3d30fa000)
        libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fb3d2c97000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fb3d2a4a000)
        libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2 (0x00007fb3d27eb000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fb3d2502000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fb3d22fd000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fb3d20ca000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fb3d1eba000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fb3d1cb5000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fb3d1a9b000)
        liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007fb3d188c000)
        libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007fb3d166e000)
        libssl3.so => /lib64/libssl3.so (0x00007fb3d1409000)
        libsmime3.so => /lib64/libsmime3.so (0x00007fb3d11e1000)
        libnss3.so => /lib64/libnss3.so (0x00007fb3d0ea6000)
        libnssutil3.so => /lib64/libnssutil3.so (0x00007fb3d0c76000)
        libplds4.so => /lib64/libplds4.so (0x00007fb3d0a72000)
        libplc4.so => /lib64/libplc4.so (0x00007fb3d086c000)
        libnspr4.so => /lib64/libnspr4.so (0x00007fb3d062d000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fb3d0405000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fb3d01ce000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fb3cff6b000)
        libfreebl3.so => /lib64/libfreebl3.so (0x00007fb3cfd68000)
		

[zyxgis]

i compile hdf5-1.12.2.tar.gz
i install h5py-3.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl to python3

[bhawkins]

Still need to see the output of ldd on one of the .so files in /python/lib/python3.9/site-packages/h5py. My guess is that it will be using a different libhdf5.so than your isce3 library uses, which is the problem. If that's the case, my usual workaround is to install h5py from source (not a binary .whl).