installation issues with cmake on windows

[clementhelsens]

hello,

I'm trying to use this tool to cluster pixels in living embryos movies, the idea is to use it to segment cells from fluorecent signal in 3d. But I'm already hitting installation issues. What I did:

conda create -n 3d_clue pip
conda activate 3d_clue
pip install cmake
pip install CLUEstering

and the error is:

Collecting CLUEstering
  Using cached CLUEstering-2.2.7.tar.gz (3.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      Traceback (most recent call last):
        File "/Users/helsens/miniconda3/envs/3d_clue/bin/cmake", line 5, in <module>
          from cmake import cmake
      ModuleNotFoundError: No module named 'cmake'
      None
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

while if I open python prompt and try from cmake import cmake it works just fine

a which cmake gives ~/miniconda3/envs/3d_clue/bin/cmake

any idea of what's going on?
thanks in advance

[clementhelsens]

Follow-up comment, if I install cmake wtih conda, it works to install CLUEstering but now, trying the example fails when trying to import clue, see:

(3d_clue2) helsens@SV-85M-017 3d_clustering % python
Python 3.12.6 | packaged by conda-forge | (main, Sep 22 2024, 14:07:06) [Clang 17.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import CLUEstering as clue
zsh: segmentation fault  python
(3d_clue2) helsens@SV-85M-017 3d_clustering % 

I'm on a mac M1

[clementhelsens]

If I try to install from source (using windows now as this is where I get most powerful GPU) I now can build and install but I get an error when loading clusterer:

(3d_clue) C:\Users\helsens\software\3d_clustering\CLUEstering>python
Python 3.11.9 | packaged by Anaconda, Inc. | (main, Apr 19 2024, 16:40:41) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import CLUE_Convolutional_Kernels as clue_kernels
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'CLUE_Convolutional_Kernels'

would it be possible to have some help with the installation?

[sbaldu]

Hi @clementhelsens, thanks for reporting the issue. Based on the last error that you showed it looks like there was an error during compilation and it was unable to generate the binding modules. Can you try to install with the verbose flag, in order to get more details? Thanks

[clementhelsens]

Thanks @sbaldu , sure I can try, but not sure how to do it.
To make it work on windows I had to change the make command to
make_command = ['cmake', '--build', 'build', '--config', 'Release']
and also the C++ version to 20

Maybe you could outline a full installation procedure from source that I could follow to make sure we are on the same page ?
Thanks for the help

[sbaldu]

Your changes to the building commands are ok. If you are installing it from source you should be able to do it by simply cloning the repository recursively and running pip install -v .

[clementhelsens]

Thanks @sbaldu. (EDIT Oct 1st)

Here is what I'm doing, still no luck

git clone --recurse-submodules https://github.com/cms-patatrack/CLUEstering.git CLUEstering_test
cd CLUEstering_test
conda create -n clue_test pip
conda activate clue_test
conda install cmake
conda install conda-forge::boost

Edit CMakeLists.txt and change set(CMAKE_CXX_STANDARD 17) to set(CMAKE_CXX_STANDARD 20)
Edit setup.py and change make_command = ['make', '-C', 'build'] to make_command = ['cmake', '--build', 'build', '--config', 'Release']

then

pip install . -v

but in the python prompt

import CLUEstering
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\helsens\software\3d_clustering\CLUEstering_test\CLUEstering\__init__.py", line 1, in <module>
    from CLUEstering.CLUEstering import clusterer
  File "C:\Users\helsens\software\3d_clustering\CLUEstering_test\CLUEstering\CLUEstering.py", line 21, in <module>
    import CLUE_Convolutional_Kernels as clue_kernels
ModuleNotFoundError: No module named 'CLUE_Convolutional_Kernels'

I'm using python 3.12, not sure if that could be problematic.

Full install log attached:
install_CLUEstering.pdf

[sbaldu]

Hi @clementhelsens, a little update. I'm trying to reproduce the error but I haven't been able to do so. On linux it works even using conda.
By looking at the log that you've given it looks like the modules are being generated correctly. One thing that you can check is that in the CLUEstering_test/build/lib/CLUEstering/lib folder there are the shared libraries produced during compilation.

[clementhelsens]

Hi @sbaldu ,
inside CLUEstering_test/build/lib/CLUEstering/ I have only __init__.py and CLUEstering.py no folder lib with shared libraries

[sbaldu]

The problem seems to be with MSVC, I'm working on finding the cause. In the meantime I'd suggest trying to use WSL if possible.

[clementhelsens]

Thanks @sbaldu , I'll give it a try. I'm not very familiar with windows, but that's what's the lab I'm in is using, so no choice...
One thing I found though is that in the CMakeLists.txt there are a few places where linux commands are used instead of cmake commands for instance,
replace

execute_process(COMMAND mkdir -p ./CLUEstering/lib)

by

file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/lib/CLUEstering/lib)

and (1 example out of several)

set_target_properties(CLUE_CPU_Serial PROPERTIES LIBRARY_OUTPUT_DIRECTORY
                                                 ./lib/CLUEstering/lib/)

by

set_target_properties(CLUE_Convolutional_Kernels PROPERTIES
    LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib/CLUEstering/lib)

also the symlinks ln -sfare linux specific

Finally as the librairies are not .so on windows, I think the CMakeLists.txt and possibly the setup.py should also take this into account.
I tried a few things, it seemed compile well, but still same error when loading.

Thanks for the support!
Clement

[clementhelsens]

Small update, with WSL it does work out of the box

[sbaldu]

Small update, with WSL it does work out of the box

I'm glad to hear it. If you have any questions feel free to ask.

Thanks also for pointing out the problems with the CMake file. We developed it with Linux in mind, but I'll fix them ASAP in order to make it work on Windows as well.

[clementhelsens]

On that topic, I just sent you an email to ask you some help about the clustering itself using 3D images :)