Cannot install for Python 3.12

[pintergreg]

I use poetry, but I think the problem is with the Python version (3.12), as I cannot install with pip either.

poetry install
Updating dependencies
Resolving dependencies... (5.9s)

Writing lock file
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  - Installing pyvalhalla (3.2.0): Failed

  RuntimeError

  Unable to find installation candidates for pyvalhalla (3.2.0)

  at /usr/lib/python3.12/site-packages/poetry/installation/chooser.py:74 in choose_for
       70│ 
       71│             links.append(link)
       72│ 
       73│         if not links:
    →  74│             raise RuntimeError(f"Unable to find installation candidates for {package}")
       75│ 
       76│         # Get the best link
       77│         chosen = max(links, key=lambda link: self._sort_key(package, link))
       78│ 

Cannot install pyvalhalla.

$ pip install --user -U pyvalhalla
ERROR: Could not find a version that satisfies the requirement pyvalhalla (from versions: none)
ERROR: No matching distribution found for pyvalhalla

[baileyheading]

Did you try making an environment with python 3.11 to test your own theory?

also, what platform are you on?

[nilsnolde]

Yeah we don’t publish wheels for 3.12 yet. There’s no particular reason, I just didn’t touch the project since it came about. I can try later and hope it works without any changes.

[pintergreg]

Did you try making an environment with python 3.11 to test your own theory?

Yes, I did. And it works.

also, what platform are you on?

I was using pyvalhalla under Python 3.11 for a year, but Archlinux updated to 3.12 recently.

There’s no particular reason, I just didn’t touch the project since it came about.

I disagree with the first part of the sentence. There is a very good reason to support 3.12 as it is the latest stable Python version, which was just included into Ubuntu 24.04 LTS.

As for the second part, I can perfectly understand if you don't have time for this project (anymore).

I can try later and hope it works without any changes.

Thank you.

[nilsnolde]

I disagree with the first part of the sentence.

that’s not what I was referring to. I meant that there no reason why we didn’t publish wheels for that yet other than not touching the project for a while, there was no other need to.

anyways, we’ll try soon. I’m on Manjaro, their arch mirrors should update soon as well then, but so far we’re on 3.11.

[nilsnolde]

at least it worked for linux. we'll see now if mac/win also works. then I'll publish them for 3.12.

[nilsnolde]

it won't fix for now, see #37 for reasoning

[stefansc1]

Is there an update to this? Or at least a way for the Linux users to access the fixed version?

[baileyheading]

Is there an update to this? Or at least a way for the Linux users to access the fixed version?

to respond quickly, a way you can access them yourself is to build from https://github.com/valhalla/valhalla with python bindings on. To make things even simpler, you can just swap the static libraries that you build with a current pyvalhalla install. Just need to make sure that the python in CMakeList is pathed to a version of python 3.12

[stefansc1]

First of all, thank you for the quick reply. However, I am still unsure what to do.

to respond quickly, a way you can access them yourself is to build from https://github.com/valhalla/valhalla with python bindings on.

This seems to be the main Valhalla project. Is it really necessary to build everything if I only want to use the Python bindings? Seems like overkill.

To make things even simpler, you can just swap the static libraries that you build

... so I still need to build everything?

with a current pyvalhalla install. Just need to make sure that the python in CMakeList is pathed to a version of python 3.12

For someone mainly working with Python, this is majorly confusing. Going by the Valhalla docu, these are steps I would take:

• clone the Valhalla repo
• run scripts/install-linux-deps.sh
• cmake -B build -DCMAKE_BUILD_TYPE=Release
• make -C build -j$(nproc)
• sudo make -C build install

Not sure WHAT it installs (and if only into build or sneaking somewhere else as well) and what of it I actually need. Generating Python bindings seems to be the default in the CMakeList.txt, but I can't find where to specify the Python version to use (although there is only one installed). Then...? I still have a working pyvalhalla installation in an old virtual Python environment. I think it's only referenced in lib/python3.10/site-packages/valhalla/, which contains a file called _valhalla.cpython-310-x86_64-linux-gnu.so, alongside some Python wrappers. If I just copy that directory plus pyvalhalla.libs and dist-info over to my Python3.12 environment, I get ModuleNotFoundError: No module named 'valhalla._valhalla' from the actor.py. So I am not sure if just copying any lib/so files from the Valhalla build is enough.

[baileyheading]

You can turn off a bunch of stuff that you don't need yes through the CMakeList parameters.

I also found it a bit tricky to make it choose the right python environment so I manually set it.
Everywhere it mentions python I set it like this.

## Python bindings
if(ENABLE_PYTHON_BINDINGS)

  set(Python_EXECUTABLE "/Users/me/miniconda3/envs/env_name/bin/python")

When you copied over the .so did you rename it to be the same as in the valhalla folder in site packages. it sounds like you're very close to having it right.

EDIT: the reason I say to use the existing ones for pyvalhalla instead of the python bindings ones is because it didn't work out the box for me. I think that the maker for this repo probably encountered and fixed various issues and modified things a little possibly

[stefansc1]

You can turn off a bunch of stuff that you don't need yes through the CMakeList parameters.

Good to know. I will have a closer look at the options in CMakeLists, but without intimate knowledge of valhalla I doubt it will make much sense to me. Probably faster to let it built stuff than trying to understand why it builds stuff.

I also found it a bit tricky to make it choose the right python environment

Shouldn't be a problem for me, I only have Python 3.12 installed.

When you copied over the .so did you rename it to be the same as in the valhalla folder in site packages.

So far, I haven't built anything, just copied three folders over from the old virtual environment into the new one to see what happens. So no renaming (yet). I wanted to wait for confirmation if the steps are going into the right direction before building.

[baileyheading]

the python bindings will build into here (for reference)

valhalla/build/src/bindings/python/valhalla

You can turn these two off as I believe they are basically making routing tiles and the http routing service I think (if I'm not mistaken).. maybe you may even like to use these.

-DENABLE_DATA_TOOLS=OFF
-DENABLE_SERVICES=OFF

I would recommend trying to build it and learning by trial and error really, nothing to lose. I suspect that scripts/install-linux-deps.sh installs them onto your system as they are needed to build Valhalla but linux is the one system I don't build for actually.

If it goes wrong, delete the build folder and try and figure out where it all went wrong, and build again

EDIT: also ModuleNotFoundError: No module named 'valhalla._valhalla'.. I would not expect the .so file to work that is made for 3.10 in 3.12 at all. that's not the error I would expect but either way it needs to be the one made with the right python

[chrstnbwnkl]

Yep for now you need to build valhalla yourself. You can just pass the target to CMake, so it will only build what's necessary:

cmake --build build --target python_valhalla 

Also, if you want to install it into a virtual environment, it should be sufficient to activate that env in your shell when configuring the build:

source <path/to/venv>
cmake -B build -DENABLE_PYTHON_BINDINGS=ON

# should output something like 
-- Installing python modules to <path/to/env>

[stefansc1]

For future reference, this is what I did and seemed to work:

# get Valhalla sources with all submodules
git clone --recurse-submodules --remote-submodules https://github.com/valhalla/valhalla.git
cd valhalla
# install dependencies
./scripts/install-linux-deps.sh
# build Valhalla
cmake -B build -DENABLE_PYTHON_BINDINGS=ON -DENABLE_DATA_TOOLS=OFF -DENABLE_SERVICES=OFF -DCMAKE_BUILD_TYPE=Release
# compile sources
make -C build -j$(nproc)

In the meantime, copy pyvalhalla.libs, pyvalhalla-3.2.0.dist-info and valhalla directories from old virtual environment into new one. After building has finished, copy build/src/bindings/python/valhalla/python_valhalla.cpython-312-x86_64-linux-gnu.so to the virtual environment lib/python3.12/site-packages/valhalla/. The old .so file can be deleted. I had slight issues with imports, so I replaced from ._valhalla import * in __init__.py and from ._valhalla import _Actor in actor.py with

try:
    from .python_valhalla import *
except ModuleNotFoundError:
    from python_valhalla import *

and

try:
    from .python_valhalla import _Actor
except ModuleNotFoundError:
    from python_valhalla import _Actor

(exactly as in the new Valhalla Python files). There were some other differences in the Valhalla Python files, for example, config.get_config() has lost the verbose argument. Therefore, I am using the old Python files with the modified imports, which works for my purposes.

Sidenote: cmake does not know target argument

[thomthom]

Thank you for posting back the steps to build the 3.12 bindings. Its been on my list of look into that as well. (Though for me I need Windows build). But good to have a more details set of steps.