KeyError: 'pkgs_dirs'

[valeriupredoi]

Hi @maresb Ben, we're seeing this failure at creating a linux64 conda lock file:

• conda-lock 2.5.7 build pyhd8ed1ab_0 conda-forge
• click 8.1.7 build unix_pyh707e725_0 conda-forge
• invocation: conda-lock lock --platform linux-64 -f environment.yml --mamba --kind explicit
• see the actual fail at https://github.com/ESMValGroup/ESMValTool/actions/runs/11286190143/job/31498329250
• trace:

Traceback (most recent call last):
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/bin/conda-lock", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 1403, in lock
    lock_func(
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 1111, in run_lock
    make_lock_files(
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 393, in make_lock_files
    fresh_lock_content = create_lockfile_from_spec(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 836, in create_lockfile_from_spec
    deps = _solve_for_arch(
           ^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_lock.py", line 748, in _solve_for_arch
    conda_deps = solve_conda(
                 ^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_solver.py", line 159, in solve_conda
    dry_run_install = solve_specs_for_arch(
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_solver.py", line 383, in solve_specs_for_arch
    return _reconstruct_fetch_actions(conda, platform, dryrun_install)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/miniconda3/envs/esmvaltool-fromlock/lib/python3.12/site-packages/conda_lock/conda_solver.py", line 277, in _reconstruct_fetch_actions
    for d in json.loads(
             ^^^^^^^^^^^
KeyError: 'pkgs_dirs'

Unfortunately I can't reproduce this on my local machine (ie all works fine, with same dependencies), but it is 100% reproducible on the Github Actions machine. Any clues? Many thanks in advance! 🍻

[maresb]

Hey there @valeriupredoi! Working on an OSS project with a visible workflow makes this a whole lot easier than usual to diagonse! 😁

I'd recommend adding a workflow dispatch trigger to give you the following button:

I think this is probably due to Micromamba v2. Experiment in progress at
ESMValGroup/ESMValTool#3773

[valeriupredoi]

ah interesting, cheers, Ben! Will go there now 🍺

[maresb]

It was indeed v2: ESMValGroup/ESMValTool#3771

I'm a bit concerned about the KeyError reported above. This needs more investigation.

[maresb]

Thanks so much @valeriupredoi for the bug report! It turns out it was an obscure bug in conda-lock, triggered from a side-effect of the upgrade. In this case mamba v2 was not at fault. Once I release conda-lock v3 you can unpin mamba.

[valeriupredoi]

excellent @maresb - glad I could help, and well done for figuring out what the actual problem was! Many thanks, I will make a note for us, so we unpin mamba as soon as v3 comes out. Also, close this then? 🍺

[valeriupredoi]

ah nvmd, just saw #733 is still open 👍