qiime2__diversity__core_metrics_phylogenetic error

[bernt-matthias]

2023.5.1+q2galaxy.2023.5.0.2

let me know if I should provide additional info.

This plugin encountered an error:
Command '['ssu', '-i', '/tmp/qiime2/b
erntm/data/0181e66c-c1cd-4a8f-841f-b2
6e294cdf4f/data/feature-table.biom',
'-t', '/tmp/qiime2/berntm/data/095f38
0b-f423-4a9a-a922-70c3d6fc7284/data/t
ree.nwk', '-m', 'unweighted', '-o',
'/tmp/q2-LSMatFormat-jwa2_krh']'
returned non-zero exit status 1.
                                                                                                                                                                                                                                                               
:(
Matplotlib created a temporary config/cache directory at /tmp/matplotlib-okxb46uu because the default path (/home/qiime2/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Current file:     /opt/conda/conda-bld/unifrac-binaries_1664395747097/work/src/unifrac_cmp.cpp
        function: _ZN6su_acc21UnifracUnweightedTaskIdE4_runEjPKd
        line:     558
This file was compiled: -ta=tesla:cc35,cc50,cc60,cc60,cc70,cc75,cc80,cc80
/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/sklearn/metrics/pairwise.py:1776: DataConversionWarning: Data was converted to boolean for metric jaccard
  warnings.warn(msg, DataConversionWarning)
/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:143: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.008521727917718724 and the largest is 5.200826700955479.
  warn(
/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:143: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.0934827577673413 and the largest is 4.863627869126633.
  warn(
Traceback (most recent call last):
  File "/opt/conda/envs/qiime2-2023.5/bin/q2galaxy", line 11, in <module>
    sys.exit(root())
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2galaxy/__main__.py", line 98, in run
    action_runner(plugin, action, config)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2galaxy/core/drivers/action.py", line 30, in action_runner
    results = _execute_action(action, action_kwargs,
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2galaxy/core/drivers/stdio.py", line 38, in wrapped
    return function(*args, **kwargs)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2galaxy/core/drivers/action.py", line 115, in _execute_action
    return action(**action_kwargs)
  File "<decorator-gen-176>", line 2, in core_metrics_phylogenetic
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
    outputs = self._callable_executor_(
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 590, in _callable_executor_
    outputs = self._callable(scope.ctx, **view_args)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_diversity/_core_metrics.py", line 66, in core_metrics_phylogenetic
    dms += unweighted_unifrac(table=cr.rarefied_table, phylogeny=phylogeny,
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/context.py", line 140, in deferred_action
    return action_obj._bind(
  File "<decorator-gen-724>", line 2, in unweighted_unifrac
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
    outputs = self._callable_executor_(
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 509, in _callable_executor_
    output_views = self._callable(**view_args)
  File "<decorator-gen-133>", line 2, in unweighted_unifrac
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 69, in _disallow_empty_tables
    return wrapped_function(*args, **kwargs)
  File "<decorator-gen-132>", line 2, in unweighted_unifrac
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 112, in _validate_requested_cpus
    return wrapped_function(*bound_arguments.args, **bound_arguments.kwargs)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_diversity_lib/beta.py", line 221, in unweighted_unifrac
    _omp_cmd_wrapper(threads, cmd)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 128, in _omp_cmd_wrapper
    return _run_external_cmd(cmd, verbose=verbose, env=env)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_diversity_lib/_util.py", line 122, in _run_external_cmd
    return subprocess.run(cmd, check=True, env=env)
  File "/opt/conda/envs/qiime2-2023.5/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ssu', '-i', '/tmp/qiime2/berntm/data/0181e66c-c1cd-4a8f-841f-b26e294cdf4f/data/feature-table.biom', '-t', '/tmp/qiime2/berntm/data/095f380b-f423-4a9a-a922-70c3d6fc7284/data/tree.nwk', '-m', 'unweighted', '-o', '/tmp/q2-LSMatFormat-jwa2_krh']' returned non-zero exit status 1.

[bernt-matthias]

@ebolyen the error is actually:

https://forum.qiime2.org/t/plugin-error-from-diversity-error-with-ssu-computing-phylogenetic-diversity-metrics/25108/7
biocore/unifrac-binaries#24

this could be solved by prefixing the command with export UNIFRAC_USE_GPU=N &&. Wondering how we can solve this. Locally on my instance I can set this environment variable for all jobs. But this bug will stall the progress on the qiime2 Galaxy workflows unless a new version fixing this is released.

Wondering how/if this could be fixed. Just open a PR and update the single tool?

[ebolyen]

@bernt-matthias, maybe. This is definitely pretty frustrating. I think another approach would be to force our Docker containers to set that environment variable. As it will almost invariably go poorly in other contexts as well.

[ebolyen]

qiime2/distributions@9118e0a

Will need to rebuild the docker images, but then it should get picked up by the upcoming toolshed release.

EDIT: currently re-running https://github.com/qiime2/distributions/actions/runs/9308363278