add test showing error during import job preparion with latest Galaxy #51
Conversation
the following Traceback can be found in the logs (the test itself is not
executed)
```
Traceback (most recent call last):
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/jobs/runners/__init__.py", line 291, in prepare_job
job_wrapper.prepare()
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/jobs/__init__.py", line 1254, in prepare
) = tool_evaluator.build()
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/tools/evaluation.py", line 566, in build
global_tool_logs(self._build_config_files, config_file, "Building Config Files")
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/tools/evaluation.py", line 96, in global_tool_logs
raise e
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/tools/evaluation.py", line 92, in global_tool_logs
return func()
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/tools/evaluation.py", line 633, in _build_config_files
self.__write_workdir_file(config_filename, config_text, param_dict, is_template=is_template)
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/tools/evaluation.py", line 756, in __write_workdir_file
value = fill_template(content, context=context, python_template_version=self.tool.python_template_version)
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/util/template.py", line 143, in fill_template
raise first_exception or e
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/util/template.py", line 87, in fill_template
return unicodify(t, log_exception=False)
File "/tmp/tmpiottroee/galaxy-dev/lib/galaxy/util/__init__.py", line 1153, in unicodify
value = str(value)
File "/home/berntm/.planemo/gx_venv_3/lib/python3.10/site-packages/Cheetah/Template.py", line 1053, in __unicode__
return getattr(self, mainMethName)()
File "cheetah_DynamicallyCompiledCheetahTemplate_1705952649_1165519_57877.py", line 119, in respond
File "cheetah_DynamicallyCompiledCheetahTemplate_1705952649_1165519_57877.py", line 102, in stringify
File "cheetah_DynamicallyCompiledCheetahTemplate_1705952649_1165519_57877.py", line 113, in stringify
AttributeError: 'TreeDict' object has no attribute 'is_collection'
```
bernt-matthias
force-pushed
the
topic/import-error
branch
from
8125f49
to
7ae4b68
Compare
|
ping @ebolyen |
|
@bernt-matthias, is this urgent on your end? @ebolyen is out till this Thursday, but I may be able to get ahold of him. Let me know. |
not so urgent :) no worries. Let me know if I can help :) |
|
Thanks @bernt-matthias! |
bernt-matthias
force-pushed
the
topic/import-error
branch
from
1b3249e
to
9a15f2b
Compare
|
Ok. I am satisfied that this works and am now merging it as a stop gap until the next q2galaxy release makes more permanent changes. Thank you @bernt-matthias and @mvdbeek |
|
I think this was not deployed. The test toolshed is currently not working. Probably you need to https://github.com/galaxyproject/tools-iuc/blob/0da1ca313d02ca780a671247e434a424fa67887b/.github/workflows/pr.yaml#L392 For this PR its probably the easiest to push manually to the toolshed. |
|
@bernt-matthias I made that change, and it got us past the previous block, but there is now an issue with templating out one of the tools that I can't properly track down. I know where it's coming from, but I don't know what's causing it to happen (or honestly it's more like I don't know why it hasn't always been a problem). I'll most likely need to consult with @ebolyen about it when he comes in on Thursday since he knows the most about this code-base. I apologize for the delay. |
|
If you point me to the error I might have a look. I guess it would be sufficient to create another PR (introducing a mock change in the tool) and merge it. Then it should be deployed. |
|
@bernt-matthias The issue we're seeing now doesn't seem to have anything to do with Galaxy itself it appears to be purely a q2galaxy thing to do with how the validator for the input for the
Now it is checking if the provided type is in the list:
Causing an artifact of the type The bottom one is more accurate to how the type expression is written inside of RE: mock changing the tool, that's just the thing the existing tool in this repo works (though I have no idea how it was templated in the way it is) it's the one that's being re-rendered in this CI action that fails, so we are unable to re-render the tools. CI has gotten past the action it was failing on yesterday. I'm not sure if you need all the tools to re-render to be able to do what you need to do to deploy the import you manually modified. I suspect not now that I think about it. |
I think it would be sufficient to push the latest version of the import tool to the Galaxy toolshed. Maybe you can do this manually? |
|
@bernt-matthias I believe it should be there now via this |
The following Traceback can be found in the logs (the test itself is not executed)
I also tried to understand what the config file is doing. If I get it right it generates a json (where some elements are stripped).
Did you know that a json file containing the inputs can also be generated with:
see https://docs.galaxyproject.org/en/master/dev/schema.html#tool-configfiles-inputs. Guess stripping elements could be done with a little helper script that could run in the tool (before
q2galaxy).Seems that this fails with 23.1 but worked in 23.0. I will try to bisect this, but it's unlikely that this is a Galaxy bug ... the configfile just seems to access some Galaxy internals that may be subject to change.
I also would be a huge fan of having a few tests for the fastq import here.
In order to get the test running I needed to add
collection_type="list"to the collection parameters. Might also be a good thing to add.