Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nextflow updates causing errors in execution #978

Open
anoronh4 opened this issue Dec 20, 2022 · 3 comments
Open

nextflow updates causing errors in execution #978

anoronh4 opened this issue Dec 20, 2022 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@anoronh4
Copy link
Collaborator

  1. trace file rolling is now disabled in newer versions of nextflow: Custom trace file rotation/renaming is broken in 22.10.0 nextflow-io/nextflow#3317. With our current configuration trace { file = 'trace.txt' }, there are errors with resume because nextflow is attempting to write to the same location without rolling the previous file (unless we specify overwrite = true). Similar applies to execution report and execution timeline. We should change how we output trace files and possibly adjust tempodeliver to expect trace files with different names.
  2. nextflow sometimes ignores config directives written with the dot notation (e.g. process.errorStrategies rather than process {}), documented at process directive not always applying to run nextflow-io/nextflow#3154. The developers said they plan to address it but i am still finding the issue in a recent release (22.10.3). We should put dot notation configs into braces {} to avoid issues going forward.
@anoronh4 anoronh4 self-assigned this Dec 20, 2022
@anoronh4 anoronh4 added the bug Something isn't working label Dec 20, 2022
@anoronh4 anoronh4 added this to the 2.0 milestone Dec 20, 2022
@gongyixiao gongyixiao removed this from the 2.0 milestone Mar 24, 2023
@gongyixiao
Copy link
Collaborator

Both issues were not present in nextflow 21.10.6 or earlier.

@anoronh4
Copy link
Collaborator Author

anoronh4 commented Jun 15, 2023

for fixing how the trace file is written, i wonder if it would make sense to use an implementation such as the following:

https://github.com/mskcc/forte/blob/b556deb986ac6df217c4f2b0c6536794c7a69d4f/nextflow.config#L50-L51

https://github.com/mskcc/forte/blob/b556deb986ac6df217c4f2b0c6536794c7a69d4f/nextflow.config#L167-L171

(not sure why the code preview isn't showing up) basically where all execution reports go to a named, unique folder each time the run initializes. that way we can save all trace files and have them be organized.

@gongyixiao
Copy link
Collaborator

I think this should work fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants