Error: An unexpected error occurred while trying to open file snpEff.jar

[NorikazuHARA]

Operating System

CentOS 7

Other Linux

No response

Workflow Version

v2.3.1-g6a6daa6

Workflow Execution

Command line (Local)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-human-variation
--bam 'wf-human-variation-demo/demo.bam'
--ref 'wf-human-variation-demo/demo.fasta'
--bed 'wf-human-variation-demo/demo.bed'
--sample_name 'DEMO'
--snp
--sv
--mod
--phased
-profile standard
.nextflow.log

Workflow Execution - CLI Execution Profile

None

What happened?

I ran the wf-human-variation workflow for the first time on the demo data on my local server.
During annotation with snpEff, I encountered an error:

"Error: An unexpected error occurred while trying to open file /opt/custflow/epi2meuser/conda/share/snpeff-5.1-2/snpEff.jar"

Relevant log output

Sep-06 14:19:24.595 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-human-variation --bam wf-human-variation-demo/demo.bam --ref wf-human-variation-demo/demo.fasta --bed wf-human-variation-demo/demo.bed --sample_name DEMO --snp --sv --mod --phased -profile standard
Sep-06 14:19:24.678 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W  ~  version 24.04.4
Sep-06 14:19:24.705 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/hara/.nextflow/plugins; core-plugins: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]
Sep-06 14:19:24.719 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Sep-06 14:19:24.720 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Sep-06 14:19:24.724 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.12.0 in 'deployment' mode
Sep-06 14:19:24.739 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Sep-06 14:19:24.756 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/hara/.nextflow/scm
Sep-06 14:19:25.485 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-human-variation.git
Sep-06 14:19:25.547 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Sep-06 14:19:25.560 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-human-variation.git
Sep-06 14:19:26.394 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config
Sep-06 14:19:26.396 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config
Sep-06 14:19:26.404 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/hara/.nextflow/secrets/store.json
Sep-06 14:19:26.407 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@477021ee] - activable => nextflow.secret.LocalSecretsProvider@477021ee
Sep-06 14:19:26.418 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Sep-06 14:19:26.799 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Sep-06 14:19:26.816 [main] DEBUG nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-human-variation` [backstabbing_bell] DSL2 - revision: 6a6daa65aa [master]
Sep-06 14:19:26.818 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Sep-06 14:19:26.818 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Sep-06 14:19:26.903 [main] DEBUG nextflow.Session - Session UUID: 840bf982-026e-4c74-8c1e-964a364c6c51
Sep-06 14:19:26.904 [main] DEBUG nextflow.Session - Run name: backstabbing_bell
Sep-06 14:19:26.904 [main] DEBUG nextflow.Session - Executor pool size: 96
Sep-06 14:19:26.914 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Sep-06 14:19:26.920 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=288; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Sep-06 14:19:26.942 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 24.04.4 build 5917
  Created: 01-08-2024 07:05 UTC (16:05 JDT)
  System: Linux 3.10.0-957.27.2.el7.x86_64
  Runtime: Groovy 4.0.21 on OpenJDK 64-Bit Server VM 11.0.13+7-b1751.21
  Encoding: UTF-8 (ANSI_X3.4-1968)
  Process: 285209@DPeR840 [192.168.12.110]
  CPUs: 96 - Mem: 376.2 GB (60 GB) - Swap: 32 GB (32 GB)
Sep-06 14:19:26.962 [main] DEBUG nextflow.Session - Work-dir: /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work [nfs]
Sep-06 14:19:27.032 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Sep-06 14:19:27.063 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Sep-06 14:19:27.180 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Sep-06 14:19:27.200 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 97; maxThreads: 1000
Sep-06 14:19:27.349 [main] DEBUG nextflow.Session - Session start
Sep-06 14:19:27.353 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/[:]/execution/trace.txt
Sep-06 14:19:27.377 [main] DEBUG nextflow.Session - Using default localLib path: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/lib
Sep-06 14:19:27.381 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/lib
Sep-06 14:19:27.382 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/lib/nfcore_external_java_deps.jar
Sep-06 14:19:29.749 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Sep-06 14:19:39.906 [main] INFO  nextflow.Nextflow - 
�[0;92m||||||||||   �[0m�[2m_____ ____ ___ ____  __  __ _____      _       _
�[0;92m||||||||||  �[0m�[2m| ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
�[0;33m|||||       �[0m�[2m|  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
�[0;33m|||||       �[0m�[2m| |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
�[0;94m||||||||||  �[0m�[2m|_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
�[0;94m||||||||||  �[0m�[1mwf-human-variation v2.3.1-g6a6daa6�[0m
�[2m--------------------------------------------------------------------------------�[0m
�[1mCore Nextflow options�[0m
  �[0;34mrevision       : �[0;32mmaster�[0m
  �[0;34mrunName        : �[0;32mbackstabbing_bell�[0m
  �[0;34mcontainerEngine: �[0;32mdocker�[0m
  �[0;34mcontainer      : �[0;32m[withLabel:wf_human_snp:ontresearch/wf-human-variation-snp:sha17e686336bf6305f9c90b36bc52ff9dd1fa73ee9, withLabel:wf_human_sv:ontresearch/wf-human-variation-sv:shac591518dd32ecc3936666c95ff08f6d7474e9728, withLabel:wf_human_mod:ontresearch/modkit:shae137452eb41f5f12b790774cafe15bc97f48d4d0, withLabel:wf_cnv:ontresearch/wf-cnv:sha428cb19e51370020ccf29ec2af4eead44c6a17c2, withLabel:wf_human_str:ontresearch/wf-human-variation-str:shadd2f2963fe39351d4e0d6fa3ca54e1064c6ec057, withLabel:snpeff_annotation:ontresearch/snpeff:shadcc812849019640d4d2939703fbb8777256e41ad, withLabel:wf_common:ontresearch/wf-common:shad399cf22079b5b153920ac39ee40095a677933f1, withLabel:spectre:ontresearch/spectre:sha49a9fe474da9860f84f08f17f137b47a010b1834, default:ontresearch/wf-human-variation:sha2b856c1f358ddf1576217a336bc0e9864b6dc0ed]�[0m
  �[0;34mlaunchDir      : �[0;32m/disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp�[0m
  �[0;34mworkDir        : �[0;32m/disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work�[0m
  �[0;34mprojectDir     : �[0;32m/home/hara/.nextflow/assets/epi2me-labs/wf-human-variation�[0m
  �[0;34muserName       : �[0;32mhara�[0m
  �[0;34mprofile        : �[0;32mstandard�[0m
  �[0;34mconfigFiles    : �[0;32m/home/hara/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config�[0m

�[1mWorkflow Options�[0m
  �[0;34msv             : �[0;32mtrue�[0m
  �[0;34msnp            : �[0;32mtrue�[0m
  �[0;34mmod            : �[0;32mtrue�[0m

�[1mMain options�[0m
  �[0;34msample_name    : �[0;32mDEMO�[0m
  �[0;34mbam            : �[0;32mwf-human-variation-demo/demo.bam�[0m
  �[0;34mref            : �[0;32mwf-human-variation-demo/demo.fasta�[0m
  �[0;34mbed            : �[0;32mwf-human-variation-demo/demo.bed�[0m
  �[0;34mphased         : �[0;32mtrue�[0m

!! Only displaying parameters that differ from the pipeline defaults !!
�[2m--------------------------------------------------------------------------------�[0m
If you use epi2me-labs/wf-human-variation for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x


�[2m--------------------------------------------------------------------------------�[0m
This is epi2me-labs/wf-human-variation v2.3.1-g6a6daa6.
�[2m--------------------------------------------------------------------------------�[0m
Sep-06 14:19:40.054 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `wf_common` for process with name prepare_reference:cram_cache
Sep-06 14:19:40.068 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Sep-06 14:19:40.068 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Sep-06 14:19:40.075 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Sep-06 14:19:40.082 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=96; memory=376.2 GB; capacity=96; pollInterval=100ms; dumpInterval=5m
Sep-06 14:19:40.085 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)

~~~~~
~~~~~


Sep-06 14:22:49.503 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Sep-06 14:22:49.503 [Task submitter] INFO  nextflow.Session - [94/0a2970] Submitted process > refine_with_sv (1)
Sep-06 14:22:49.508 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Sep-06 14:22:49.509 [Task submitter] INFO  nextflow.Session - [86/eea2ff] Submitted process > sv:annotate_sv_vcf (1)
Sep-06 14:22:50.538 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 121; name: sv:annotate_sv_vcf (1); status: COMPLETED; exit: 1; error: -; workDir: /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work/86/eea2ffaf582140b7daf89a6933b9f0]
Sep-06 14:22:50.542 [TaskFinalizer-8] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=sv:annotate_sv_vcf (1); work-dir=/disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work/86/eea2ffaf582140b7daf89a6933b9f0
  error [nextflow.exception.ProcessFailedException]: Process `sv:annotate_sv_vcf (1)` terminated with an error exit status (1)
Sep-06 14:22:50.564 [TaskFinalizer-8] ERROR nextflow.processor.TaskProcessor - Error executing process > 'sv:annotate_sv_vcf (1)'

Caused by:
  Process `sv:annotate_sv_vcf (1)` terminated with an error exit status (1)


Command executed:

  if [ "*" == '*' ]; then
      # SV is quick to annotate, dont bother breaking it apart
      INPUT_FILENAME=input.vcf.gz
      OUTPUT_LABEL="sv"
  else
      # SNP is slow to annotate, we'll break it apart by contig
      # and merge it back later. filter the master VCF to current contig
      # Also, relabel chrM as chrMT so SnpEff can annotate
      bcftools view -r * input.vcf.gz | sed 's/^chrM\t/chrMT\t/' | bgzip > input.chr.vcf.gz
  
      INPUT_FILENAME=input.chr.vcf.gz
      OUTPUT_LABEL="sv.*"
  fi
  
  # deal with samples which aren't hg19 or hg38
  if [[ "hg38" != "hg38" ]] && [[ "hg38" != "hg19" ]]; then
      # return the original VCF and index as the outputs
      cp ${INPUT_FILENAME} DEMO.wf_${OUTPUT_LABEL}.vcf.gz
      cp ${INPUT_FILENAME}.tbi DEMO.wf_${OUTPUT_LABEL}.vcf.gz.tbi
  else
      # do some annotation
      if [[ "hg38" == "hg38" ]]; then
          snpeff_db="GRCh38.p13"
          clinvar_vcf="${CLINVAR_PATH}/clinvar_GRCh38.vcf.gz"
  
      elif [[ "hg38" == "hg19" ]]; then
          snpeff_db="GRCh37.p13"
          clinvar_vcf="${CLINVAR_PATH}/clinvar_GRCh37.vcf.gz"
      fi
  
      # Revert back chrMT to chrM
      snpEff -Xmx5g ann -noStats -noLog $snpeff_db ${INPUT_FILENAME} | sed 's/^chrMT\t/chrM\t/' > DEMO.intermediate.snpeff_annotated.vcf
  
      # Add ClinVar annotations
      SnpSift annotate $clinvar_vcf DEMO.intermediate.snpeff_annotated.vcf | bgzip > DEMO.wf_${OUTPUT_LABEL}.vcf.gz
      tabix DEMO.wf_${OUTPUT_LABEL}.vcf.gz
  
      # tidy up
      rm DEMO.intermediate*
  fi

Command exit status:
  1

Command output:
  (empty)

Command error:
  Picked up JAVA_TOOL_OPTIONS: -Xlog:disable -Xlog:all=warning:stderr
  Error: An unexpected error occurred while trying to open file /opt/custflow/epi2meuser/conda/share/snpeff-5.1-2/snpEff.jar

Work dir:
  /disk5/dna/bySample/nanopore/210010977/wf-human-variation/tmp/work/86/eea2ffaf582140b7daf89a6933b9f0

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
Sep-06 14:22:50.569 [TaskFinalizer-8] DEBUG nextflow.Session - Session aborted -- Cause: Process `sv:annotate_sv_vcf (1)` terminated with an error exit status (1)
Sep-06 14:22:50.601 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Sep-06 14:22:50.607 [TaskFinalizer-8] DEBUG nextflow.Session - The following nodes are still active:
[process] concat_refined_snp
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: output_label
  port 2: (cntrl) -     ; channel: $

[process] bed_filter
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (queue) OPEN  ; channel: bed
  port 2: (value) bound ; channel: subworkflow
  port 3: (value) bound ; channel: file_type
  port 4: (cntrl) -     ; channel: $

[process] annotate_snp_vcf
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: genome
  port 2: (value) bound ; channel: output_label
  port 3: (cntrl) -     ; channel: $

[process] concat_snp_vcfs
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: output_label
  port 2: (cntrl) -     ; channel: $

[process] sift_clinvar_snp_vcf
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (queue) OPEN  ; channel: genome
  port 2: (value) bound ; channel: output_label
  port 3: (cntrl) -     ; channel: $

[process] vcfStats
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

[process] report_snp:makeReport
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (value) bound ; channel: versions
  port 2: (value) bound ; channel: params.json
  port 3: (queue) OPEN  ; channel: clinvar_vcf
  port 4: (cntrl) -     ; channel: $

[process] output_snp
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: fname
  port 1: (cntrl) -     ; channel: $

[process] combine_metrics_json
  status=ACTIVE
  port 0: (value) OPEN  ; channel: jsons
  port 1: (queue) OPEN  ; channel: flagstat.tsv
  port 2: (queue) OPEN  ; channel: hists
  port 3: (queue) OPEN  ; channel: -
  port 4: (queue) OPEN  ; channel: mosdepth.summary.txt
  port 5: (queue) OPEN  ; channel: haplocheck
  port 6: (queue) OPEN  ; channel: sex
  port 7: (cntrl) -     ; channel: $

[process] publish_artifact
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: fname
  port 1: (cntrl) -     ; channel: $

Sep-06 14:22:51.475 [main] DEBUG nextflow.Session - Session await > all processes finished
Sep-06 14:22:51.475 [main] DEBUG nextflow.Session - Session await > all barriers passed
Sep-06 14:22:51.806 [main] WARN  n.processor.TaskPollingMonitor - Killing running tasks (3)
Sep-06 14:22:51.819 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill refine_with_sv (3) -- command: kill -TERM 372743; exit: 1 
 bash: line 0: kill: (372743) - No such process

Sep-06 14:22:51.824 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill refine_with_sv (2) -- command: kill -TERM 372750; exit: 1 
 bash: line 0: kill: (372750) - No such process

Sep-06 14:22:51.827 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill refine_with_sv (1) -- command: kill -TERM 372762; exit: 1 
 bash: line 0: kill: (372762) - No such process

Sep-06 14:22:51.829 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=117; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=1; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=3; succeedDuration=34m 59s; failedDuration=962ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=51; peakCpus=54; peakMemory=216 GB; ]
Sep-06 14:22:51.830 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Sep-06 14:22:51.831 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Sep-06 14:22:52.825 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Sep-06 14:22:52.922 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Sep-06 14:22:52.952 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

no

Other demo data information

No response

[vlshesketh]

Hi @NorikazuHARA, apologies with the delay in getting back to you about this. I'm afraid I've been unable to reproduce this error locally - the workflow has been updated to v2.4.1, please can you try again with the latest version by doing:

nextflow pull epi2me-labs/wf-human-variation

and let us know if you're still getting the same error?

[NorikazuHARA]

@vlshesketh Thanks for your reply.

I updated the workflow to v2.4.1 and tried with it.
But I got the same error during annotation with snpEff.

The other tasks seem to work fine.
So I'll try to annotate vcf files manually instead.

Anyway, thanks a lot!