Please use set -e (ideally set -euo pipefail) in smaltalign*.sh shell scripts to make them fail early & loudly

[Masterxilo]

Ideally, prefix the shell script with sth like

#!/bin/bash
set -euo pipefail
IFS=$'\n\t'
trap 'echo FATAL ERROR AT $0:$LINENO' ERR

see http://redsymbol.net/articles/unofficial-bash-strict-mode/

[Masterxilo]

I just had a very silent failure which is probably a dependency problem, but smaltalign.sh crashed way later than the original error vcf2fasta: error while loading shared libraries: libtabixpp.so.0: cannot open shared object file: No such file or directory happened:

# =-=-= End of Hash Index Stats =-=-=
# Processing query reads ...
# Processed 27990 single reads.
# smalt: Time spent setting up hash index:0 seconds
# smalt: Time spent mapping reads:7 seconds
# smalt: Total elapsed wall clock time: 7 seconds
vcf2fasta: error while loading shared libraries: libtabixpp.so.0: cannot open shared object file: No such file or directory
mv: cannot stat 'merged_1_unknown*': No such file or directory
=-=-= lofreq with 1 cores =-=-=
INFO [2023-08-11 19:17:10,908]: Using 1 threads with following basic args: lofreq call -f /output/reference.fasta merged_1.bam

INFO [2023-08-11 19:17:10,914]: Adding 3 commands to mp-pool
Number of substitution tests performed: 12315
Number of indel tests performed: 0
INFO [2023-08-11 19:17:18,394]: Executing lofreq filter -i /tmp/lofreq2_call_parallelxouez8hm/concat.vcf.gz -o merged_1_lofreq.vcf --snvqual-thresh 61 --indelqual-thresh 20

INFO [2023-08-11 19:17:18,980]: Using 1 threads with following basic args: lofreq call --call-indels -f /output/reference.fasta merged_1_indel.bam

INFO [2023-08-11 19:17:18,985]: Adding 3 commands to mp-pool
Number of substitution tests performed: 12288
Number of indel tests performed: 313
INFO [2023-08-11 19:17:30,375]: Executing lofreq filter -i /tmp/lofreq2_call_parallel04jmgyhr/concat.vcf.gz -o merged_1_lofreq_indel.vcf --snvqual-thresh 61 --indelqual-thresh 45
...
    count

Error in file(con, "r") : cannot open the connection
Calls: %>% ... data.frame -> unlist -> strsplit -> readLines -> file
In addition: Warning message:
In file(con, "r") :
  cannot open file '/output/merged_1_cons.fasta': No such file or directory
Execution halted

[MaryamZaheri]

Thank you for sharing, could you please send me the dataset that causes the error by email?

[Masterxilo]

Sorry I don't have that data set at hand right now, but we will keep observing and keep you posted.

But in general, I think it would make sense for the script/pipeline to exit as soon as possible when anything expected happened.

Today I did another run, on a fastq file with samples of the env region (I think... and I had only an R1 but no R2 file, so no paired reads...) and everything seemed to go fine, all scripts reported exit code 0, but no merged_lofreq_indel_hq.vcf file was produced...