segmentation fault at ivar trim step

[uphl-biongs]

segmentation fault at ivar trim step

[vangalamaheshh]

I used 'samtools' for trimming and I got past the trim step. However, pipeline now fails at ivar variants step.

• Mahesh

[erinyoung]

Thank you for including the log file! It's also helpful to include the nextflow error.

The ivar bedfile warning can be ignored, iVar trim works fine on this bedfile that I stole from artic. That's not the source of the error.

There's a known "complication" with samtools mpileup in that it will sometimes exceed the available memory.

There are some ways around this that I'm working on implementing for the next iteration of the workflow.

• Just attempt to -resume workflow. This can sometimes work.
• In the config file, change params.mpileup_depth to something smaller. For a consensus file, I imagine params.mpileup_depth=200 is probably sufficient. Default is set to 8000, which is overkill.
• Adjust the memory setting for the process in your config file to include something like :

process {
   withName:ivar_consensus{
       cpus = 2
       memory {2.GB * task.attempt}
       errorStrategy = 'retry'
       maxRetries = 3
    }
 }

[vangalamaheshh]

Hi @erinyoung -

Thank you very much for debugging this with me.
I ran the pipeline afresh after updating the config as you suggested:
The pipeline fails at the ivar step, here's the full log from Nextflow ....

executor > local (5)
[76/75ad53] process > seqyclean (SRR13786672_1.fa... [100%] 1 of 1 ✔
[- ] process > fastp -
[05/70f2cf] process > bwa (SRR13786672_1.fastq.gz) [100%] 1 of 1 ✔
[- ] process > minimap2 -
[88/6ae3e1] process > fastqc (SRR13786672_1.fastq... [100%] 1 of 1 ✔
[69/68efa4] process > sort (SRR13786672_1.fastq.gz) [100%] 1 of 1 ✔
[d0/fe3b00] process > ivar_trim (SRR13786672_1.fa... [100%] 1 of 1, failed: 1 ✘
[- ] process > samtools_trim -
[- ] process > ivar_variants -
[- ] process > ivar_consensus -
[- ] process > bcftools_variants -
[- ] process > bamsnap -
[- ] process > samtools_stats -
[- ] process > samtools_coverage -
[- ] process > samtools_flagstat -
[- ] process > kraken2 -
[- ] process > bedtools -
[- ] process > samtools_ampliconstats -
[- ] process > pangolin -
[- ] process > nextclade -
[- ] process > summary -
[- ] process > combined_summary -
[- ] process > mafft -
[- ] process > snpdists -
[- ] process > iqtree -
[- ] process > rename -
[- ] process > combine_fastas -

Pulling Singularity image docker://staphb/ivar:latest [cache /project/umw_paul_langlois/dst/devel/umv/projects/sars-cov2/work/singularity/staphb-ivar-latest.img]
Error executing process > 'ivar_trim (SRR13786672_1.fastq.gz)'

Caused by:
Process ivar_trim (SRR13786672_1.fastq.gz) terminated with an error exit status (127)

Command executed:

mkdir -p ivar_trim logs/ivar_trim
log_file=logs/ivar_trim/SRR13786672_1.fastq.gz.db5be1c4-6748-4acf-bbca-b343fd3e9a98.log
err_file=logs/ivar_trim/SRR13786672_1.fastq.gz.db5be1c4-6748-4acf-bbca-b343fd3e9a98.err

time stamp + capturing tool versions

date | tee -a $log_file $err_file > /dev/null
ivar version >> $log_file

trimming the reads

ivar trim -e -i SRR13786672_1.fastq.gz.sorted.bam -b artic_V3_nCoV-2019.bed -p ivar_trim/SRR13786672_1.fastq.gz.primertrim 2>> $err_file >> $log_file

sorting and indexing the trimmed bams

samtools sort ivar_trim/SRR13786672_1.fastq.gz.primertrim.bam -o ivar_trim/SRR13786672_1.fastq.gz.primertrim.sorted.bam 2>> $err_file >> $log_file
samtools index ivar_trim/SRR13786672_1.fastq.gz.primertrim.sorted.bam 2>> $err_file >> $log_file

Command exit status:
127

Command output:
(empty)

Command error:
WARNING: underlay of /etc/localtime required more than 50 (97) bind mounts
FATAL: kernel too old

Work dir:
/project/umw_paul_langlois/dst/devel/umv/projects/sars-cov2/work/d0/fe3b00d7c13e729c53a08dc252315d

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out
Pulling Singularity image docker://staphb/parallel-perl:latest [cache /project/umw_paul_langlois/dst/devel/umv/projects/sars-cov2/work/singularity/staphb-parallel-perl-latest.img]

[vangalamaheshh]

@erinyoung -

Also, if this helps ...

cat SRR13786672_1.fastq.gz.d6dcaab8-0db4-49b7-aeb8-a2f6d8fe6497.log
Tue Mar 23 15:36:05 EDT 2021
samtools 1.12
Using htslib 1.12
Copyright (C) 2021 Genome Research Ltd.

Samtools compilation details:
Features: build=configure curses=yes
CC: gcc
CPPFLAGS:
CFLAGS: -Wall -g -O2
LDFLAGS:
HTSDIR: htslib-1.12
LIBS:
CURSES_LIB: -lncurses

HTSlib compilation details:
Features: build=configure plugins=no libcurl=yes S3=yes GCS=yes libdeflate=no lzma=yes bzip2=yes htscodecs=1.0
CC: gcc
CPPFLAGS:
CFLAGS: -Wall -g -O2 -fvisibility=hidden
LDFLAGS: -fvisibility=hidden

HTSlib URL scheme handlers present:
built-in: preload, data, file
S3 Multipart Upload: s3w, s3w+https, s3w+http
Amazon S3: s3+https, s3+http, s3
Google Cloud Storage: gs+http, gs+https, gs
libcurl: imaps, pop3, http, smb, gopher, ftps, imap, smtp, smtps, rtsp, ftp, telnet, rtmp, ldap, https, ldaps, tftp, pop3s, smbs, dict
crypt4gh-needed: crypt4gh
mem: mem

For some reason, samtools sort command is failing at ivar_trim step?
Let me know.

Thank you,
Mahesh

[erinyoung]

Woah, that's a new error.

This is what is causing the workflow to fail:

WARNING: underlay of /etc/localtime required more than 50 (97) bind mounts
FATAL: kernel too old

From what I read here (https://hpcc.umd.edu/hpcc/help/software/singularity.html) :

Kernel too old errors
Singularity containers can allow one to run binaries created for other Linux distributions and/or versions, to a point. Although one can replace most of the libraries, etc. that the binary sees, one cannot change the kernel that is running. So no matter which latest and greatest linux distribution your container was built with, it will still be using the kernel of the operating system running on the host node.

What OS are you using?

What command are you using? Did you try running this with docker instead?

nextflow run Cecret.nf -c configs/docker.config

[erinyoung]

nextflow run Cecret.nf -c configs/docker.config

Uses docker (so you'd need to have docker installed) instead of singularity. Perhaps this will help?