Error in rule PRIMARY_NT_reformat

[shandley]

Error in rule PRIMARY_NT_reformat

[mihinduk]

I ran 134 samples and did NOT generate this error.

[beardymcjohnface]

Hi Leran,
This looks like an MMSeqs issue. Do you have a scheduler log for this? If you used my example profile, the scheduler log should be in logs/PRIMARY_NT_reformat/

[Leran10]

Hi Michael,

Here is the message from the .err file:

[Fri Oct 29 00:26:25 2021]
rule PRIMARY_NT_reformat:
    input: hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/queryDB, /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/nt/virus_primary_nt/sequenceDB, /opt/apps/labs/sahlab
/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/tax/taxonomy
    output: hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/firsthit.index, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.lineage, hecatomb_out
/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.tsv
    log: hecatomb_out/STDERR/t09.mmseqs_PRIMARY_nt_summary.log
    jobid: 0
    benchmark: hecatomb_out/BENCHMARKS/t09.pNT_stat.txt
    threads: 2
    resources: mem_mb=16000, time=1440, jobs=100


        { # Filter TopHit results
        mmseqs filterdb hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/firsthit --extract-lines 1;
        # Convert to alignments
        mmseqs convertalis hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/queryDB /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/nt/virus_primary_nt/sequenceDB hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/firsthit hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8;
        # Assign taxonomy
        cut -f2 hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8 |             awk -F '|' '{ print$2 }' |             taxonkit lineage --data-dir /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/tax/taxonomy > hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.lineage;
        # Reformat TopHit viral lineage information
        taxonkit reformat --data-dir /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/tax/taxonomy hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.lineage -i 2             -f "{k}        {p}     {c}     {o}     {f}     {g}     {s}" -F --fill-miss-rank |
            cut --complement -f2 > hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.tsv;
        } &> hecatomb_out/STDERR/t09.mmseqs_PRIMARY_nt_summary.log
        
Activating conda environment: /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/conda/50d9b1df3388aa515e6b7d65d03fdcd8
[Fri Oct 29 00:26:26 2021]
Error in rule PRIMARY_NT_reformat:
    jobid: 0
    output: hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/firsthit.index, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.lineage, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.tsv
    log: hecatomb_out/STDERR/t09.mmseqs_PRIMARY_nt_summary.log (check log file(s) for error message)
    conda-env: /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/conda/50d9b1df3388aa515e6b7d65d03fdcd8
    shell:
        
        { # Filter TopHit results
        mmseqs filterdb hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/firsthit --extract-lines 1;
        # Convert to alignments
        mmseqs convertalis hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/queryDB /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/nt/virus_primary_nt/sequenceDB hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/firsthit hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8;
        # Assign taxonomy
        cut -f2 hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8 |             awk -F '|' '{ print$2 }' |             taxonkit lineage --data-dir /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/tax/taxonomy > hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.lineage;
        # Reformat TopHit viral lineage information
        taxonkit reformat --data-dir /opt/apps/labs/sahlab/software/miniconda3/envs/hecatomb2/snakemake/workflow/../../databases/tax/taxonomy hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.lineage -i 2             -f "{k}        {p}     {c}     {o}     {f}     {g}     {s}" -F --fill-miss-rank |
            cut --complement -f2 > hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/primary_nt.tsv;
        } &> hecatomb_out/STDERR/t09.mmseqs_PRIMARY_nt_summary.log
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Job counts:
        count   jobs
        1       PRIMARY_NT_reformat
        1

But I reran the hecatomb yesterday, it seems this step succeeded this time:

[Fri Oct 29 10:31:37 2021]
rule PRIMARY_NT_parsing:
    input: hecatomb_out/RESULTS/MMSEQS_AA_PRIMARY/MMSEQS_AA_PRIMARY_unclassified.fasta, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/results/result.m8
    output: hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/classified_seqs.fasta, hecatomb_out/RESULTS/MMSEQS_NT_PRIMARY/unclassified_seqs.fasta
    log: hecatomb_out/STDERR/t10.pNT_parse.log
    jobid: 1543
    benchmark: hecatomb_out/BENCHMARKS/t10.pNT_parse.txt
    threads: 2
    resources: mem_mb=16000, time=1440, jobs=100

Submitted job 1543 with external jobid 'Submitted batch job 33425586'.
[Fri Oct 29 10:33:10 2021]
Finished job 1543.
2 of 19 steps (11%) done

And now this job failed at the sankey_diagram step:

[Fri Oct 29 15:05:15 2021]
Error in rule sankey_diagram:
    jobid: 1573
    output: hecatomb_report/Sankey.svg
    log: hecatomb_out/STDERR/sankey_diagram.log (check log file(s) for error message)
    cluster_jobid: Submitted batch job 33429883

Error executing rule sankey_diagram on cluster (jobid: 1573, external: Submitted batch job 33429883, jobscript: /scratch/sahlab/SIV_vaccine_hecatomb/.snakemake/tmp.7t_gxd0s/snakejob.sankey_diagram.1573.sh). For error details see the cluster log and the log files of the involved rule(s).
Trying to restart job 1573.

[beardymcjohnface]

The sankey issue I believe is resolved with the new bioconda builds. We hadn't specified the required dependencies.

Feel free to reopen if you run into more MMSeqs issues.