License: GNU General Public License v3.0
This is MetaPhage, a nextflow pipeline for automatic phage discovery. MetaPhage requires a Linux environment.
π MetaPhage documentation is available online
A new version with updated Miners is available in the dev branch:
This pipeline consists of several modules, as summarised in the workflow below.
Hi, I am trying to use the MetaPhage on my HPC but I have encountered some issues when I tried to create the environment.
*conda env create -n MetaPhage --file env.yaml
Channels:
- anaconda
- bioconda
- conda-forge
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- r-cli==3.0.1=r41hc72bb7e_0
Current channels:
- https://conda.anaconda.org/anaconda/linux-64
- https://conda.anaconda.org/bioconda/linux-64
- https://conda.anaconda.org/conda-forge/linux-64
- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/r/linux-64
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Hi,
I'm trying to use MetaPhage v2 beta.
When I tried running the program with the demo data, virsorter2 seemed to be throwing an error.
Below is the excerpt of the nextflow log.
-----
Mar-01 23:32:44.967 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'virsorter2 (megahit-SRR8653090)'
Caused by:
Process virsorter2 (megahit-SRR8653090) terminated with an error exit status (1)
Command executed:
# run virsorter2
virsorter run -j 8 -d /home/kangin/Programs/MetaPhage/db/virsorter/virsorter2/ --min-length 0 -i SRR8653090_megahit_contigs.fasta --include-groups dsDNAphage,ssDNA -w SRR8653090_virsorter2 -l SRR8653090 --rm-tmpdir
# add miner flag at each fasta header
sed 's/^>/>virsorter2_/' SRR8653090_virsorter2/SRR8653090-final-viral-combined.fa > SRR8653090_virsorter2/SRR8653090-tmp-final-viral-combined.fa
sed 's/||/ /' SRR8653090_virsorter2/SRR8653090-tmp-final-viral-combined.fa > SRR8653090_virsorter2/SRR8653090-correct-final-viral-combined.fa
rm SRR8653090_virsorter2/SRR8653090-tmp-final-viral-combined.fa
Command exit status:
1
Command output:
(empty)
Command error:
2 merge_hmm_gff_features_by_group
2 merge_provirus_call_by_group_by_split
1 merge_provirus_call_from_groups
5 merge_split_hmmtbl
10 merge_split_hmmtbl_by_group
10 merge_split_hmmtbl_by_group_tmp
1 pick_viral_fullseq
1 preprocess
1 split_faa
2 split_faa_by_group
2 split_gff_by_group
61
[2023-03-01 18:05 INFO] # of seqs < 0 bp and removed: 0
[2023-03-01 18:05 INFO] # of circular seqs: 34
[2023-03-01 18:05 INFO] # of linear seqs : 13545
[2023-03-01 18:05 INFO] Finish spliting circular contig file with common rbs
[2023-03-01 18:05 INFO] Finish spliting linear contig file with common rbs
[2023-03-01 18:12 INFO] Step 1 - preprocess finished.
[2023-03-01 23:32 INFO] Step 2 - extract-feature finished.
[2023-03-01 23:32 ERROR] See error details in SRR8653090_virsorter2/log/iter-0/step3-classify/all-score-ssDNA.log
[Wed Mar 1 23:32:44 2023]
Error in rule classify_by_group:
jobid: 57
output: iter-0/ssDNA/all.pdg.clf
conda-env: /home/kangin/Programs/MetaPhage/db/virsorter/virsorter2/conda_envs/ca248a1a
shell:
Log=SRR8653090_virsorter2/log/iter-0/step3-classify/all-score-ssDNA.log
python /home/kangin/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/classify.py iter-0/ssDNA/all.pdg.ftr /home/kangin/Programs/MetaPhage/db/virsorter/virsorter2/group/ssDNA/model ssDNA iter-0/ssDNA/all.pdg.clf 2> $Log || { echo "See error details in $Log" | python /home/kangin/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/echo.py --level error; exit 1; }
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
[2023-03-01 23:32 ERROR] See error details in SRR8653090_virsorter2/log/iter-0/step3-classify/all-score-dsDNAphage.log
[Wed Mar 1 23:32:44 2023]
Error in rule classify_by_group:
jobid: 56
output: iter-0/dsDNAphage/all.pdg.clf
conda-env: /home/kangin/Programs/MetaPhage/db/virsorter/virsorter2/conda_envs/ca248a1a
shell:
Log=SRR8653090_virsorter2/log/iter-0/step3-classify/all-score-dsDNAphage.log
python /home/kangin/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/classify.py iter-0/dsDNAphage/all.pdg.ftr /home/kangin/Programs/MetaPhage/db/virsorter/virsorter2/group/dsDNAphage/model dsDNAphage iter-0/dsDNAphage/all.pdg.clf 2> $Log || { echo "See error details in $Log" | python /home/kangin/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/echo.py --level error; exit 1; }
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Exiting because a job execution failed. Look above for error message
*** An error occurred. Detailed errors may not be printed for certain rules. Refer to the log file of the failed command for troubleshooting
Issues can be raised at: https://github.com/jiarong/VirSorter2/issues
Work dir:
/home/kangin/Programs/MetaPhage/work/ee/927252a8fccb827839641922502588`
-----
I checked the two log files indicated by the error message:
SRR8653090_virsorter2/log/iter-0/step3-classify/all-score-ssDNA.log
SRR8653090_virsorter2/log/iter-0/step3-classify/all-score-dsDNAphage.log
The last part of the two log files seem to indicate the same error.
-----
File "/home/kangin/Programs/MetaPhage/db/virsorter/virsorter2/conda_envs/ca248a1a/lib/python3.8/site-packages/numpy/init.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'float'.
np.float was a deprecated alias for the builtin float. To avoid this error in existing code, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here.
-----
Is there any way to avoid this error?
Thanks.
Hi, Mattia.
I'm running tutorial stable for metaphage and I found this error and I actually don't know how I can fix it:
Error executing process > 'summary (megahit)'
Caused by:
Process summary (megahit) terminated with an error exit status (1)
Command executed:
Rscript /media/omicas/data/apps/MetaPhage-0.3.3/bin/Rscript/summary_report.R count_table.csv megahit_vOTUs_consensus.fasta metadata.csv Infant_delivery_type /media/omicas/data/apps/MetaPhage-0.3.3/MetaPhage-output
Command exit status:
1
Command output:
(empty)
Command error:
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'as.data.frame': dim(X) must have a positive length
Calls: as.data.frame -> apply
EjecuciΓ³n interrumpida
Work dir:
/media/omicas/data/apps/MetaPhage-0.3.3/work/8e/d5fbff060d20179cf69e548d02a7d4
Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run.
In the other hand, I was trying to run the pipeline for tutorial beta, but I'm having an error related to numpy and that was the log file content:
Error in rule classify_by_group:
jobid: 57
output: iter-0/ssDNA/all.pdg.clf
conda-env: /media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/db/virsorter/virsorter2/conda_envs/f1f2e930
shell:
Log=/media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/work/1b/6c9dcbb09335b1d01611163e3b9fdb/SRR8653218_virsorter2/log/iter-0/step3-classify/all-score-ssDNA.log
python /home/omicas/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/classify.py iter-0/ssDNA/all.pdg.ftr /media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/db/virsorter/virsorter2/group/ssDNA/model ssDNA iter-0/ssDNA/all.pdg.clf 2> $Log || { echo "See error details in $Log" | python /home/omicas/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/echo.py --level error; exit 1; }
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
[2024-02-28 00:33 ERROR] See error details in /media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/work/1b/6c9dcbb09335b1d01611163e3b9fdb/SRR8653218_virsorter2/log/iter-0/step3-classify/all-score-dsDNAphage.log
[Wed Feb 28 00:33:53 2024]
Error in rule classify_by_group:
jobid: 56
output: iter-0/dsDNAphage/all.pdg.clf
conda-env: /media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/db/virsorter/virsorter2/conda_envs/f1f2e930
shell:
Log=/media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/work/1b/6c9dcbb09335b1d01611163e3b9fdb/SRR8653218_virsorter2/log/iter-0/step3-classify/all-score-dsDNAphage.log
python /home/omicas/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/classify.py iter-0/dsDNAphage/all.pdg.ftr /media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/db/virsorter/virsorter2/group/dsDNAphage/model dsDNAphage iter-0/dsDNAphage/all.pdg.clf 2> $Log || { echo "See error details in $Log" | python /home/omicas/miniconda3/envs/metaphage2/lib/python3.7/site-packages/virsorter/./scripts/echo.py --level error; exit 1; }
And the log file information is:File "/media/omicas/data/Datos_Hernan/MetaPhage/MetaPhage/db/virsorter/virsorter2/conda_envs/f1f2e930/lib/python3.8/site-packages/sklearn/linear_model/_least_angle.py", line 30, in
method='lar', copy_X=True, eps=np.finfo(np.float).eps,
File "/home/omicas/.local/lib/python3.8/site-packages/numpy/init.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'float'.
np.float was a deprecated alias for the builtin float. To avoid this error in existing code, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
I really don't know how could I fix this, but I'd like to use this version instead the stable.
Thanks for your help.
It appears the link to singularity image is down. Running:
wget -O $METAPHAGE_DIR/containers/metaphage.simg "https://s3.climb.ac.uk/ifrqmra-metaphage/v1.0/metaphage.simg"
generates the following error:
--2023-03-24 15:41:12-- https://s3.climb.ac.uk/ifrqmra-metaphage/v1.0/metaphage.simg
Resolving s3.climb.ac.uk (s3.climb.ac.uk)... 147.188.173.14
Connecting to s3.climb.ac.uk (s3.climb.ac.uk)|147.188.173.14|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-24 15:41:13 ERROR 404: Not Found.
Is there a newer link I can access?
Is it possible to reassess alpha and beta diversity without running entire MetaPhage pipeline?
Particularly I want to change alpha_var and beta_var variables.
Hello,
Thank you for this pipeline. I was trying to test the channel using your example dataset. I have followed the tutorial for the stable version as indicated in the documentation. I have installed using the Conda environment.
I am receiving an error (at the bowtie2_derep (megahit) step) while trying to run the example dataset on a cluster server using sbatch:
I hope that you can guide me to fix the issue.
A copy of the error (log):
Error executing process > 'bowtie2_derep (megahit)'
Caused by:
Process bowtie2_derep (megahit) terminated with an error exit status (1)
Command executed:
bowtie2-build --threads 16 megahit_vOTUs_consensus.fasta megahit_vOTUs_consensus.fasta_index
bowtie2 -p 16 -x megahit_vOTUs_consensus.fasta_index -1 SRR8652861_dephixed_R1.fastq.gz -2 SRR8652861_dephixed_R2.fastq.gz -S SRR8652861_megahit_vOTUs_consensus.fasta.sam
samtools view -@ 16 -S -b SRR8652861_megahit_vOTUs_consensus.fasta.sam > SRR8652861_megahit_vOTUs_consensus.fasta.bam
samtools sort -@ 16 SRR8652861_megahit_vOTUs_consensus.fasta.bam -o SRR8652861_megahit_vOTUs_consensus.fasta.sorted.bam
samtools index -@ 16 SRR8652861_megahit_vOTUs_consensus.fasta.sorted.bam
samtools flagstat -@ 16 SRR8652861_megahit_vOTUs_consensus.fasta.sorted.bam > mappingstats_SRR8652861_megahit_vOTUs_consensus.fasta.txt
qualimap bamqc -nt 16 -outdir qualimap_bamqc_SRR8652861_megahit_vOTUs_consensus.fasta.folder -bam SRR8652861_megahit_vOTUs_consensus.fasta.sorted.bam
Command exit status:
1
Command output:
color: 0
reverse: 1
Total time for backward call to driver() for mirror index: 00:00:04
Java memory size is set to 1200M
Launching application...
QualiMap v.2.2.2-dev
Built on 2019-11-11 14:05
Selected tool: bamqc
Available memory (Mb): 33
Max memory (Mb): 1258
Starting bam qc....
Loading sam header...
Loading locator...
Loading reference...
Number of windows: 400, effective number of windows: 2168
Chunk of reads size: 1000
Number of threads: 16
Processed 216 out of 2168 windows...
Processed 432 out of 2168 windows...
Processed 648 out of 2168 windows...
Processed 864 out of 2168 windows...
Processed 1080 out of 2168 windows...
Processed 1296 out of 2168 windows...
Processed 1512 out of 2168 windows...
Processed 1728 out of 2168 windows...
Processed 1944 out of 2168 windows...
Processed 2160 out of 2168 windows...
Total processed windows:2168
Number of reads: 6169922
Number of valid reads: 635378
Number of correct strand reads:0
Inside of regions...
Num mapped reads: 635378
Num mapped first of pair: 322271
Num mapped second of pair: 313107
Num singletons: 27810
Time taken to analyze reads: 15
Computing descriptors...
numberOfMappedBases: 78310008
referenceSize: 17623649
numberOfSequencedBases: 78296311
numberOfAs: 19666957
Computing per chromosome statistics...
Computing histograms...
Overall analysis time: 15
end of bam qc
Computing report...
Command error:
2824935 (91.57%) aligned concordantly 0 times
182943 (5.93%) aligned concordantly exactly 1 time
77083 (2.50%) aligned concordantly >1 times
----
2824935 pairs aligned concordantly 0 times; of these:
26438 (0.94%) aligned discordantly 1 time
----
2798497 pairs aligned 0 times concordantly or discordantly; of these:
5596994 mates make up the pairs; of these:
5534544 (98.88%) aligned 0 times
27263 (0.49%) aligned exactly 1 time
35187 (0.63%) aligned >1 times
10.30% overall alignment rate
[bam_sort_core] merging from 0 files and 16 in-memory blocks...
Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using 'localhost:10.0' as the value of the DISPLAY variable.
at java.desktop/sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
at java.desktop/sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:102)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/sun.awt.X11GraphicsEnvironment.(X11GraphicsEnvironment.java:61)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at java.desktop/java.awt.GraphicsEnvironment$LocalGE.createGE(GraphicsEnvironment.java:101)
at java.desktop/java.awt.GraphicsEnvironment$LocalGE.(GraphicsEnvironment.java:83)
at java.desktop/java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:129)
at java.desktop/sun.awt.X11.XToolkit.(XToolkit.java:231)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:588)
at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:583)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/java.awt.Toolkit.getDefaultToolkit(Toolkit.java:582)
at java.desktop/sun.swing.SwingUtilities2.getSystemMnemonicKeyMask(SwingUtilities2.java:2212)
at java.desktop/javax.swing.plaf.basic.BasicLookAndFeel.initComponentDefaults(BasicLookAndFeel.java:1096)
at java.desktop/javax.swing.plaf.metal.MetalLookAndFeel.initComponentDefaults(MetalLookAndFeel.java:440)
at java.desktop/javax.swing.plaf.basic.BasicLookAndFeel.getDefaults(BasicLookAndFeel.java:150)
at java.desktop/javax.swing.plaf.metal.MetalLookAndFeel.getDefaults(MetalLookAndFeel.java:1560)
at java.desktop/javax.swing.UIManager.setLookAndFeel(UIManager.java:587)
at java.desktop/javax.swing.UIManager.setLookAndFeel(UIManager.java:629)
at java.desktop/javax.swing.UIManager.initializeDefaultLAF(UIManager.java:1404)
at java.desktop/javax.swing.UIManager.initialize(UIManager.java:1517)
at java.desktop/javax.swing.UIManager.maybeInitialize(UIManager.java:1483)
at java.desktop/javax.swing.UIManager.getDefaults(UIManager.java:709)
at java.desktop/javax.swing.UIManager.getColor(UIManager.java:751)
at org.jfree.chart.JFreeChart.(JFreeChart.java:258)
at org.jfree.chart.ChartFactory.createXYLineChart(ChartFactory.java:1748)
at org.bioinfo.ngs.qc.qualimap.beans.BamQCChart.render(BamQCChart.java:192)
at org.bioinfo.ngs.qc.qualimap.beans.BamQCRegionReporter.computeChartsBuffers(BamQCRegionReporter.java:673)
at org.bioinfo.ngs.qc.qualimap.main.BamQcTool.execute(BamQcTool.java:263)
at org.bioinfo.ngs.qc.qualimap.main.NgsSmartTool.run(NgsSmartTool.java:190)
at org.bioinfo.ngs.qc.qualimap.main.NgsSmartMain.main(NgsSmartMain.java:113)
Hi, I am trying to run the tool but get this error message. I have used the newproject.py to create the .conf file I use that to alter my cluster file and get the following error message,
base * /home/crk_tgms5/miniconda
MetaPhage /home/crk_tgms5/miniconda/envs/MetaPhage
PhageBoost-env /home/crk_tgms5/miniconda/envs/PhageBoost-env
ProphET /home/crk_tgms5/miniconda/envs/ProphET
TraDIS /home/crk_tgms5/miniconda/envs/TraDIS
bacterialassembly /home/crk_tgms5/miniconda/envs/bacterialassembly
cutadaptenv /home/crk_tgms5/miniconda/envs/cutadaptenv
flye /home/crk_tgms5/miniconda/envs/flye
N E X T F L O W ~ version 21.04.0
Launching main.nf [thirsty_brahmagupta] - revision: 22a1357f42
====================================================
| ___/| '_ \ / _ |/ ` |/ _ Input: /home/crk_tgms5/MetaPhage/datasets/Greg
Metadata: /home/crk_tgms5/MetaPhage/datasets/Greg/MetaPhage_c4g569fp_metadata
Databases: /home/crk_tgms5/MetaPhage/db
Output: /home/crk_tgms5/MetaPhage/MetaPhage_out
Resources: 16 CPUs, 64 GB RAM
No metadata supplied! Metadata are mandatory for beta diversity and heatmaps plots!
If you are interested in these plots, simply provide a metadata .csv file using params.metadata in your config file!
For more informations, check the metadata paragraph on the wiki at https://github.com/MattiaPandolfoVR/MetaPhage#input_files
No such variable: ch_metadata_checker
-- Check script 'main.nf' at line: 79 or see '.nextflow.log' file for more details
Any help would be much appreciated.
Thanks for the pipeline, curious to try it out!
After installing dependencies in a conda environment - Krona, VIBRANT and QUAST databases normally need to be set up and/or updated:
Krona installed. You still need to manually update the taxonomy
databases before Krona can generate taxonomic reports. The update
script is ktUpdateTaxonomy.sh. The default location for storing
taxonomic databases is /.../.conda/envs/MetaPhage/opt/krona/taxonomy
If you would like the taxonomic data stored elsewhere, simply replace
this directory with a symlink. For example:
rm -rf /.../.conda/envs/MetaPhage/opt/krona/taxonomy
mkdir /path/on/big/disk/taxonomy
ln -s /path/on/big/disk/taxonomy /.../.conda/envs/MetaPhage/opt/krona/taxonomy
ktUpdateTaxonomy.sh
|
Please run download-db.sh to download all required VIBRANT database files to /.../.conda/envs/MetaPhage/share/vibrant-1.2.0/databases/
| The default QUAST package does not include:
* GRIDSS (needed for structural variants detection)
* SILVA 16S rRNA database (needed for reference genome detection in metagenomic datasets)
* BUSCO tools and databases (needed for searching BUSCO genes) -- works in Linux only!
To be able to use those, please run
quast-download-gridss
quast-download-silva
quast-download-busco
Is this covered by database download section or should this also be done in addition before downloading the databases with the python script?
Thanks
Hello,
Thanks again for this amazing work. I wonder whether host (human) reads are removed at the preprocessing step. I only see the removal of the PhiX 174 genome, so not so sure whether there is another step that removes the host genome (I think this step is pretty standard in other metagenomics preprocessing pipelines).
If the host reads are not removed, do you think this will affect the latter steps of bacterial taxonomy and bacteriophage identification?
Hi developer,
I was following installation instructions for MetaPhage2. When installing the databases, the following message was returned. I tried many times with the command: ./bin/python/db_manager.py -o ./db/ -m 6, and the same error message showed every time. Then I checked the db folder and only found virsorter there.
How should I fix this problem? Thank you!
π Downloading bundle 2021.1: 6 databases total
virsorter found: skipping
π¦ Preparing to download PhiX reference
π¦ Preparing to download MiniKraken
π¦ Preparing to download Vibrant
π¦ Preparing to download Phigaro
π¦ Preparing to download vConTACT2
β
PhiX reference downloaded
Error downloading https://warwick.s3.climb.ac.uk/ifrqmra-metaphage/v1.0/2022-01-inphared.tar.gz:
<urlopen error retrieval incomplete: got only 42298052 out of 362798790 bytes>
β
vConTACT2 downloaded
tar (child): 2022-01-inphared.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
rm: cannot remove '2022-01-inphared.tar.gz': No such file or directory
Hi,
I have been trying to work with the example dataset and I got this error is there something I'm doing wrong?
Thanks
Error executing process > 'vibrant (megahit-SRR8652861)'
Caused by:
Process vibrant (megahit-SRR8652861) terminated with an error exit status (1)
Command executed:
VIBRANT_run.py -t 16 -i SRR8652861_megahit_contigs.fasta -d /home/crk_tgms5/MetaPhage/db/vibrant/legacy/ -m /home/crk_tgms5/MetaPhage/db/vibrant/legacy/ -virome
mv VIBRANT_/ SRR8652861_vibrant/
sed 's/^>/>vibrant_/' SRR8652861_vibrant/VIBRANT_phages_SRR8652861_megahit_contigs/SRR8652861_megahit_contigs.phages_combined.fna > SRR8652861_vibrant/VIBRANT_phages_SRR8652861_megahit_contigs/SRR8652861_correct_vibrant.fna
rm -f temp
Command exit status:
1
Command output:
(empty)
Command error:
/cm/local/apps/environment-modules/4.0.0//init/bash: line 15: MODULES_USE_COMPAT_VERSION: unbound variable
Traceback (most recent call last):
File "/home/crk_tgms5/miniconda/envs/MetaPhage/bin/VIBRANT_annotation.py", line 1708, in
AMG_dict.update({str(annotations[n]):str(annotations[n+1]).replace("$~&", " ").replace('^@%','"') + '\t' + str(annotations[n+2]) + '\t' + str(ko_name) + '\t' + str(annotations[n+6]) + '\t' + str(pfam_name)})
NameError: name 'pfam_name' is not defined
Traceback (most recent call last):
File "/home/crk_tgms5/miniconda/envs/MetaPhage/bin/VIBRANT_run.py", line 633, in
with open(str(out_folder)+'VIBRANT_AMG_individuals_' + str(base) + '.tsv', 'r') as annotations:
FileNotFoundError: [Errno 2] No such file or directory: 'VIBRANT_AMG_individuals_SRR8652861_megahit_contigs.tsv'
Work dir:
/home/crk_tgms5/MetaPhage/work/0d/98d8f82c8bfa2d1a2073b0c0f57814
Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line
Hello!
I was trying to set up MetaPhage-0.3.3 and I ran into an error when working on the demo test example. I was unsure why there seems to be a problem with memory when the resources seem to be allocated. Thanks for your help in advance!
This was the program output:
nextflow run main.nf -c demo.conf
N E X T F L O W ~ version 21.04.0
Launching `main.nf` [elated_gates] - revision: afb318457b
====================================================
__ __ _ _____ _
| \/ | | | | __ \| |
| \ / | ___| |_ __ _| |__) | |__ __ _ __ _ ___
| |\/| |/ _ \ __/ _` | ___/| '_ \ / _` |/ _` |/ _ \
| | | | __/ || (_| | | | | | | (_| | (_| | __/
|_| |_|\___|\__\__,_|_| |_| |_|\__,_|\__, |\___|
__/ |
|___/
====================================================
Input: MetaPhage-0.3.3/demo
Metadata: MetaPhage-0.3.3/demo/MetaPhage_v7lmcdpa_metadata
Databases: MetaPhage-0.3.3/db
Output: MetaPhage-0.3.3/MetaPhage-output
Resources: 16 CPUs, 64 GB RAM
Local avail `memory` attribute cannot zero. Expression: (availMemory > 0). Values: availMemory = 0
-- Check script 'main.nf' at line: 74 or see '.nextflow.log' file for more details
Greetings,
I am trying to reproduce the tutorial (https://mattiapandolfovr.github.io/MetaPhage/tutorial), and I got the following errors at the stage of downloading the databases:
`./bin/python/db_manager.py -o ./db/ -m 6
db_manager.py requires wget to be installed
π¦ Preparing to download PhiX reference
π¦ Preparing to download MiniKraken
π¦ Preparing to download Vibrant
π¦ Preparing to download Virsorter
π¦ Preparing to download Phigaro
π¦ Preparing to download vConTACT2
β
PhiX reference downloaded
β
vConTACT2 downloaded
gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now`
As far as I can see, all databases except vConTACT2 could be downloaded (I see the corresponding catalogs in ${METAPHAGE_DIR}/db). Unfortunately, I cannot realize what is going wrong, thus I cannot fix this error.
Perhaps you could help me with it?
Sincerely,
Matvey
Hi There,
I get the following error when running MetaPhage even though the kraken2 and krona output files (*_output and *_report as well as *_krak_krona_abundancies.html) are created and the directory structure looks correct. Any idea where it might be going wrong?
**Error executing process > 'kraken_file (Creating the table...)'
Caused by:
Process `kraken_file (Creating the table...)` terminated with an error exit status (1)
Command executed:
Rscript /home/lonnie/MetaPhage/bin/Rscript/kraken_files.R /home/lonnie/MetaPhage_test metadata.csv
Command exit status:
1
Command output:
Input: /home/lonnie/MetaPhage_test
Command error:
Error: ERROR: taxonomy/kraken2 not found in: /home/lonnie/MetaPhage_test/taxonomy/kraken2
Execution halted
Work dir**: /home/lonnie/MetaPhage/work/fc/7042892072d65cde5b05740cc3c63d
Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run****
Thank you in advance.
Regards
Lonnie
Hello,
I am receiving an error running MetaPhage2. When I tried to download the databases using db_manager.py, I received an error that I cannot download the inphared database. Following the link on the db_manager.py, I downloaded the 2022-01-inphared.tar.gz file manually. All the files from this database are from Jan 17, 2022.
The pipeline keeps crashing at the step diamond_vcontact2 (megahit) because it is looking for inphared databases from Jun1, 2022 (not Jan 17, 2022). I have looked on the inphared website, but no databases were generated on Jun 1, 2022.
error:
Vcontact db: /scratch/hdd2/MetaPhage/db/inphared
Vcontact file head: 1Jun2022_vConTACT2_
Vcontact mod: Jun2022
Reference DB faa: /scratch/hdd2/MetaPhage/db/inphared/1Jun2022_vConTACT2_proteins.faa
Gene to genome: /scratch/hdd2/MetaPhage/db/inphared/1Jun2022_vConTACT2_gene_to_genome.csv
Reference DB: /scratch/hdd2/MetaPhage/db/inphared/Jun2022_reference_db.dmnd
These are the files I downloaded from inphared:
Do you have any suggestions on fixing this error?
Thank you so much!
Carmen
WARN: Killing pending tasks (4)
executor > local (44)
[4e/034777] process > csv_validator (Checking met... [100%] 1 of 1 β
[4c/db9b0a] process > db_manager (Downloading mis... [100%] 1 of 1 β
[c2/16fdb6] process > fastp (SRR8653245) [100%] 10 of 10 β
[0c/361b70] process > remove_phix (SRR8652969) [100%] 10 of 10 β
[07/ec88bb] process > kraken2 (SRR8652969) [100%] 8 of 8
[- ] process > krona [ 0%] 0 of 8
[00/177922] process > megahit (SRR8652969) [100%] 9 of 9
[- ] process > metaquast -
[9a/8a07d5] process > deepvirfinder (megahit-SRR8... [ 0%] 0 of 8
[- ] process > phigaro [ 0%] 0 of 9
[- ] process > vibrant [ 0%] 0 of 9
[de/42f554] process > virfinder (megahit-SRR8653090) [ 11%] 1 of 9, failed: 1
[- ] process > virsorter2 [ 0%] 0 of 9
[- ] process > cdhit -
[- ] process > checkV -
[- ] process > prodigal -
[- ] process > bowtie2_derep -
[- ] process > covtocounts2 -
[- ] process > diamond_vcontact2 -
[- ] process > vcontact2 -
[- ] process > graphanalyzer -
[- ] process > kraken_file -
[- ] process > miner_comparison -
[- ] process > checkv_table -
[- ] process > summary -
[- ] process > phylo_obj -
[- ] process > file_chopper -
[- ] process > taxonomy_table -
[- ] process > alpha_diversity -
[- ] process > beta_diversity -
[- ] process > heatmap -
[- ] process > violin_plots -
[- ] process > multiqc -
Error executing process > 'virfinder (megahit-SRR8653090)'
Caused by:
Process virfinder (megahit-SRR8653090) terminated with an error exit status (1)
Command executed:
Rscript /scratch/xxxx/MetaPhage/bin/Rscript/virfinder_execute.R SRR8653090_megahit_contigs.fasta 8 /scratch/xxxx/MetaPhage
mv results.txt SRR8653090_results.txt
mv viral_sequences.fasta SRR8653090_viral_sequences.fasta
python /scratch/xxxxx/MetaPhage/bin/python/pvalue_virfinder.py SRR8653090
seqtk subseq ./SRR8653090_megahit_contigs.fasta ./SRR8653090_filtered_headers.txt > SRR8653090_viral_tmp_sequences.fasta
sed 's/^>/>virfinder_/' SRR8653090_viral_tmp_sequences.fasta > SRR8653090_viral_sequences.fasta
rm SRR8653090_viral_tmp_sequences.fasta
Command exit status:
1
Command output:
(empty)
Command error:
The following objects are masked from βpackage:statsβ:
IQR, mad, sd, var, xtabs
The following objects are masked from βpackage:baseβ:
anyDuplicated, append, as.data.frame, basename, cbind, colnames,
dirname, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
grepl, intersect, is.unsorted, lapply, Map, mapply, match, mget,
order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
rbind, Reduce, rownames, sapply, setdiff, sort, table, tapply,
union, unique, unsplit, which.max, which.min
Loading required package: S4Vectors
Loading required package: stats4
Attaching package: βS4Vectorsβ
The following objects are masked from βpackage:Matrixβ:
expand, unname
The following objects are masked from βpackage:baseβ:
expand.grid, I, unname
Loading required package: IRanges
Attaching package: βIRangesβ
The following object is masked from βpackage:VirFinderβ:
reverse
Loading required package: XVector
Loading required package: GenomeInfoDb
Error: package or namespace load failed for βGenomeInfoDbβ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
there is no package called βGenomeInfoDbDataβ
Failed with error: βpackage βGenomeInfoDbβ could not be loadedβ
Loading required package: parallel
Loading required package: Biostrings
Loading required package: GenomeInfoDb
Error: package or namespace load failed for βGenomeInfoDbβ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
there is no package called βGenomeInfoDbDataβ
Failed with error: βpackage βGenomeInfoDbβ could not be loadedβ
Error in readDNAStringSet(inFaFile) :
could not find function "readDNAStringSet"
Calls: parVF.pred
Execution halted
Work dir:
/scratch/xxxx/MetaPhage/work/de/42f554c8476418693669732ae2c46e
Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run
slurmstepd: error: *** JOB 4280770 STEPD TERMINATED ON p-sc-2022 AT 2023-06-27T16:06:27 DUE TO JOB NOT ENDING WITH SIGNALS ***
slurmstepd: error: Container 1369132 in cgroup plugin has 2 processes, giving up after 159 sec
Hello, I am getting the following error when trying to download the pre-build image:
wget "https://s3.climb.ac.uk/ifrqmra-metaphage/v1.0/metaphage.simg" --2023-09-13 15:01:15-- https://s3.climb.ac.uk/ifrqmra-metaphage/v1.0/metaphage.simg Resolving s3.climb.ac.uk (s3.climb.ac.uk)... 147.188.173.14 Connecting to s3.climb.ac.uk (s3.climb.ac.uk)|147.188.173.14|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2023-09-13 15:01:15 ERROR 404: Not Found.
Is there a new link? Perhaps with MetaphageV2?
Thank you!
As mentionned in a previous issue, some of the download links for developmental version of MetaPhage2are still not accessible:
I think you mentionned that the link for Metaphage1 has been solved, but I encounter the same problem for the Metaphage2 links... All those related to the uk server apparently:
https://s3.climb.ac.uk/ifrqmra-metaphage/v2.0/virsorter2.tar.gz
https://s3.climb.ac.uk/ifrqmra-metaphage/v2.0/checkv.tar.gz
https://s3.climb.ac.uk/ifrqmra-metaphage/v2.0/inphared.tar.gz
Thank you very much
Hi,
I ran metaphage2 and after mining steps, the CDhit step appeared to be time limited since the program stopped after 24h of calculation...
I went to the NextFlow.config file and set new time limit for all the different steps.
I used the option -resume to start over where it stopped before.
It is now 38h since the command started, and I still observe 0% for CDHit
[80/8e9f76] process > cdhit (megahit) [ 0%] 0 of 1
I used the htop command to see whether something is running... and got that MetaPhage/bin/python/miner_comparison.py megahit is still running since 38h...
Is there something more to do to see whether the program is blocked or if it is really running ? Or have I done something wrong ?
The dataset is 16 differents metavriomes... each with around 10 millions reads .
thank you
Caused by:
Process vcontact2 (megahit) terminated with an error exit status (1)
Command executed:
vcontact2_local -t 4 --blast-fp allVSall_Jan2022_app.csv --rel-mode 'Diamond' --proteins-fp viral_genomes_combined.csv --db 'None' --pcs-mode MCL --vcs-mode ClusterONE --c1-bin /srv/scratch/z3192995/Softwares/MetaPhage-0.3.3/bin/cluster_one-1.0.jar --output-dir ./
Command exit status:
1
Command output:
============================This is vConTACT2 0.9.19============================
----------------------------------Pre-Analysis----------------------------------
------------------------------Reference databases-------------------------------
-------------------------------Protein clustering-------------------------------
----------------------------------Loading data----------------------------------
------------------------Calculating Similarity Networks-------------------------
Command error:
25 .................... 0.08 1.33 1.00/0.90/1.00 1.00 1.00 0.11 0
26 .................... 0.07 1.32 1.00/0.92/1.00 1.00 1.00 0.11 0
27 .................... 0.11 1.33 1.00/0.93/1.00 1.00 1.00 0.11 0
28 .................... 0.16 1.32 1.00/0.85/1.00 1.00 1.00 0.11 0
29 .................... 0.14 1.32 1.00/0.84/1.00 1.00 1.00 0.11 0
30 .................... 0.05 1.32 1.00/0.95/1.00 1.00 1.00 0.11 0
31 .................... 0.00 1.34 1.00/1.00/1.00 1.00 1.00 0.11 0
32 .................... 0.00 1.31 1.00/1.00/1.00 1.00 1.00 0.11 0
[mcl] jury pruning marks: <43,90,98>, out of 100
[mcl] jury pruning synopsis: <61.6 or satisfactory> (cf -scheme, -do log)
[mcl] output is in ./allVSall_Jan2022_app.csv_mcl20.clusters
[mcl] 101605 clusters found
[mcl] output is in ./allVSall_Jan2022_app.csv_mcl20.clusters
Please cite:
Stijn van Dongen, Graph Clustering by Flow Simulation. PhD thesis,
University of Utrecht, May 2000.
( http://www.library.uu.nl/digiarchief/dip/diss/1895620/full.pdf
or http://micans.org/mcl/lit/svdthesis.pdf.gz)
OR
Stijn van Dongen, A cluster algorithm for graphs. Technical
Report INS-R0010, National Research Institute for Mathematics
and Computer Science in the Netherlands, Amsterdam, May 2000.
( http://www.cwi.nl/ftp/CWIreports/INS/INS-R0010.ps.Z
or http://micans.org/mcl/lit/INS-R0010.ps.Z)
οΏ½[1;42mINFOοΏ½[1;0m:vcontact2: Building the cluster and profiles (this may take some time...)
If it fails, try re-running using --blast-fp flag and specifiying merged.self-diamond.tab (or merged.self-blastp.tab)
οΏ½[1;42mINFOοΏ½[1;0m:vcontact2: Saving intermediate files...
οΏ½[1;42mINFOοΏ½[1;0m:vcontact2: Read 1981872 entries (dropped 117915 singletons) from ./vConTACT_profiles.csv
Traceback (most recent call last):
File "/srv/scratch/z3192995/Softwares/MetaPhage-0.3.3/bin/vcontact2_local", line 761, in
main(options)
File "/srv/scratch/z3192995/Softwares/MetaPhage-0.3.3/bin/vcontact2_local", line 596, in main
args.mod_sig, args.mod_shared_min)
File "/opt/software/conda_env/lib/python3.7/site-packages/vcontact2/pcprofiles.py", line 71, in init
self.ntw = self.network(self.matrix, self.singletons, thres=sig, max_sig=max_sig, threads=self.threads)
File "/opt/software/conda_env/lib/python3.7/site-packages/vcontact2/pcprofiles.py", line 150, in network
final_results = list(chain.from_iterable([r.get() for r in results]))
File "/opt/software/conda_env/lib/python3.7/site-packages/vcontact2/pcprofiles.py", line 150, in
final_results = list(chain.from_iterable([r.get() for r in results]))
File "/opt/software/conda_env/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
File "/opt/software/conda_env/lib/python3.7/multiprocessing/pool.py", line 431, in _handle_tasks
put(task)
File "/opt/software/conda_env/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/opt/software/conda_env/lib/python3.7/multiprocessing/connection.py", line 393, in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
Work dir:
/srv/scratch/z3192995/Softwares/MetaPhage-0.3.3/work/3b/80c6bf421b91e9658ef6c9633eb62e
Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out
Sep.-21 11:15:16.961 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process vcontact2 (megahit) terminated with an error exit status (1)
Sep.-21 11:15:17.034 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] summary
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) bound ; channel: -
port 2: (queue) OPEN ; channel: metadata
port 3: (cntrl) - ; channel: $
[process] phylo_obj
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) OPEN ; channel: -
port 2: (queue) OPEN ; channel: metadata
port 3: (cntrl) - ; channel: $
[process] file_chopper
status=ACTIVE
port 0: (value) bound ; channel: -
port 1: (value) bound ; channel: -
port 2: (value) bound ; channel: -
port 3: (cntrl) - ; channel: $
[process] taxonomy_table
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) bound ; channel: -
port 2: (value) bound ; channel: -
port 3: (value) bound ; channel: -
port 4: (value) bound ; channel: -
port 5: (queue) OPEN ; channel: metadata
port 6: (cntrl) - ; channel: $
[process] heatmap
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) OPEN ; channel: -
port 2: (queue) OPEN ; channel: metadata
port 3: (cntrl) - ; channel: $
[process] alpha_diversity
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) bound ; channel: -
port 2: (queue) OPEN ; channel: metadata
port 3: (cntrl) - ; channel: $
[process] betadiversity
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) bound ; channel: -
port 2: (queue) OPEN ; channel: metadata
port 3: (cntrl) - ; channel: $
[process] violin_plots
status=ACTIVE
port 0: (value) OPEN ; channel: -
port 1: (value) bound ; channel: -
port 2: (queue) OPEN ; channel: metadata
port 3: (cntrl) - ; channel: $
[process] multiqc
status=ACTIVE
port 0: (value) bound ; channel: min_comp
port 1: (value) OPEN ; channel: summary_table
port 2: (value) OPEN ; channel: taxo_table
port 3: (value) bound ; channel: fastp
port 4: (value) bound ; channel: kraken2
port 5: (value) bound ; channel: krakenfiles
port 6: (value) bound ; channel: metaquast
port 7: (cntrl) - ; channel: $
Sep.-21 11:15:17.062 [Actor Thread 51] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process taxonomy_table -- offending value: []
Sep.-21 11:15:17.074 [main] DEBUG nextflow.Session - Session await > all processes finished
Sep.-21 11:15:17.074 [main] DEBUG nextflow.Session - Session await > all barriers passed
Sep.-21 11:15:17.089 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (1)
Sep.-21 11:15:17.128 [Actor Thread 54] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process summary -- offending value: []
Sep.-21 11:15:17.134 [Actor Thread 34] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process phylo_obj -- offending value: []
Sep.-21 11:15:17.159 [Actor Thread 42] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process alpha_diversity -- offending value: []
Sep.-21 11:15:17.162 [Actor Thread 53] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process violin_plots -- offending value: []
Sep.-21 11:15:17.273 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=7; ignoredCount=6; cachedCount=2754; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=1; succeedDuration=0ms; failedDuration=8h 6m 41s; cachedDuration=740d 14h 9m 28s;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=8; peakMemory=850 GB; ]
Sep.-21 11:15:17.273 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Sep.-21 11:15:18.849 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data:
[{"cpuUsage":null,"process":"csv_validator","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":null,"cpu":null,"time":{"mean":15008,"min":14,"q1":7511,"q2":15008,"q3":22505,"max":30002,"minLabel":"csv_validator (Checking metadata format...)","maxLabel":"csv_validator (Checking metadata format...)","q1Label":"csv_validator (Checking metadata format...)","q2Label":"csv_validator (Checking metadata format...)","q3Label":"csv_validator (Checking metadata format...)"},"writes":null},{"cpuUsage":null,"process":"db_manager","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":null,"cpu":null,"time":{"mean":6,"min":6,"q1":6,"q2":6,"q3":6,"max":6,"minLabel":"db_manager (Downloading missing databases...)","maxLabel":"db_manager (Downloading missing databases...)","q1Label":"db_manager (Downloading missing databases...)","q2Label":"db_manager (Downloading missing databases...)","q3Label":"db_manager (Downloading missing databases...)"},"writes":null},{"cpuUsage":null,"process":"fastp","mem":null,"memUsage":null,"timeUsage":{"mean":0.4,"min":0,"q1":0.26,"q2":0.42,"q3":0.51,"max":1.1,"minLabel":"fastp (MTG11)","maxLabel":"fastp (MTG193)","q1Label":"fastp (MTG106)","q2Label":"fastp (MTG240)","q3Label":"fastp (MTG105)"},"vmem":null,"reads":null,"cpu":null,"time":{"mean":334421.04,"min":3,"q1":215050.25,"q2":345176.5,"q3":419863.25,"max":909466,"minLabel":"fastp (MTG11)","maxLabel":"fastp (MTG193)","q1Label":"fastp (MTG106)","q2Label":"fastp (MTG240)","q3Label":"fastp (MTG105)"},"writes":null},{"cpuUsage":null,"process":"remove_phix","mem":null,"memUsage":null,"timeUsage":{"mean":4.04,"min":0,"q1":2.38,"q2":4.39,"q3":5.56,"max":9.13,"minLabel":"remove_phix (MTG196)","maxLabel":"remove_phix (MTG197)","q1Label":"remove_phix (MTG244)","q2Label":"remove_phix (MTG191)","q3Label":"remove_phix (MTG95)"},"vmem":null,"reads":null,"cpu":null,"time":{"mean":1661840.71,"min":2,"q1":1252599,"q2":1795282,"q3":2247892.5,"max":3615418,"minLabel":"remove_phix (MTG196)","maxLabel":"remove_phix (MTG197)","q1Label":"remove_phix (MTG14)","q2Label":"remove_phix (MTG71)","q3Label":"remove_phix (MTG141)"},"writes":null},{"cpuUsage":null,"process":"megahit","mem":null,"memUsage":null,"timeUsage":{"mean":2.69,"min":0.09,"q1":1.54,"q2":2.79,"q3":3.9,"max":7.63,"minLabel":"megahit (MTG124)","maxLabel":"megahit (MTG206)","q1Label":"megahit (MTG92)","q2Label":"megahit (MTG14)","q3Label":"megahit (MTG128)"},"vmem":null,"reads":null,"cpu":null,"time":{"mean":4554019.47,"min":150010,"q1":2600022,"q2":4715107,"q3":6595145,"max":12905304,"minLabel":"megahit (MTG124)","maxLabel":"megahit (MTG206)","q1Label":"megahit (MTG92)","q2Label":"megahit (MTG14)","q3Label":"megahit (MTG128)"},"writes":null},{"cpuUsage":null,"process":"kraken2","mem":null,"memUsage":null,"timeUsage":{"mean":0.1,"min":0,"q1":0.06,"q2":0.1,"q3":0.13,"max":0.24,"minLabel":"kraken2 (MTG109)","maxLabel":"kraken2 (MTG35)","q1Label":"kraken2 (MTG173)","q2Label":"kraken2 (MTG86)","q3Label":"kraken2 (MTG9)"},"vmem":null,"reads":null,"cpu":null,"time":{"mean":169668.33,"min":3,"q1":100096,"q2":165028,"q3":225007.5,"max":405015,"minLabel":"kraken2 (MTG109)","maxLabel":"kraken2 (MTG35)","q1Label":"kraken2 (MTG173)","q2Label":"kraken2 (MTG86)","q3Label":"kraken2 (MTG9)"},"writes":null},{"cpuUsage":{"mean":30.39,"min":19.83,"q1":29.65,"q2":31.23,"q3":31.93,"max":32.99,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG116)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG243)","q3Label":"phigaro (megahit-MTG18)"},"process":"phigaro","mem":{"mean":1754877678.93,"min":235552768,"q1":1397683200,"q2":1894144000,"q3":2092212224,"max":2646978560,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG206)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG144)","q3Label":"phigaro (megahit-MTG216)"},"memUsage":{"mean":0.74,"min":0.1,"q1":0.59,"q2":0.8,"q3":0.89,"max":1.12,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG206)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG144)","q3Label":"phigaro (megahit-MTG216)"},"timeUsage":{"mean":0.85,"min":0,"q1":0.44,"q2":0.89,"q3":1.19,"max":2.53,"minLabel":"phigaro (megahit-MTG88)","maxLabel":"phigaro (megahit-MTG189)","q1Label":"phigaro (megahit-MTG123)","q2Label":"phigaro (megahit-MTG110)","q3Label":"phigaro (megahit-MTG51)"},"vmem":{"mean":3533602816,"min":1861632000,"q1":3195725824,"q2":3693125632,"q3":3862291456,"max":4366585856,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG206)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG215)","q3Label":"phigaro (megahit-MTG233)"},"reads":{"mean":337070461316.47,"min":13348054716,"q1":181601174938.75,"q2":354356440736.5,"q3":427482254177.75,"max":938075899275,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG206)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG207)","q3Label":"phigaro (megahit-MTG233)"},"cpu":{"mean":243.08,"min":158.6,"q1":237.23,"q2":249.8,"q3":255.48,"max":263.9,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG116)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG243)","q3Label":"phigaro (megahit-MTG18)"},"time":{"mean":1445977.45,"min":3,"q1":750250,"q2":1505043,"q3":2005209,"max":4285024,"minLabel":"phigaro (megahit-MTG88)","maxLabel":"phigaro (megahit-MTG189)","q1Label":"phigaro (megahit-MTG123)","q2Label":"phigaro (megahit-MTG110)","q3Label":"phigaro (megahit-MTG51)"},"writes":{"mean":284621516.67,"min":21626497,"q1":159583972.5,"q2":296635670.5,"q3":363578674.75,"max":759350996,"minLabel":"phigaro (megahit-MTG135)","maxLabel":"phigaro (megahit-MTG206)","q1Label":"phigaro (megahit-MTG9)","q2Label":"phigaro (megahit-MTG207)","q3Label":"phigaro (megahit-MTG233)"}},{"cpuUsage":{"mean":89.71,"min":63.53,"q1":88.18,"q2":93.13,"q3":94.81,"max":95.68,"minLabel":"virfinder (megahit-MTG206)","maxLabel":"virfinder (megahit-MTG161)","q1Label":"virfinder (megahit-MTG215)","q2Label":"virfinder (megahit-MTG171)","q3Label":"virfinder (megahit-MTG207)"},"process":"virfinder","mem":{"mean":8490123660.39,"min":572166144,"q1":5676242944,"q2":9535422464,"q3":11629950976,"max":18717032448,"minLabel":"virfinder (megahit-MTG107)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG125)","q2Label":"virfinder (megahit-MTG144)","q3Label":"virfinder (megahit-MTG152)"},"memUsage":{"mean":2.64,"min":0.18,"q1":1.76,"q2":2.96,"q3":3.61,"max":5.81,"minLabel":"virfinder (megahit-MTG107)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG125)","q2Label":"virfinder (megahit-MTG144)","q3Label":"virfinder (megahit-MTG152)"},"timeUsage":{"mean":2.51,"min":0.02,"q1":0.9,"q2":2.08,"q3":3.71,"max":12.76,"minLabel":"virfinder (megahit-MTG11)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG200)","q2Label":"virfinder (megahit-MTG184)","q3Label":"virfinder (megahit-MTG230)"},"vmem":{"mean":16773056445.94,"min":1744420864,"q1":15052929024,"q2":20091191296,"q3":22176671744,"max":29266702336,"minLabel":"virfinder (megahit-MTG107)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG125)","q2Label":"virfinder (megahit-MTG144)","q3Label":"virfinder (megahit-MTG152)"},"reads":{"mean":164417387331.06,"min":7280835685,"q1":83226501300.5,"q2":167385789825,"q3":238320060570,"max":436582997727,"minLabel":"virfinder (megahit-MTG107)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG117)","q2Label":"virfinder (megahit-MTG170)","q3Label":"virfinder (megahit-MTG233)"},"cpu":{"mean":717.65,"min":508.2,"q1":705.4,"q2":745,"q3":758.5,"max":765.4,"minLabel":"virfinder (megahit-MTG206)","maxLabel":"virfinder (megahit-MTG161)","q1Label":"virfinder (megahit-MTG215)","q2Label":"virfinder (megahit-MTG171)","q3Label":"virfinder (megahit-MTG207)"},"time":{"mean":4247145.88,"min":34977,"q1":1525038,"q2":3514795,"q3":6270127,"max":21587306,"minLabel":"virfinder (megahit-MTG11)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG200)","q2Label":"virfinder (megahit-MTG184)","q3Label":"virfinder (megahit-MTG230)"},"writes":{"mean":44072091.45,"min":2648329,"q1":23277410.5,"q2":44937395,"q3":63392602,"max":113077072,"minLabel":"virfinder (megahit-MTG107)","maxLabel":"virfinder (megahit-MTG206)","q1Label":"virfinder (megahit-MTG117)","q2Label":"virfinder (megahit-MTG170)","q3Label":"virfinder (megahit-MTG233)"}},{"cpuUsage":{"mean":4.19,"min":4.19,"q1":4.19,"q2":4.19,"q3":4.19,"max":4.19,"minLabel":"krona (MTG144)","maxLabel":"krona (MTG144)","q1Label":"krona (MTG144)","q2Label":"krona (MTG144)","q3Label":"krona (MTG144)"},"process":"krona","mem":{"mean":8937472,"min":8937472,"q1":8937472,"q2":8937472,"q3":8937472,"max":8937472,"minLabel":"krona (MTG144)","maxLabel":"krona (MTG144)","q1Label":"krona (MTG144)","q2Label":"krona (MTG144)","q3Label":"krona (MTG144)"},"memUsage":null,"timeUsage":{"mean":0.33,"min":0,"q1":0,"q2":0,"q3":0,"max":31.94,"minLabel":"krona (MTG208)","maxLabel":"krona (MTG211)","q1Label":"krona (MTG113)","q2Label":"krona (MTG207)","q3Label":"krona (MTG185)"},"vmem":{"mean":34816000,"min":34816000,"q1":34816000,"q2":34816000,"q3":34816000,"max":34816000,"minLabel":"krona (MTG144)","maxLabel":"krona (MTG144)","q1Label":"krona (MTG144)","q2Label":"krona (MTG144)","q3Label":"krona (MTG144)"},"reads":{"mean":3808089,"min":3808089,"q1":3808089,"q2":3808089,"q3":3808089,"max":3808089,"minLabel":"krona (MTG144)","maxLabel":"krona (MTG144)","q1Label":"krona (MTG144)","q2Label":"krona (MTG144)","q3Label":"krona (MTG144)"},"cpu":{"mean":33.5,"min":33.5,"q1":33.5,"q2":33.5,"q3":33.5,"max":33.5,"minLabel":"krona (MTG144)","maxLabel":"krona (MTG144)","q1Label":"krona (MTG144)","q2Label":"krona (MTG144)","q3Label":"krona (MTG144)"},"time":{"mean":551972.99,"min":0,"q1":4,"q2":5,"q3":8,"max":54041619,"minLabel":"krona (MTG119)","maxLabel":"krona (MTG211)","q1Label":"krona (MTG131)","q2Label":"krona (MTG233)","q3Label":"krona (MTG11)"},"writes":{"mean":3177046,"min":3177046,"q1":3177046,"q2":3177046,"q3":3177046,"max":3177046,"minLabel":"krona (MTG144)","maxLabel":"krona (MTG144)","q1Label":"krona (MTG144)","q2Label":"krona (MTG144)","q3Label":"krona (MTG144)"}},{"cpuUsage":{"mean":47.99,"min":20.56,"q1":49.14,"q2":53.1,"q3":54.2,"max":56.79,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG243)","q1Label":"metaquast (megahit-MTG125)","q2Label":"metaquast (megahit-MTG207)","q3Label":"metaquast (megahit-MTG225)"},"process":"metaquast","mem":{"mean":3458609939.69,"min":270602240,"q1":2628040704,"q2":3887888384,"q3":4039144448,"max":7018143744,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG206)","q1Label":"metaquast (megahit-MTG117)","q2Label":"metaquast (megahit-MTG18)","q3Label":"metaquast (megahit-MTG233)"},"memUsage":{"mean":1.46,"min":0.11,"q1":1.11,"q2":1.65,"q3":1.71,"max":2.97,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG206)","q1Label":"metaquast (megahit-MTG117)","q2Label":"metaquast (megahit-MTG18)","q3Label":"metaquast (megahit-MTG233)"},"timeUsage":{"mean":0.03,"min":0,"q1":0,"q2":0.02,"q3":0.06,"max":0.09,"minLabel":"metaquast (megahit-MTG65)","maxLabel":"metaquast (megahit-MTG216)","q1Label":"metaquast (megahit-MTG196)","q2Label":"metaquast (megahit-MTG204)","q3Label":"metaquast (megahit-MTG64)"},"vmem":{"mean":14797299869.54,"min":4602318848,"q1":14326585344,"q2":15556239360,"q3":15628702720,"max":18506715136,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG206)","q1Label":"metaquast (megahit-MTG180)","q2Label":"metaquast (megahit-MTG152)","q3Label":"metaquast (megahit-MTG18)"},"reads":{"mean":2144846140.23,"min":62180376,"q1":1391652687,"q2":2164120751,"q3":2927618749,"max":5709932887,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG206)","q1Label":"metaquast (megahit-MTG180)","q2Label":"metaquast (megahit-MTG18)","q3Label":"metaquast (megahit-MTG242)"},"cpu":{"mean":383.92,"min":164.5,"q1":393.15,"q2":424.8,"q3":433.6,"max":454.3,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG243)","q1Label":"metaquast (megahit-MTG125)","q2Label":"metaquast (megahit-MTG207)","q3Label":"metaquast (megahit-MTG225)"},"time":{"mean":43921.13,"min":2,"q1":5,"q2":34993,"q3":93369.75,"max":155480,"minLabel":"metaquast (megahit-MTG65)","maxLabel":"metaquast (megahit-MTG216)","q1Label":"metaquast (megahit-MTG196)","q2Label":"metaquast (megahit-MTG204)","q3Label":"metaquast (megahit-MTG64)"},"writes":{"mean":209154640.46,"min":6107160,"q1":136485924.75,"q2":209769109.5,"q3":283793527.25,"max":554674901,"minLabel":"metaquast (megahit-MTG107)","maxLabel":"metaquast (megahit-MTG206)","q1Label":"metaquast (megahit-MTG180)","q2Label":"metaquast (megahit-MTG18)","q3Label":"metaquast (megahit-MTG242)"}},{"cpuUsage":{"mean":88.41,"min":66.58,"q1":87.27,"q2":90.33,"q3":92.34,"max":96.4,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG170)","q2Label":"vibrant (megahit-MTG234)","q3Label":"vibrant (megahit-MTG117)"},"process":"vibrant","mem":{"mean":3375876193.52,"min":1298702336,"q1":3138478080,"q2":3385958400,"q3":3587250176,"max":4463226880,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG152)","q2Label":"vibrant (megahit-MTG51)","q3Label":"vibrant (megahit-MTG224)"},"memUsage":{"mean":1.43,"min":0.55,"q1":1.33,"q2":1.43,"q3":1.52,"max":1.89,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG152)","q2Label":"vibrant (megahit-MTG51)","q3Label":"vibrant (megahit-MTG224)"},"timeUsage":{"mean":2.13,"min":0,"q1":1.14,"q2":2.21,"q3":3.05,"max":5.85,"minLabel":"vibrant (megahit-MTG88)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG48)","q2Label":"vibrant (megahit-MTG215)","q3Label":"vibrant (megahit-MTG237)"},"vmem":{"mean":13773771824.76,"min":11766231040,"q1":13493915648,"q2":13704747008,"q3":14092999680,"max":15457185792,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG234)","q1Label":"vibrant (megahit-MTG207)","q2Label":"vibrant (megahit-MTG9)","q3Label":"vibrant (megahit-MTG162)"},"reads":{"mean":875084249855.64,"min":21861536585,"q1":395953582978.25,"q2":918872576335,"q3":1195924683919.5,"max":2362192684620,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG90)","q2Label":"vibrant (megahit-MTG144)","q3Label":"vibrant (megahit-MTG233)"},"cpu":{"mean":707.25,"min":532.6,"q1":698.18,"q2":722.6,"q3":738.73,"max":771.2,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG170)","q2Label":"vibrant (megahit-MTG234)","q3Label":"vibrant (megahit-MTG117)"},"time":{"mean":3605992.7,"min":6,"q1":1931355.75,"q2":3733973,"q3":5152375,"max":9900522,"minLabel":"vibrant (megahit-MTG88)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG48)","q2Label":"vibrant (megahit-MTG215)","q3Label":"vibrant (megahit-MTG237)"},"writes":{"mean":4000888147.88,"min":480912365,"q1":2068301670.75,"q2":4151713789,"q3":5329383954.75,"max":10070897393,"minLabel":"vibrant (megahit-MTG107)","maxLabel":"vibrant (megahit-MTG206)","q1Label":"vibrant (megahit-MTG90)","q2Label":"vibrant (megahit-MTG215)","q3Label":"vibrant (megahit-MTG233)"}},{"cpuUsage":{"mean":48.36,"min":21.91,"q1":48.48,"q2":49.68,"q3":51.34,"max":54.78,"minLabel":"virsorter (megahit-MTG107)","maxLabel":"virsorter (megahit-MTG143)","q1Label":"virsorter (megahit-MTG243)","q2Label":"virsorter (megahit-MTG44)","q3Label":"virsorter (megahit-MTG134)"},"process":"virsorter","mem":{"mean":1498153545.14,"min":73568256,"q1":1310136320,"q2":1488900096,"q3":1693243392,"max":3160010752,"minLabel":"virsorter (megahit-MTG107)","maxLabel":"virsorter (megahit-MTG206)","q1Label":"virsorter (megahit-MTG134)","q2Label":"virsorter (megahit-MTG80)","q3Label":"virsorter (megahit-MTG49)"},"memUsage":{"mean":0.63,"min":0.03,"q1":0.55,"q2":0.63,"q3":0.72,"max":1.34,"minLabel":"virsorter (megahit-MTG107)","maxLabel":"virsorter (megahit-MTG206)","q1Label":"virsorter (megahit-MTG134)","q2Label":"virsorter (megahit-MTG80)","q3Label":"virsorter (megahit-MTG49)"},"timeUsage":{"mean":7.34,"min":0.07,"q1":3.7,"q2":7.53,"q3":10.38,"max":20.3,"minLabel":"virsorter (megahit-MTG11)","maxLabel":"virsorter (megahit-MTG241)","q1Label":"virsorter (megahit-MTG12)","q2Label":"virsorter (megahit-MTG66)","q3Label":"virsorter (megahit-MTG247)"},"vmem":{"mean":2085359518.48,"min":1438695424,"q1":1937147904,"q2":2068013056,"q3":2261074944,"max":3317575680,"minLabel":"virsorter (megahit-MTG135)","maxLabel":"virsorter (megahit-MTG206)","q1Label":"virsorter (megahit-MTG44)","q2Label":"virsorter (megahit-MTG171)","q3Label":"virsorter (megahit-MTG89)"},"reads":{"mean":2473296715320.1,"min":24510356282,"q1":1156211579141.75,"q2":2461510519300.5,"q3":3320736571825.25,"max":6671255427097,"minLabel":"virsorter (megahit-MTG107)","maxLabel":"virsorter (megahit-MTG206)","q1Label":"virsorter (megahit-MTG90)","q2Label":"virsorter (megahit-MTG207)","q3Label":"virsorter (megahit-MTG216)"},"cpu":{"mean":386.89,"min":175.3,"q1":387.88,"q2":397.45,"q3":410.7,"max":438.2,"minLabel":"virsorter (megahit-MTG107)","maxLabel":"virsorter (megahit-MTG143)","q1Label":"virsorter (megahit-MTG243)","q2Label":"virsorter (megahit-MTG44)","q3Label":"virsorter (megahit-MTG134)"},"time":{"mean":12415232.44,"min":125003,"q1":6252644.5,"q2":12735828.5,"q3":17556658,"max":34347825,"minLabel":"virsorter (megahit-MTG11)","maxLabel":"virsorter (megahit-MTG241)","q1Label":"virsorter (megahit-MTG12)","q2Label":"virsorter (megahit-MTG66)","q3Label":"virsorter (megahit-MTG247)"},"writes":{"mean":5191838818.38,"min":3466515870,"q1":4308430730,"q2":5144729015.5,"q3":5823503981,"max":8142167870,"minLabel":"virsorter (megahit-MTG107)","maxLabel":"virsorter (megahit-MTG206)","q1Label":"virsorter (megahit-MTG44)","q2Label":"virsorter (megahit-MTG36)","q3Label":"virsorter (megahit-MTG216)"}},{"cpuUsage":{"mean":3.15,"min":1.95,"q1":2.55,"q2":3.15,"q3":3.75,"max":4.35,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"process":"kraken_file","mem":{"mean":167720960,"min":136265728,"q1":151993344,"q2":167720960,"q3":183448576,"max":199176192,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"memUsage":{"mean":0.16,"min":0.13,"q1":0.14,"q2":0.16,"q3":0.17,"max":0.19,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"timeUsage":{"mean":0.02,"min":0.01,"q1":0.01,"q2":0.02,"q3":0.02,"max":0.02,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"vmem":{"mean":551051108352,"min":1256132608,"q1":276153620480,"q2":551051108352,"q3":825948596224,"max":1100846084096,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"reads":{"mean":22450649.5,"min":22450251,"q1":22450450.25,"q2":22450649.5,"q3":22450848.75,"max":22451048,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"cpu":{"mean":25.2,"min":15.6,"q1":20.4,"q2":25.2,"q3":30,"max":34.8,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"time":{"mean":14288,"min":7938,"q1":11113,"q2":14288,"q3":17463,"max":20638,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"},"writes":{"mean":1814778.5,"min":1814778,"q1":1814778.25,"q2":1814778.5,"q3":1814778.75,"max":1814779,"minLabel":"kraken_file (Creating the table...)","maxLabel":"kraken_file (Creating the table...)","q1Label":"kraken_file (Creating the table...)","q2Label":"kraken_file (Creating the table...)","q3Label":"kraken_file (Creating the table...)"}},{"cpuUsage":{"mean":41.24,"min":41.24,"q1":41.24,"q2":41.24,"q3":41.24,"max":41.24,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"process":"cdhit","mem":{"mean":10033971200,"min":10033971200,"q1":10033971200,"q2":10033971200,"q3":10033971200,"max":10033971200,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"memUsage":{"mean":4.25,"min":4.25,"q1":4.25,"q2":4.25,"q3":4.25,"max":4.25,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"timeUsage":{"mean":87.14,"min":87.14,"q1":87.14,"q2":87.14,"q3":87.14,"max":87.14,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"vmem":{"mean":10280259584,"min":10280259584,"q1":10280259584,"q2":10280259584,"q3":10280259584,"max":10280259584,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"reads":{"mean":15813546332,"min":15813546332,"q1":15813546332,"q2":15813546332,"q3":15813546332,"max":15813546332,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"cpu":{"mean":329.9,"min":329.9,"q1":329.9,"q2":329.9,"q3":329.9,"max":329.9,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"time":{"mean":147448440,"min":147448440,"q1":147448440,"q2":147448440,"q3":147448440,"max":147448440,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"},"writes":{"mean":11514215007,"min":11514215007,"q1":11514215007,"q2":11514215007,"q3":11514215007,"max":11514215007,"minLabel":"cdhit (megahit)","maxLabel":"cdhit (megahit)","q1Label":"cdhit (megahit)","q2Label":"cdhit (megahit)","q3Label":"cdhit (megahit)"}},{"cpuUsage":{"mean":10.43,"min":10.43,"q1":10.43,"q2":10.43,"q3":10.43,"max":10.43,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"process":"miner_comparison","mem":{"mean":218849280,"min":218849280,"q1":218849280,"q2":218849280,"q3":218849280,"max":218849280,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"memUsage":{"mean":0.2,"min":0.2,"q1":0.2,"q2":0.2,"q3":0.2,"max":0.2,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"timeUsage":{"mean":0.01,"min":0.01,"q1":0.01,"q2":0.01,"q3":0.01,"max":0.01,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"vmem":{"mean":1597853696,"min":1597853696,"q1":1597853696,"q2":1597853696,"q3":1597853696,"max":1597853696,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"reads":{"mean":33296329,"min":33296329,"q1":33296329,"q2":33296329,"q3":33296329,"max":33296329,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"cpu":{"mean":83.4,"min":83.4,"q1":83.4,"q2":83.4,"q3":83.4,"max":83.4,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"time":{"mean":5839,"min":5839,"q1":5839,"q2":5839,"q3":5839,"max":5839,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"},"writes":{"mean":53351,"min":53351,"q1":53351,"q2":53351,"q3":53351,"max":53351,"minLabel":"miner_comparison (megahit)","maxLabel":"miner_comparison (megahit)","q1Label":"miner_comparison (megahit)","q2Label":"miner_comparison (megahit)","q3Label":"miner_comparison (megahit)"}},{"cpuUsage":{"mean":88.31,"min":75.78,"q1":87.58,"q2":89.61,"q3":90.89,"max":96.83,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"process":"bowtie2_derep","mem":{"mean":3545338183.68,"min":2233745408,"q1":3692344320,"q2":3696320512,"q3":3709361152,"max":3720884224,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"memUsage":{"mean":1.03,"min":0.65,"q1":1.07,"q2":1.08,"q3":1.08,"max":1.08,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"timeUsage":{"mean":6.04,"min":1.24,"q1":4.06,"q2":6.18,"q3":7.75,"max":29.81,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"vmem":{"mean":4743094435.84,"min":2811809792,"q1":4726558720,"q2":4729724928,"q3":4772298752,"max":5015592960,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"reads":{"mean":74592586778.26,"min":14571501752,"q1":56347343288.5,"q2":81003792131.5,"q3":97751033811.25,"max":138307553968,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"cpu":{"mean":353.24,"min":303.1,"q1":350.33,"q2":358.45,"q3":363.58,"max":387.3,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"time":{"mean":5005126.17,"min":1026055,"q1":3362260.75,"q2":5114532.5,"q3":6417790.5,"max":24682486,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"},"writes":{"mean":55494852370.32,"min":10389839728,"q1":41531031223.5,"q2":60426277192,"q3":72996352932.75,"max":106518862336,"minLabel":"bowtie2_derep (megahit)","maxLabel":"bowtie2_derep (megahit)","q1Label":"bowtie2_derep (megahit)","q2Label":"bowtie2_derep (megahit)","q3Label":"bowtie2_derep (megahit)"}},{"cpuUsage":{"mean":12.46,"min":12.46,"q1":12.46,"q2":12.46,"q3":12.46,"max":12.46,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"process":"prodigal","mem":{"mean":64598016,"min":64598016,"q1":64598016,"q2":64598016,"q3":64598016,"max":64598016,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"memUsage":{"mean":0.03,"min":0.03,"q1":0.03,"q2":0.03,"q3":0.03,"max":0.03,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"timeUsage":{"mean":4.06,"min":4.06,"q1":4.06,"q2":4.06,"q3":4.06,"max":4.06,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"vmem":{"mean":129359872,"min":129359872,"q1":129359872,"q2":129359872,"q3":129359872,"max":129359872,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"reads":{"mean":1112394836,"min":1112394836,"q1":1112394836,"q2":1112394836,"q3":1112394836,"max":1112394836,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"cpu":{"mean":99.7,"min":99.7,"q1":99.7,"q2":99.7,"q3":99.7,"max":99.7,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"time":{"mean":6861344,"min":6861344,"q1":6861344,"q2":6861344,"q3":6861344,"max":6861344,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"},"writes":{"mean":818432597,"min":818432597,"q1":818432597,"q2":818432597,"q3":818432597,"max":818432597,"minLabel":"prodigal (megahit)","maxLabel":"prodigal (megahit)","q1Label":"prodigal (megahit)","q2Label":"prodigal (megahit)","q3Label":"prodigal (megahit)"}},{"cpuUsage":{"mean":98.55,"min":98.55,"q1":98.55,"q2":98.55,"q3":98.55,"max":98.55,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"process":"diamond_vcontact2","mem":{"mean":6461841408,"min":6461841408,"q1":6461841408,"q2":6461841408,"q3":6461841408,"max":6461841408,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"memUsage":{"mean":0.93,"min":0.93,"q1":0.93,"q2":0.93,"q3":0.93,"max":0.93,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"timeUsage":{"mean":35.83,"min":35.83,"q1":35.83,"q2":35.83,"q3":35.83,"max":35.83,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"vmem":{"mean":7013941248,"min":7013941248,"q1":7013941248,"q2":7013941248,"q3":7013941248,"max":7013941248,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"reads":{"mean":20611520290,"min":20611520290,"q1":20611520290,"q2":20611520290,"q3":20611520290,"max":20611520290,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"cpu":{"mean":394.2,"min":394.2,"q1":394.2,"q2":394.2,"q3":394.2,"max":394.2,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"time":{"mean":29671262,"min":29671262,"q1":29671262,"q2":29671262,"q3":29671262,"max":29671262,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"},"writes":{"mean":21771625070,"min":21771625070,"q1":21771625070,"q2":21771625070,"q3":21771625070,"max":21771625070,"minLabel":"diamond_vcontact2 (megahit)","maxLabel":"diamond_vcontact2 (megahit)","q1Label":"diamond_vcontact2 (megahit)","q2Label":"diamond_vcontact2 (megahit)","q3Label":"diamond_vcontact2 (megahit)"}},{"cpuUsage":{"mean":4.74,"min":4.74,"q1":4.74,"q2":4.74,"q3":4.74,"max":4.74,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"process":"file_chopper","mem":{"mean":6270713856,"min":6270713856,"q1":6270713856,"q2":6270713856,"q3":6270713856,"max":6270713856,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"memUsage":{"mean":5.84,"min":5.84,"q1":5.84,"q2":5.84,"q3":5.84,"max":5.84,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"timeUsage":{"mean":28.33,"min":28.33,"q1":28.33,"q2":28.33,"q3":28.33,"max":28.33,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"vmem":{"mean":7387615232,"min":7387615232,"q1":7387615232,"q2":7387615232,"q3":7387615232,"max":7387615232,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"reads":{"mean":1944927816,"min":1944927816,"q1":1944927816,"q2":1944927816,"q3":1944927816,"max":1944927816,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"cpu":{"mean":37.9,"min":37.9,"q1":37.9,"q2":37.9,"q3":37.9,"max":37.9,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"time":{"mean":23457605,"min":23457605,"q1":23457605,"q2":23457605,"q3":23457605,"max":23457605,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"},"writes":{"mean":1734579337,"min":1734579337,"q1":1734579337,"q2":1734579337,"q3":1734579337,"max":1734579337,"minLabel":"file_chopper (megahit)","maxLabel":"file_chopper (megahit)","q1Label":"file_chopper (megahit)","q2Label":"file_chopper (megahit)","q3Label":"file_chopper (megahit)"}},{"cpuUsage":null,"process":"vcontact2","mem":null,"memUsage":null,"timeUsage":{"mean":8.82,"min":8.82,"q1":8.82,"q2":8.82,"q3":8.82,"max":8.82,"minLabel":"vcontact2 (megahit)","maxLabel":"vcontact2 (megahit)","q1Label":"vcontact2 (megahit)","q2Label":"vcontact2 (megahit)","q3Label":"vcontact2 (megahit)"},"vmem":null,"reads":null,"cpu":null,"time":{"mean":7300238,"min":7300238,"q1":7300238,"q2":7300238,"q3":7300238,"max":7300238,"minLabel":"vcontact2 (megahit)","maxLabel":"vcontact2 (megahit)","q1Label":"vcontact2 (megahit)","q2Label":"vcontact2 (megahit)","q3Label":"vcontact2 (megahit)"},"writes":null},{"cpuUsage":null,"process":"covtocounts2","mem":null,"memUsage":null,"timeUsage":null,"vmem":null,"reads":null,"cpu":null,"time":null,"writes":null}]
Sep.-21 11:15:20.029 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Sep.-21 11:15:20.138 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
Hi,
I encountered an error with heatmap(megahit) step of MetaPhage v2 beta.
Below is the error message.
-----
Error executing process > 'heatmap (megahit)'
Caused by:
Process heatmap (megahit) terminated with an error exit status (1)
Command executed:
Rscript /home/kangin/Programs/MetaPhage.V2/bin/Rscript/heatmap.R count_table.csv taxonomy_table.csv metadata.csv Sample
Command exit status:
1
Command output:
[1] "Closer" "Accession" "Status" "VC"
[5] "Level" "Weight" "Host" "BaltimoreGroup"
[9] "Realm" "Kingdom" "Phylum" "Class"
[13] "Order" "Family" "Subfamily" "Genus"
Command error:
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'as.data.frame': incorrect number of dimensions
Calls: as.data.frame -> .handleSimpleError -> h
Execution halted
Work dir:
/home/kangin/Programs/MetaPhage.V2/work/43/e676af10556ccd47e3f57f0541b464
Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out
-----
I analyzed only two samples (viromes), and the metadata file contained only one column named "Sample".
Is this error due to too small number of samples or metadata fields, or due to a bug?
Thank you.
Hi,
I encountered an error likely at the graphanalyzer step.
The excerpt of the log file is as below.
How can I avoid this error?
According to the error message, I may be able to try setting "low_memory=False". But, I don't know how to do it.
Thanks.
Dec-08 02:18:15.086 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 682; name: vcontact2 (megahit); status: COMPLETED; exit: 0; error: -; workDir: /home/johy/Virome/MetaPhage/work/e8/5b76587e3dad3cfc3f66e2241cf432]
Dec-08 02:18:15.113 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Dec-08 02:18:15.114 [Task submitter] INFO nextflow.Session - [4a/c0e4fb] Submitted process > graphanalyzer (megahit)
Dec-08 02:22:13.345 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:27:13.400 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:32:13.458 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:37:13.512 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:42:13.575 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:47:13.639 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:52:13.695 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 02:57:13.751 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:02:13.803 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:07:13.855 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:12:13.906 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:17:13.962 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:22:13.993 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:27:14.045 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:32:14.099 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:37:14.154 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:42:14.210 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:47:14.272 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:52:14.336 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 03:57:14.385 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:02:14.439 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:07:14.480 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:12:14.525 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:18:33.782 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:25:33.114 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:30:42.370 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:35:59.280 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below
~> TaskHandler[id: 686; name: graphanalyzer (megahit); status: RUNNING; exit: -; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:36:13.864 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 686; name: graphanalyzer (megahit); status: COMPLETED; exit: 1; error: -; workDir: /home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4]
Dec-08 04:36:14.458 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'graphanalyzer (megahit)'
Caused by:
Process graphanalyzer (megahit) terminated with an error exit status (1)
Command executed:
python /home/johy/Virome/MetaPhage/bin/python/graphanalyzer.py --threads 60 --graph c1.ntw --csv genome_by_genome_overview.csv --metas /home/johy/Virome/MetaPhage/db/inphared/*data_excluding_refseq.tsv --prefix vOTU --output ./ --suffix megahit
cp custom_taxonomy_table_megahit_mqc.txt custom_taxonomy_table_mqc.txt
python /home/johy/Virome/MetaPhage/bin/python/taxonomy_table_namer.py
cp custom_taxonomy_table_mqc.txt custom_taxonomy_table_megahit_mqc.csv
sed -i 1,5d custom_taxonomy_table_megahit_mqc.csv
mv custom_taxonomy_table_megahit_mqc.csv taxonomy_table.csv
rm custom_taxonomy_table_megahit_mqc.txt
rm custom_taxonomy_table_mqc.txt
Command exit status:
1
Command output:
Completed subgraph 1981/21876 (vOTU_12161)
Completed subgraph 1982/21876 (vOTU_12409)
Completed subgraph 1983/21876 (vOTU_12285)
Completed subgraph 1984/21876 (vOTU_12435)
Completed subgraph 1985/21876 (vOTU_12437)
Completed subgraph 1986/21876 (vOTU_12440)
Completed subgraph 1987/21876 (vOTU_12414)
Completed subgraph 1988/21876 (vOTU_12442)
Completed subgraph 1989/21876 (vOTU_1238)
Completed subgraph 1990/21876 (vOTU_12353)
Completed subgraph 1991/21876 (vOTU_12443)
Completed subgraph 1992/21876 (vOTU_12322)
Completed subgraph 1993/21876 (vOTU_12428)
Completed subgraph 1994/21876 (vOTU_12448)
Completed subgraph 1995/21876 (vOTU_12431)
Completed subgraph 1996/21876 (vOTU_12438)
Completed subgraph 1997/21876 (vOTU_12445)
Completed subgraph 1998/21876 (vOTU_12454)
Completed subgraph 1999/21876 (vOTU_12452)
Completed subgraph 2000/21876 (vOTU_12447)
Completed subgraph 2001/21876 (vOTU_12458)
Completed subgraph 2002/21876 (vOTU_12397)
Completed subgraph 2003/21876 (vOTU_1240)
Completed subgraph 2004/21876 (vOTU_12123)
Completed subgraph 2005/21876 (vOTU_12398)
Completed subgraph 2006/21876 (vOTU_12467)
Completed subgraph 2007/21876 (vOTU_1187)
Completed subgraph 2008/21876 (vOTU_12401)
Completed subgraph 2009/21876 (vOTU_12371)
Completed subgraph 2010/21876 (vOTU_12457)
Completed subgraph 2011/21876 (vOTU_12474)
Completed subgraph 2012/21876 (vOTU_12477)
Completed subgraph 2013/21876 (vOTU_12380)
Completed subgraph 2014/21876 (vOTU_12462)
Completed subgraph 2015/21876 (vOTU_12479)
Completed subgraph 2016/21876 (vOTU_12471)
Completed subgraph 2017/21876 (vOTU_12455)
Completed subgraph 2018/21876 (vOTU_1200)
Completed subgraph 2019/21876 (vOTU_12483)
Completed subgraph 2020/21876 (vOTU_12468)
Completed subgraph 2021/21876 (vOTU_12488)
Completed subgraph 2022/21876 (vOTU_12490)
Completed subgraph 2023/21876 (vOTU_12449)
Completed subgraph 2024/21876 (vOTU_12494)
Completed subgraph 2025/21876 (vOTU_12433)
Completed subgraph 2026/21876 (vOTU_12485)
Completed subgraph 2027/21876 (vOTU_12481)
Completed subgraph 2028/21876 (vOTU_1241)
Completed subgraph 2029/21876 (vOTU_12497)
Completed subgraph 2030/21876 (vOTU_12432)
Command error:
sys:1: DtypeWarning: Columns (5,8) have mixed types. Specify dtype option on import or set low_memory=False.
Traceback (most recent call last):
File "/home/johy/Virome/MetaPhage/bin/python/graphanalyzer.py", line 1105, in
subgraphCreator(graph, csv_edit, df_results, parameters.output, parameters.suffix, max_weight, parameters.prefix, parameters.threads)
File "/home/johy/Virome/MetaPhage/bin/python/graphanalyzer.py", line 986, in subgraphCreator
scaffold = future.result() # blocking call: wait for this task to completed
File "/home/johy/miniconda3/envs/metaphage2/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/home/johy/miniconda3/envs/metaphage2/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
Work dir:
/home/johy/Virome/MetaPhage/work/4a/c0e4fbbd73533920c5fc5905cc5be4
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh
Dec-08 04:36:14.498 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process graphanalyzer (megahit) terminated with an error exit status (1)
Dec-08 04:36:14.527 [Actor Thread 344] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process phylo_obj -- offending value: []
Dec-08 04:36:14.604 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
Hi,
Thank you very much for making this great pipeline.
I will start using this pipeline from today with my shotgun metagenome samples.
But I also viral contigs generated using Phamb (https://github.com/RasmussenLab/phamb) from the same samples.
Is there any way I can add these contigs in the dereplication stage of MetaPhage?
Regards,
Bhim
Preparing to download vConTACT2
Error downloading https://s3.climb.ac.uk/ifrqmra-metaphage/v1.0/2022-01-inphared.tar.gz:
HTTP Error 404: Not Found
Thank you!
Hello,
I have sucessfully installed and run metaphage using the test dataset. When I run on my samples I get an error:
Error executing process > 'kraken_file (Creating the table...)'
Caused by:
Process `kraken_file (Creating the table...)` terminated with an error exit status (1)
Command executed:
Rscript /nfs3_ib/ip29-ib/ip29/fortierlc_group/programs/MetaPhage/bin/Rscript/kraken_files.R /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis
/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3 metadata.csv
Command exit status:
1
Command output:
Create DT version of the data.frame with kraken AND krona files
Command error:
source: /opt/software/conda_env/etc/conda/activate.d/activate-binutils_linux-64.sh:8:18: parameter expansion requires a literal
source: /opt/software/conda_env/etc/conda/activate.d/activate-gcc_linux-64.sh:8:18: parameter expansion requires a literal
source: /opt/software/conda_env/etc/conda/activate.d/activate-gfortran_linux-64.sh:8:18: parameter expansion requires a literal
source: /opt/software/conda_env/etc/conda/activate.d/activate-gxx_linux-64.sh:8:18: parameter expansion requires a literal
Failed to create bus connection: No such file or directory
Warning message:
In system("timedatectl", intern = TRUE) :
running command 'timedatectl' had status 1
Error: Problem with `filter()` input `..1`.
i Input `..1` is `matching_kronas %in% matching_kronas`.
x Input `..1` must be of size 8 or 1, not size 6.
Backtrace:
x
1. +-dplyr::filter(df_krona_fp, matching_kronas %in% matching_kronas)
2. +-dplyr:::filter.data.frame(df_krona_fp, matching_kronas %in% matching_kronas)
3. | \-dplyr:::filter_rows(.data, ..., caller_env = caller_env())
4. | +-base::withCallingHandlers(...)
5. | \-mask$eval_all_filter(dots, env_filter)
6. +-dplyr:::abort_glue(...)
7. | +-rlang::exec(abort, message = message, class = class, !!!data)
8. | \-(function (message = NULL, class = NULL, ..., trace = NULL, parent = NULL, ...
9. | \-rlang:::signal_abort(cnd)
10. | \-base::signalCondition(cnd)
11. \-(function (e) ...
Execution halted
Work dir:
/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/work/a3/bb376f28623fefb73ebabc60733740
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
To further debug, I manually ran the R script and pinpoint the line that fails:
(MetaPhage) |16:24:56|jflucier@ip29:[2022_metaphage_reanalysis]> R
R version 4.1.1 (2021-08-10) -- "Kick Things"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(DT)
> library(readr)
> library(dplyr)
Attaching package: βdplyrβ
The following objects are masked from βpackage:statsβ:
filter, lag
The following objects are masked from βpackage:baseβ:
intersect, setdiff, setequal, union
> library(plyr)
------------------------------------------------------------------------------
You have loaded plyr after dplyr - this is likely to cause problems.
If you need functions from both plyr and dplyr, please load plyr first, then dplyr:
library(plyr); library(dplyr)
------------------------------------------------------------------------------
Attaching package: βplyrβ
The following objects are masked from βpackage:dplyrβ:
arrange, count, desc, failwith, id, mutate, rename, summarise,
summarize
> library(seqinr)
Attaching package: βseqinrβ
The following object is masked from βpackage:plyrβ:
count
The following object is masked from βpackage:dplyrβ:
count
> library(tidyverse)
ββ Attaching packages βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ tidyverse 1.3.1 ββ
β ggplot2 3.3.5 β purrr 0.3.4
β tibble 3.1.3 β stringr 1.4.0
β tidyr 1.1.4 β forcats 0.5.1
ββ Conflicts ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ tidyverse_conflicts() ββ
β plyr::arrange() masks dplyr::arrange()
β purrr::compact() masks plyr::compact()
β seqinr::count() masks plyr::count(), dplyr::count()
β plyr::failwith() masks dplyr::failwith()
β dplyr::filter() masks stats::filter()
β plyr::id() masks dplyr::id()
β dplyr::lag() masks stats::lag()
β plyr::mutate() masks dplyr::mutate()
β plyr::rename() masks dplyr::rename()
β plyr::summarise() masks dplyr::summarise()
β plyr::summarize() masks dplyr::summarize()
> library(gtools)
> library(magrittr)
Attaching package: βmagrittrβ
The following object is masked from βpackage:purrrβ:
set_names
The following object is masked from βpackage:tidyrβ:
extract
> file_paths <- c("/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3")
> kraken_path <- file.path(file_paths, "taxonomy/kraken2")
> kraken_path
[1] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/kraken2"
> if (! file.exists(kraken_path)){
stop("ERROR: taxonomy/kraken2 folder not found in: ", kraken_path, "\n")
}
> krona_path <- file.path(file_paths, "taxonomy/krona")
> if (! file.exists(krona_path)){
cat("WARNING: taxonomy/krona folder not found in: ", krona_path, "\n")
}
> file_meta <- c("/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/work/a3/bb376f28623fefb73ebabc60733740/metadata.c
sv")
> if (! file.exists(file_meta)){
stop("ERROR: metadata file not found in: ", file_meta, "\n")
}
> metadata <- read.delim(file_meta, row.names = 1, sep = ",", check.names = F)
metadata$Sample <- rownames(metadata)
metadata <- metadata %>%
select(Sample, everything())
> metadata
Sample Projet DaysPostInd Group MouseID Specie
VMC11 VMC11 202001_mouse_virome3 D0 EXP-D0 740-D0 Mouse
VMC16 VMC16 202001_mouse_virome3 D0 EXP-D0 792-D0 Mouse
VMC18 VMC18 202001_mouse_virome3 D0 EXP-D0 794-D0 Mouse
VMC19 VMC19 202001_mouse_virome3 D21 EXP-D21 740-D21 Mouse
VMC24 VMC24 202001_mouse_virome3 D21 EXP-D21 792-D21 Mouse
VMC26 VMC26 202001_mouse_virome3 D21 EXP-D21 794-D21 Mouse
> krakens_path <- list.files(path = kraken_path,
pattern = "_report.txt",
recursive = TRUE,
full.names=T)
> df_kraken_fp <- as.data.frame(krakens_path)
> krakens_names = unique(basename(krakens_path))
krakens_names <- sapply(strsplit(krakens_names,"_report.txt"), `[`, 1)
matching_krakens = krakens_names[krakens_names %in% metadata$Sample]
> matching_krakens
[1] "VMC11" "VMC16" "VMC18" "VMC19" "VMC24" "VMC26"
> krakens_names_fp = as.data.frame(
grep(paste(matching_krakens,collapse="|"), df_kraken_fp$krakens_path, value=TRUE)
)
> colnames(krakens_names_fp) <- c("krakens_path")
> krakens_fp = paste0("../", str_extract(krakens_names_fp$krakens_path,
"taxonomy/kraken2/.+/.+_report.txt"))
> df_krakens = tibble("Sample" = matching_krakens, file = krakens_fp) %>%
mutate(file = str_replace_all(file,
'([^;]*)taxonomy/kraken2/.+/', ''),
path_krakens = file.path(krakens_fp),
Kraken_Report = paste0('<a target=_blank href=',
path_krakens, '>', file,'</a>'))
df_krakens$file <- NULL
df_krakens$path_krakens <- NULL
> if(file.exists(krona_path)){
cat("Create DT version of the data.frame with kraken AND krona files\n")
# read the krona files and create a data.frame of paths
kronas_path <- list.files(path = krona_path,
".+_krak_krona_abundancies.html",
recursive = TRUE,
full.names=T)
+
+
> message("allo")
allo
> if(file.exists(krona_path)){
message("krona")
+ } else {
+ message("no krona")
+ }
krona
> kronas_path <- list.files(path = krona_path,
".+_krak_krona_abundancies.html",
recursive = TRUE,
full.names=T)
> kronas_path
[1] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC11/VMC11_krak_krona_abundancies.html"
[2] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC13/VMC13_krak_krona_abundancies.html"
[3] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC16/VMC16_krak_krona_abundancies.html"
[4] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC18/VMC18_krak_krona_abundancies.html"
[5] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC19/VMC19_krak_krona_abundancies.html"
[6] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC21/VMC21_krak_krona_abundancies.html"
[7] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC24/VMC
24_krak_krona_abundancies.html"
[8] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC26/VMC
26_krak_krona_abundancies.html"
> df_krona_fp <- as.data.frame(kronas_path)
> kronas_path
[1] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC11/VMC
11_krak_krona_abundancies.html"
[2] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC13/VMC
13_krak_krona_abundancies.html"
[3] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC16/VMC
16_krak_krona_abundancies.html"
[4] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC18/VMC
18_krak_krona_abundancies.html"
[5] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC19/VMC
19_krak_krona_abundancies.html"
[6] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC21/VMC
21_krak_krona_abundancies.html"
[7] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC24/VMC
24_krak_krona_abundancies.html"
[8] "/nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC26/VMC
26_krak_krona_abundancies.html"
> kronas_names = unique(basename(kronas_path))
kronas_names
kronas_names <- sapply(strsplit(kronas_names,"_krak_krona_abundancies.html"), `[`, 1)
matching_kronas = kronas_names[kronas_names %in% metadata$Sample]
[1] "VMC11_krak_krona_abundancies.html" "VMC13_krak_krona_abundancies.html"
[3] "VMC16_krak_krona_abundancies.html" "VMC18_krak_krona_abundancies.html"
[5] "VMC19_krak_krona_abundancies.html" "VMC21_krak_krona_abundancies.html"
[7] "VMC24_krak_krona_abundancies.html" "VMC26_krak_krona_abundancies.html"
> kronas_names_fp <- filter(df_krona_fp,
matching_kronas %in% matching_kronas)
Error: Problem with `filter()` input `..1`.
βΉ Input `..1` is `matching_kronas %in% matching_kronas`.
β Input `..1` must be of size 8 or 1, not size 6.
Run `rlang::last_error()` to see where the error occurred.
> matching_kronas
[1] "VMC11" "VMC16" "VMC18" "VMC19" "VMC24" "VMC26"
> df_krona_fp
kronas_path
1 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC11/VMC11_krak_krona_abundancies.html
2 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC13/VMC13_krak_krona_abundancies.html
3 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC16/VMC16_krak_krona_abundancies.html
4 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC18/VMC18_krak_krona_abundancies.html
5 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC19/VMC19_krak_krona_abundancies.html
6 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC21/VMC21_krak_krona_abundancies.html
7 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC24/VMC24_krak_krona_abundancies.html
8 /nfs3_ib/ip29-ib/ip29/fortierlc_group/analysis/2022_metaphage_reanalysis/MetaPhage-output/202001_mouse_virome3/taxonomy/krona/VMC26/VMC26_krak_krona_abundancies.html
My metadata file is the following:
Sample,Projet,DaysPostInd,Group,MouseID,Specie
VMC11,202001_mouse_virome3,D0,EXP-D0,740-D0,Mouse
VMC16,202001_mouse_virome3,D0,EXP-D0,792-D0,Mouse
VMC18,202001_mouse_virome3,D0,EXP-D0,794-D0,Mouse
VMC19,202001_mouse_virome3,D21,EXP-D21,740-D21,Mouse
VMC24,202001_mouse_virome3,D21,EXP-D21,792-D21,Mouse
VMC26,202001_mouse_virome3,D21,EXP-D21,794-D21,Mouse
Thank for your help in advance
JF
Hello,
Using this command to download databases for the beta version of MetaPhage 2,
./bin/python/db_manager.py -o ./db/ -m 6 -r 2022.1
throws this error:
db_manager.py: error: unrecognized arguments: -r 2022.1
Can you please check?
Hi,
As I explained in another issue #72, I'm experiencing an error at the graphanlyzer step.
So, I tried skipping graphanalyzer step by editing the "nextflow.config" file as follows.
skip_graphanalyzer = true
But, this seems to lead other errors at the next steps. Below is the excerpt of the log file.
Dec-15 13:27:38.173 [Actor Thread 186] INFO nextflow.processor.TaskProcessor - [a5/a121c7] Cached process > bowtie2_derep (megahit)
Dec-15 13:27:38.184 [Actor Thread 195] INFO nextflow.processor.TaskProcessor - [02/9358f0] Cached process > checkV (megahit)
Dec-15 13:27:38.188 [Actor Thread 186] INFO nextflow.processor.TaskProcessor - [1a/0e52c2] Cached process > covtocounts2 (megahit)
Dec-15 13:27:38.188 [Actor Thread 100] INFO nextflow.processor.TaskProcessor - [03/70d651] Cached process > checkv_table (Creating checkV table...)
Dec-15 13:27:38.205 [Actor Thread 121] INFO nextflow.processor.TaskProcessor - [98/de8b31] Cached process > diamond_vcontact2 (megahit)
Dec-15 13:27:38.206 [Actor Thread 239] INFO nextflow.processor.TaskProcessor - [a3/06c0e5] Cached process > summary (megahit)
Dec-15 13:27:38.213 [Actor Thread 241] INFO nextflow.processor.TaskProcessor - [e8/5b7658] Cached process > vcontact2 (megahit)
Dec-15 13:27:38.220 [Actor Thread 187] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process `heatmap` -- offending value: []
Dec-15 13:27:38.224 [Actor Thread 112] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process `alpha_diversity` -- offending value: []
Dec-15 13:27:38.224 [Actor Thread 110] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process `phylo_obj` -- offending value: []
Dec-15 13:27:38.224 [Actor Thread 141] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process `taxonomy_table` -- offending value: []
Dec-15 13:27:38.224 [Actor Thread 241] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process `violin_plots` -- offending value: []
Dec-15 13:27:38.251 [Actor Thread 141] ERROR nextflow.processor.TaskProcessor - Error executing process > 'taxonomy_table (null)'
Caused by:
Path value cannot be null
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Dec-15 13:27:38.254 [Actor Thread 141] DEBUG nextflow.Session - Session aborted -- Cause: Path value cannot be null
Dec-15 13:27:38.282 [Actor Thread 141] DEBUG nextflow.Session - The following nodes are still active:
[process] file_chopper
Are these errors expected when graphanalyzer step was skipped?
Thanks.
Hello, I have two questions.
Is there a way to normalize the counts to be at RPKM? can we use the vOTU Length reported in megahit_min_comp.csv for this normalization?
Can you implement Bracken (https://ccb.jhu.edu/software/bracken/) to compute and correct the abundances reported by Kraken?
Thank you!
Hi!
I'm trying to run MetaPhage in my local system (Ubuntu 18.04).
I created a config file named "local.config" and ran Metaphage with the following command with demo data.
$ nextflow run main.nf -c local.config -c demo.conf
Below is the contents of the "local.config".
------
params {
max_cpus = 48
max_memory = 384.GB
max_time = 240.h
}
process {
executor='local'
}
-----
I expected that the run would use less than 48 CPUs, but it seems that nearly all CPUs available in my system (80 CPUs) are used by the run.
This is the first time I'm using nextflow. It is difficult for me to find how to limit the number of CPUs.
Are there any ways to limit the maximum number of CPUs used by MetaPhage?
Thanks.
I am currently trying to run the demo with MetaPhage-0.3.3, I ran into a problem when executing the pipeline
Step 5 : /home/jm2338/miniconda3/envs/metaphage/bin/Scripts/Step_5_get_phage_fasta-gb.pl VIRSorter SRR8653090_virsorter >> SRR8653090_virsorter/logs/out 2>> SRR8653090_virsorter/logs/err
## Verify if this should have been a virome decontamination mode based on 10kb+ contigs
Cleaning the output directory
rm -r SRR8653090_virsorter/r_0/db :
Command error:
mv: cannot stat '*': No such file or directory
Work dir:
/local/workdir/jm2338/MetaPhage-0.3.3/work/04/a7441cd0e8ab03e5e303e900777cb1
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
I tried debugging it with the steps mentioned in a another post because it is the same file that failed, #57 , but I wasn't able to do the step re-run the virsorter2 download and setup
virsorter setup --db-dir "$METAPHAGE_DIR"/db/virsorter/virsorter2 --jobs 4
I got this error
[2023-11-12 15:43 INFO] VirSorter 2.2.4
[2023-11-12 15:43 INFO] /home/jm2338/miniconda3/envs/metaphage/bin/virsorter setup --db-dir /db/virsorter/virsorter2 --jobs 4
[2023-11-12 15:43 INFO] Setting up VirSorter2 database; this might take ~10 mins and only needs to be done once.
Traceback (most recent call last):
File "/home/jm2338/miniconda3/envs/metaphage/bin/snakemake", line 10, in <module>
sys.exit(main())
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/site-packages/snakemake/__init__.py", line 2574, in main
log_handler=log_handler,
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/site-packages/snakemake/__init__.py", line 493, in snakemake
os.makedirs(workdir)
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/db'
[2023-11-12 15:43 INFO] First attempt failed; trying the second time.
Traceback (most recent call last):
File "/home/jm2338/miniconda3/envs/metaphage/bin/snakemake", line 10, in <module>
sys.exit(main())
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/site-packages/snakemake/__init__.py", line 2574, in main
log_handler=log_handler,
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/site-packages/snakemake/__init__.py", line 493, in snakemake
os.makedirs(workdir)
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/jm2338/miniconda3/envs/metaphage/lib/python3.7/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/db'
[2023-11-12 15:43 CRITICAL] Command 'snakemake --snakefile /home/jm2338/miniconda3/envs/metaphage/lib/python3.7/site-packages/virsorter/rules/setup-retry.smk --directory /db/virsorter/virsorter2 --quiet --config Skip_deps_install=False --jobs 4 --rerun-incomplete --latency-wait 600 --nolock --use-conda --conda-prefix /db/virsorter/virsorter2/conda_envs --conda-frontend mamba ' returned non-zero exit status 1.
Then when I was trying to run the pipeline again, I got this error
Error executing process > 'virsorter (megahit-SRR8653090)'
Caused by:
Process `virsorter (megahit-SRR8653090)` terminated with an error exit status (127)
Command executed:
wrapper_phage_contigs_sorter_iPlant.pl -f SRR8653090_megahit_contigs.fasta --db 2 --wdir SRR8653090_virsorter --ncpu 16 --data-dir /local/workdir/jm2338/MetaPhage-0.3.3/db/virsorter/legacy/
cd SRR8653090_virsorter/Predicted_viral_sequences/
for FILENAME in *;
do
mv $FILENAME SRR8653090_$FILENAME;
if [ ! -e SRR8653090_VIRSorter_cat-1.fasta ]; then
touch SRR8653090_VIRSorter_cat-1.fasta
elif [ ! -e SRR8653090_VIRSorter_prophages_cat-4.fasta ]; then
touch SRR8653090_VIRSorter_prophages_cat-4.fasta
else
break
fi
done
# add a space after sequenceID in header (before flag)
sed 's/_flag=/ flag=/' SRR8653090_VIRSorter_cat-1.fasta > SRR8653090_correct_virsorter_cat-1.fasta;
sed 's/_flag=/ flag=/' SRR8653090_VIRSorter_prophages_cat-4.fasta > SRR8653090_correct_virsorter_prophages_cat-4.fasta
Command exit status:
127
Command output:
(empty)
Command error:
.command.sh: line 2: wrapper_phage_contigs_sorter_iPlant.pl: command not found
Work dir:
/local/workdir/jm2338/MetaPhage-0.3.3/work/1a/b3b6ad4dcf2854af9d83d4e0c5efe5
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
How should I fix this?
Hi,
I'm experiencing an out of memory error at the Qualimap bamqc step.
Below is the error message.
======
Command error:
Building a SMALL index
59516449 reads; of these:
59516449 (100.00%) were paired; of these:
9508796 (15.98%) aligned concordantly 0 times
11383356 (19.13%) aligned concordantly exactly 1 time
38624297 (64.90%) aligned concordantly >1 times
----
9508796 pairs aligned concordantly 0 times; of these:
500477 (5.26%) aligned discordantly 1 time
----
9008319 pairs aligned 0 times concordantly or discordantly; of these:
18016638 mates make up the pairs; of these:
11278938 (62.60%) aligned 0 times
528133 (2.93%) aligned exactly 1 time
6209567 (34.47%) aligned >1 times
90.52% overall alignment rate
[bam_sort_core] merging from 48 files and 16 in-memory blocks...
WARNING: out of memory!
Qualimap allows to set RAM size using special argument: --java-mem-size
Check more details using --help command or read the manual.
======
Hinted at the error message, I edited Qualimap part of "main.nf" file as follows. I added "--java-mem-size=24G" to the file.
======
qualimap bamqc
-nt ${task.cpus}
-outdir qualimap_bamqc_${seqID}${consensus}.folder
-bam ${seqID}${consensus}.sorted.bam
--java-mem-size=24G
"""
======
But, when I resumed the run with the edited main.nf file, I encountered another error. The error message is as below.
======
Command error:
Building a SMALL index
12889292 reads; of these:
12889292 (100.00%) were paired; of these:
7319646 (56.79%) aligned concordantly 0 times
1388181 (10.77%) aligned concordantly exactly 1 time
4181465 (32.44%) aligned concordantly >1 times
----
7319646 pairs aligned concordantly 0 times; of these:
29667 (0.41%) aligned discordantly 1 time
----
7289979 pairs aligned 0 times concordantly or discordantly; of these:
14579958 mates make up the pairs; of these:
13283039 (91.10%) aligned 0 times
179847 (1.23%) aligned exactly 1 time
1117072 (7.66%) aligned >1 times
48.47% overall alignment rate
[bam_sort_core] merging from 0 files and 16 in-memory blocks...
.command.sh: line 9: --java-mem-size=24G: command not found
======
It seems that the addition of "--java-mem-size=24G" has caused this error.
How can I solve this problem?
Thanks.
Hello, and thank you for developing MetaPhage,
Just wanted to point out that all the links to download the databases hosted on the s3.climb.ac.uk are not working and makes it impossible to install the MetaPhage2 version.
For MetaPhage1, we can still use the zenodo link you provided for Inphared, but all the links are not present on zenedo for the new version of metaphage.
Looking forward to install and test MetaPhage2!
Warm regards, Julian
Hi I have access to a 16cpu 64 GB RAM node. There is a guide to change the CPU and RAM limits but I do't know where to add this to. If adding to the main.nf file I get syntax errors for the (process { ..) not having a identifier. I I make up an identifier then I get a withLabel BigRes error. I would appreciate any help on the matter
Hi,
Very nice tool that I'm excited to try.
However, would it be possible to add an option in case I already have an assembly file and don't wish to redo the assembly?
Best
Greg
e.g. lower the requirement fastp -w 16 β are there other steps that require 16 cores?
Lowering the requirement will make it easier to run on slower machines (though of course it'll way longer but I'm ok with that). Or are there software limitations that require 16 cores?
Just adding that this param configuration doesn't seem to lower it:
// Define the maximum resources
params {
max_cpus = 8
max_memory = 60.GB
max_time = 72.h
}
Edit #2: changing the values in nextflow.config process { withLabel: { ... } } and params { max_cpus = # } fixed it for me! Ok to close.
Please update docs to make it apparently that's where you should edit (I was editing the demo.conf file), for those new to Nextflow
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google β€οΈ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.