Comments (7)
What's the size of your dataset? If it is not too large, I guess it should be fine.
Dear hifiasm team
I'm wondering how much memory the package needs for a read dataset of 30x coverage from a single human chromosome?
from hifiasm.
Thanks for fast reply. The fq.gz file is about one Gigabyte. It seems that I can not run it on my own desktop with 16 GB ram.
from hifiasm.
Could you give me the log information? I think one Gigabyte should be fine...
from hifiasm.
After 7 hours the terminal is empty. It seems that there is no log. Is it producing log in a file? The desktop becomes very slow. The program only uses 1% of one cpu and almost all of the memory.
I've also tried with small example of simulated read dataset (70 Mbyte) from small reference genome. It finished but the gfa files are empty. The following is the log for the small example. Note the last line of the log.
[M::ha_analyze_count] lowest: count[3] = 1547
[M::ha_analyze_count] highest: count[4] = 12872
[M::ha_hist_line] 2: ****************************************************************************************************> 46760342
[M::ha_hist_line] 3: ************ 1547
[M::ha_hist_line] 4: **************************************************************************************************** 12872
[M::ha_hist_line] 5: ** 268
[M::ha_hist_line] 6: ************ 1512
[M::ha_hist_line] 7: * 118
[M::ha_hist_line] 8: **** 544
[M::ha_hist_line] 9: * 90
[M::ha_hist_line] 10: ** 241
[M::ha_hist_line] rest: ***** 624
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: count[6] = 1512
[M::ha_ft_gen] peak_hom: 6; peak_het: 4
[M::ha_ft_gen::9480.725*[email protected]] ==> filtered out 2 k-mers occurring 30 or more times
[M::ha_opt_update_cov] updated max_n_chain to 100
[M::ha_pt_gen::9493.968*0.03] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9503.243*0.03] ==> indexed 3529370 positions
[M::ha_assemble::9514.903*[email protected]] ==> corrected reads for round 1
[M::ha_assemble] # bases: 149998500; # corrected bases: 0; # recorrected bases: 0
[M::ha_assemble] size of buffer: 0.005GB
[M::ha_pt_gen::9521.783*0.03] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9528.960*0.03] ==> indexed 3529370 positions
[M::ha_assemble::9540.653*[email protected]] ==> corrected reads for round 2
[M::ha_assemble] # bases: 149998500; # corrected bases: 0; # recorrected bases: 0
[M::ha_assemble] size of buffer: 0.005GB
[M::ha_pt_gen::9547.512*0.03] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9554.677*0.03] ==> indexed 3529370 positions
[M::ha_assemble::9566.112*[email protected]] ==> corrected reads for round 3
[M::ha_assemble] # bases: 149998500; # corrected bases: 0; # recorrected bases: 0
[M::ha_assemble] size of buffer: 0.005GB
[M::ha_pt_gen::9572.965*0.04] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9580.119*0.04] ==> indexed 3529370 positions
[M::ha_assemble::9588.611*[email protected]] ==> found overlaps for the final round
[M::ha_print_ovlp_stat] # overlaps: 50264
[M::ha_print_ovlp_stat] # strong overlaps: 0
[M::ha_print_ovlp_stat] # weak overlaps: 50264
[M::ha_print_ovlp_stat] # exact overlaps: 8820
[M::ha_print_ovlp_stat] # inexact overlaps: 41444
[M::ha_print_ovlp_stat] # overlaps without large indels: 50264
[M::ha_print_ovlp_stat] # reverse overlaps: 0
Writing reads to disk...
Reads has been written.
Writing ma_hit_ts to disk...
ma_hit_ts has been written.
Writing ma_hit_ts to disk...
ma_hit_ts has been written.
Writing raw unitig GFA to disk...
Writing processed unitig GFA to disk...
Writing primary contig GFA to disk...
Writing alternate contig GFA to disk...
[M::main] Version: 0.5-dirty-r246
[M::main] CMD: /home/ssm/Documents/software/hifiasm/hifiasm -o asm/out reads/pacbio.fastq.gz
[M::main] Real time: 9588.930 sec; CPU: 353.453 sec; Peak RSS: 15.123 GB
from hifiasm.
I see. For 70MB file, its coverage might be just 1 or 2, which is too small for assembler. Thus, hifiasm outputted an empty gfa. As for the 1G file, hifiasm looks like out of memory. Maybe you can try v0.3 (https://github.com/chhylp123/hifiasm/releases/tag/v0.3), its memory requirement should be lower.
from hifiasm.
For small datasets, apply -f0
to disable bloom filter, which takes 16GB by default. Your coverage is too low for hifiasm to work by the way.
from hifiasm.
Thanks for the information.
from hifiasm.
Related Issues (20)
- Possible missing one haplotype in human assemblies HOT 2
- No haploid.gfa files output in trio-binning mode HOT 3
- Hifi + Hi-c + ONT assembly fails
- In Trio-binning, always more on hap1 despite (almost) same sequences for paternal and maternal
- discontinuous assembly with shorter pacbio hifi reads but high coverage HOT 2
- Is x20 of Hifi data enough to construct draft assembly of 6.5Gb genome? HOT 1
- line 8: 110334 Aborted(core dumped) HOT 1
- Ultra Long intergration failed: no output for UL kmer counting HOT 3
- missing 8Mb sequences in the assembly HOT 5
- Empty haplotype 2 gfa files by ONT integration HOT 1
- Basic Question About HiFi Input HOT 3
- Spend too long times to run hifiasm HOT 1
- Switch error on X and Y chromosome HOT 2
- *.ovlp.bin file HOT 1
- Resolving switching error (?)
- Interchromosomal misjoin HOT 1
- Read error correction does not reduce the number of kmers present once, twice or three times
- Recreate p_ctg from p_utg
- In the diploid assembly, hfiiasm identified a value that did not exist in the k-mer plot as the "homozygous read coverage threshold".
- fungi diploid assemble phasing errors
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hifiasm.