Comments (4)
I will try to do that. The mem_gb=3, is this just a rough guess? For example, I think SnpEff requires more than 3gb, but it might also depend on the vcf, doesn't it?
from snakemake-wrappers.
Another question: I see the benefits for dealing only with "gb", but it would also be possible to allow other units, like 512m. What if the user wants to set the used amount of ram to something smaller than 1gb? I don't think that 0.5gb is allowed (but I haven't tested this). So without thinking much about it, I would rather use "mem" instead of "mem_gb" and let the user choose the unit by himself.
ps.: Nevermind, snakemake would not be able to handle this. Stupid idea.
from snakemake-wrappers.
I have a suggestion. The metric unit prefixes T, G, M, k, h and da are independent of the unit itself. So it would be great, if snakemake would support them, so that one could write
snakemake -j 1h -mem 512m
which would then be translated internally to 100 and (512 * 1024). (probably noone would use "hecto" (h), but anyway). Tthis would also be useful for other resources. With the internally translation to the base unit - "bytes" in the case of mem - we would avoid any problems with small units.
from snakemake-wrappers.
The 3
was just to a number in there instead of some placeholder, so a different default for different tools makes sense. I think the most important point is getting a unified approach of treating memory resources in all Java tools whose bioconda packages allow specification of -Xmx
.
Generally, I would think that on most machines and in most settings, specifying integer values of Gigabytes should be good enough. Memory is usually available and only a limiting factor for some tools, so generous defaults would usually make sense, to avoid job failures due to limited JVM memory resources. But I do see the setting of some tool that runs on a single core but needs a lot of memory, so that when you run lots of those jobs in parallel, it does get limiting. The easiest way out would be to be able to specify float values for --resources mem_gb=0.5
, and with a quick search of the snakemake code I couldn't find any requirement for a resource to be int
. So this might just work---I guess a quick test whether snakemake complains would be a start?
Also, I do see the appeal of a clean solution allowing metric units for memory and introducing a separate memory resource command-line parameter, but I'm not sure if this is not over-engineering the problem for now. Maybe @johanneskoester can comment on the idea and if we all agree to pursue it, this problem would head over to the snakemake repo, first... :D
from snakemake-wrappers.
Related Issues (20)
- Sickle se/pe HOT 1
- The ensembl-annotation uses the wrong download URL HOT 3
- Error while testing bbduk HOT 1
- Salmon Quant wrapper throwing segmentation fault HOT 1
- MACS2: support --extsize with fragment size input file
- bwa-memx_index meme wrapper expects specific file name? HOT 3
- Possible bowtie2 align bug HOT 4
- Generic subfolder? HOT 1
- STAR wrapper allowing different output names HOT 3
- CI: autobump workflow never finishes HOT 2
- GFFREAD wrapper produces gff output when fasta file requested HOT 1
- building the docs on a fork doesn't work HOT 2
- FastQC resource allocation not working as expected - get_mem fix? HOT 3
- MultiQC wrapper env issue with imp HOT 1
- Wrappers for Google Batch HOT 3
- ModuleNotFoundError: No module named 'snakemake_wrapper_utils HOT 10
- `bcftools reheader` no longer works with map-style `sample.tsv`
- `bcftools merge`: not able to declare dependency on index files HOT 1
- `deepvariant`: conda environment not installable HOT 3
- change to earlier docs versions on readthedocs HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snakemake-wrappers.