Comments (2)
Concern: How long does this check take for "normal" or "average" sized input file?
Not a lot of workarounds for this; basically just checking size of a GZIP file --> Don't have tools to easily increase speed of this or would take a lot of work that's not worth it
Can do a quick benchmark test with a file to see how it takes; see if reasonable, then can merge PR
from seq-tools.
On behalf of @edsu7 :
Based on below, it should take ~15 minutes for 600 million reads.
NOTE: Units are in minutes
File | # Reads | Time Rep1 | Time Rep2 | Time Rep3 | Time Average |
---|---|---|---|---|---|
BIN67_BRG1_rep3_merged_R1.fastq.gz | 77,877,363 | 3.46 | 1.45 | 1.44 | 2.116666667 |
BIN67_BRG1_rep3_merged_R2.fastq.gz | 77,877,363 | 2 | 1.46 | 1.48 | 1.646666667 |
BIN67_CONTROL_rep2_merged_R1.fastq.gz | 86,832,404 | 2.13 | 1.59 | 2.7 | 2.14 |
BIN67_CONTROL_rep2_merged_R2.fastq.gz | 86,832,404 | 2.14 | 2 | 2.6 | 2.246666667 |
BIN67_CONTROL_rep4_merged_R1.fastq.gz | 88,900,467 | 2.21 | 2 | 1.58 | 1.93 |
BIN67_CONTROL_rep4_merged_R2.fastq.gz | 88,900,467 | 2.25 | 2.5 | 2.1 | 2.283333333 |
BIN67_BRG1_rep1_merged_R1.fastq.gz | 91,202,472 | 2.3 | 2.2 | 2.3 | 2.266666667 |
BIN67_BRG1_rep1_merged_R2.fastq.gz | 91,202,472 | 2.3 | 2.6 | 2.8 | 2.566666667 |
BIN67_CONTROL_rep3_merged_R1.fastq.gz | 91,507,599 | 2.14 | 2.2 | 2.2 | 2.18 |
BIN67_CONTROL_rep3_merged_R2.fastq.gz | 91,507,599 | 2.17 | 2.9 | 2.13 | 2.4 |
BIN67_BRG1_rep2_merged_R1.fastq.gz | 99,933,061 | 2.44 | 2.15 | 2.15 | 2.246666667 |
BIN67_BRG1_rep2_merged_R2.fastq.gz | 99,933,061 | 2.51 | 2.16 | 2.15 | 2.273333333 |
BIN67_CONTROL_rep1_merged_R1.fastq.gz | 102,966,303 | 2.39 | 2.2 | 2.19 | 2.26 |
BIN67_CONTROL_rep1_merged_R2.fastq.gz | 102,966,303 | 2.4 | 2.2 | 2.33 | 2.31 |
BIN67_BRG1_rep4_merged_R1.fastq.gz | 103,097,081 | 2.43 | 2.17 | 2.21 | 2.27 |
BIN67_BRG1_rep4_merged_R2.fastq.gz | 103,097,081 | 2.39 | 2.26 | 2.34 | 2.33 |
Aggregate | 1,484,633,500 | 34.7 | 33.35 | 33.33 | 33.79 |
from seq-tools.
Related Issues (20)
- New checker (c260): fileName pattern check HOT 1
- New check: repeated identical reads from different read groups HOT 2
- Feature Request: QA from seq-tools HOT 1
- Improvement: print short message to inform user the total number of metadata files to be checked HOT 1
- New check: for BAM submission, detect whether it's paired end or not and whether it matches what's in the metadata HOT 1
- New feature: add a flag to skip md5sum check HOT 1
- 🐛 Validation does not check empty string for required fields HOT 1
- 🐛 Failing c650 test due to empty SM HOT 3
- 🐛 Regex check not matching schema in c120_permissible_char_in_rg_id.py HOT 3
- New check: for RNA-Seq BAM submit data, detect the `library_strandedness` and check whether it matches what's in the metadata HOT 3
- Add README.md for Validation Checks in the Seq-Tools HOT 6
- Build New Docker Container for Seq-Tools for Quay.io HOT 1
- Update Seq-Tools Descriptions on ARGO Doc Site HOT 1
- Detect `RNA-seq strandedness` in FASTQ HOT 9
- Add expected running duration of each check for a given normal file size input
- Performance improvements for`seq-tools` HOT 2
- Temporarily disable certain tests HOT 1
- Fix `submitter_read_group_id` HOT 1
- Add skip for c140 to platform uniqueness
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from seq-tools.