Comments (1)
I should add that I installed MIDAS using python version 3.9.6, I have just noticed that when running the second part of the tutorial (snps) there are issues with the multiprocessing package that will give this error:
TypeError: cannot pickle '_io.TextIOWrapper' object
Any idea? What version of python do you support?
FYI, the error in context is:
/usr/bin/time -p -v run_midas.py snps midas_output/SAMPLE_1 -1 /u/local/apps/midas/EXAMPLE/ex
ample/sample_1.fq.gz -t 8
MIDAS: Metagenomic Intra-species Diversity Analysis System
version 1.3.0; github.com/snayfach/MIDAS
Copyright (C) 2015-2016 Stephen Nayfach
Freely distributed under the GNU General Public License (GPLv3)
===========Parameters===========
Command: /u/local/apps/midas/1.3.2/MIDAS/scripts/run_midas.py snps midas_output/SAMPLE_1 -1 /u/local/apps/midas/EXAMPLE/example/sample_1.fq.gz -t
8
Script: run_midas.py snps
Database: /u/local/apps/midas/DB/midas_db_v1.2
Output directory: midas_output/SAMPLE_1
Remove temporary files: False
Pipeline options:
build bowtie2 database of genomes
align reads to bowtie2 genome database
use samtools to generate pileups and count variants
Database options:
include all species with >=3.0X genome coverage
Read alignment options:
input reads (unpaired): /u/local/apps/midas/EXAMPLE/example/sample_1.fq.gz
alignment speed/sensitivity: very-sensitive
alignment mode: global
number of reads to use from input: use all
number of threads for database search: 8
SNP calling options:
minimum alignment percent identity: 94.0
minimum mapping quality score: 20
minimum base quality score: 30
minimum read quality score: 20
minimum alignment coverage of reads: 0.75
trim 0 base-pairs from 3'/right end of read
================================
Reading reference data
0.0 minutes
0.1 Gb maximum memory
Building database of representative genomes
total genomes: 1
total contigs: 1
total base-pairs: 5163189
0.04 minutes
0.26 Gb maximum memory
Mapping reads to representative genomes
finished aligning
checking bamfile integrity
0.09 minutes
0.44 Gb maximum memory
Indexing bamfile
0.0 minutes
0.44 Gb maximum memory
Counting alleles
Traceback (most recent call last):
File "/u/local/apps/midas/1.3.2/MIDAS/scripts/run_midas.py", line 757, in <module>
run_program(program, args)
File "/u/local/apps/midas/1.3.2/MIDAS/scripts/run_midas.py", line 82, in run_program
snps.run_pipeline(args)
File "/u/local/apps/midas/1.3.2/MIDAS/midas/run/snps.py", line 301, in run_pipeline
pysam_pileup(args, species, contigs)
File "/u/local/apps/midas/1.3.2/MIDAS/midas/run/snps.py", line 228, in pysam_pileup
aln_stats = utility.parallel(species_pileup, argument_list, args['threads'])
File "/u/local/apps/midas/1.3.2/MIDAS/midas/utility.py", line 101, in parallel
return [r.get() for r in results]
File "/u/local/apps/midas/1.3.2/MIDAS/midas/utility.py", line 101, in <listcomp>
return [r.get() for r in results]
File "/u/local/apps/python/3.9.6/gcc-4.8.5/lib/python3.9/multiprocessing/pool.py", line 771, in get
raise self._value
File "/u/local/apps/python/3.9.6/gcc-4.8.5/lib/python3.9/multiprocessing/pool.py", line 537, in _handle_tasks
put(task)
File "/u/local/apps/python/3.9.6/gcc-4.8.5/lib/python3.9/multiprocessing/connection.py", line 211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/u/local/apps/python/3.9.6/gcc-4.8.5/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_io.TextIOWrapper' object
Command exited with non-zero status 1
Command being timed: "run_midas.py snps midas_output/SAMPLE_1 -1 /u/local/apps/midas/EXAMPLE/example/sample_1.fq.gz -t 8"
User time (seconds): 55.55
System time (seconds): 5.84
Percent of CPU this job got: 669%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.16
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 358876
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 218678
Voluntary context switches: 61387
Involuntary context switches: 631
Swaps: 0
File system inputs: 79176
File system outputs: 169336
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 1
from midas.
Related Issues (20)
- setup.py: add scripts
- test not completing sucessfully HOT 1
- Strain tracking - getting identified strain names
- test_midas.py fail for test 7 HOT 1
- Can't download database
- Building database error HOT 3
- Test 11 and 14 fails (and fixes)
- Database marker gene file "phyeco.map" has wrong genome_id label
- samtools invalid option 'f'
- build custom database error
- samtools errors HOT 1
- Samtools missing binaries HOT 3
- How can I use MIDAS to get pN/pS on a per-gene basis?
- AttributeError: Can't pickle local object 'parallel.<locals>.init_worker' HOT 1
- test_class (__main__._07_RunSNPs) ... FAIL
- Request for information on how are marker genes chosen in custom database ? HOT 4
- Population diversity with intermediate frequency HOT 1
- cannot download the database midas_db_v1.2.tar.gz HOT 2
- no outputs for speciation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from midas.