jlanga / smsk_popoolation Goto Github PK
View Code? Open in Web Editor NEWSnakemake pipeline for Popoolation and Popoolation2
License: MIT License
Snakemake pipeline for Popoolation and Popoolation2
License: MIT License
Working through the example data, I ran using "snakemake --use-conda --cores 24"
However, I got an error in rule "raw_extract_genome" that the 2R.fa.gz file (supplied example data) is not in gzip format. Is it the example data or the command at fault?
GATK in mind (indel realigner + base recalibrator)
I got the following error while running the example dataset. The file 2R.mpileup.log is empty - possibly a problem with fifo. System administrator checked and mkfifo is definitely installedon the system being used. The version installed is 8.28.
System administrator also tried recreating the software (running the snakemake pipeline) on another of our development machines and hit other problems with packages that are definitely installed but it doesn’t seem to like it.
I'm currently testing the popoolation2 part by running each step separately from command line.
Activating conda environment: /home/dyern/lstm_scratch/smsk_popoolation/.snakemake/conda/5af75ca214322542d38ca9626894a8ef
mkfifo: cannot create fifo 'results/mpileup/filt/pop2/pop2.2R.mpileup': Operation not permitted
[Mon Dec 6 12:29:00 2021]
Error in rule mpileup_popoolation_filter_indels:
jobid: 27
output: results/mpileup/filt/pop2/pop2.2R.mpileup, results/mpileup/filt/pop2/pop2.2R.mpileup.gz
log: results/mpileup/filt/pop2/pop2.2R.mpileup.log (check log file(s) for error message)
conda-env: /home/dyern/lstm_scratch/smsk_popoolation/.snakemake/conda/5af75ca214322542d38ca9626894a8ef
shell:
mkfifo results/mpileup/filt/pop2/pop2.2R.mpileup
(cat results/mpileup/filt/pop2/pop2.2R.mpileup | gzip --fast > results/mpileup/filt/pop2/pop2.2R.mpileup.gz &)
perl src/popoolation_1.2.2/basic-pipeline/filter-pileup-by-gtf.pl --input <(gzip --decompress --stdout results/mpileup/raw/pop2/pop2.2R.mpileup.gz) --gtf results/mpileup/filt/pop2/pop2.2R.gtf --output results/mpileup/filt/pop2/pop2.2R.mpileup 2> results/mpileup/filt/pop2/pop2.2R.mpileup.log 1>&2
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
[Mon Dec 6 12:29:03 2021]
Finished job 6.
33 of 71 steps (46%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/dyern/lstm_scratch/smsk_popoolation/.snakemake/log/2021-12-06T122740.426984.snakemake.log
Hi,
I went through all the past issue and hit the same problems as previous users.
/smsk_popoolation$ snakemake --conda-create-envs-only
Error: you need to specify the maximum number of CPU cores to be used at the same time. If you want to use N cores, say --cores N or -cN. For all cores on your system (be sure that this is appropriate) use --cores all. For no parallelization use --cores 1 or -c1.
When I go with certain number of cpus or --cores all always get following error:
ModuleNotFoundError in line 3 of /home/*/smsk_popoolation/workflow/Snakefile:
No module named 'pandas'
I tried to work it around by installing pandas in the env or adding pandas to dependencies in smsk_popoolation/workflow/rules/helpers/environment.yml, nothing changed, same error with pandas occurs
Could you update documentation to workable version ?
Thank you
When sorting, samtools writes a bunch of temporary files next to the final bam. if snakemake
is interrupted, it removes some of the files, but not the intermediate ones. Use /tmp/
to store the intermediate files and for easier cleanup later.
picard uses by default just 1Gb of RAM. It causes problems when working on big files.
Add something in config.yaml to expand it and modify the map and mpileup steps
Related to RG tags
Hi, thank you for your contributions!
While I want to install smsk_popoolation after installing anaconda3, I met an issue like the following.
git clone https://github.com/jlanga/smsk_popoolation.git smsk_popoolation #run well
cd smsk_popoolation # run well
snakemake --use-conda --create-envs-only #dont't work and get the message
The program 'snakemake' is currently not installed. To run 'snakemake' please ask your administrator to install the package 'snakemake'.
so I need install 'snakemake' before using snakemake or 'snakemaker' is a script of 'smsk_popoolation'?
Any reply will be welcomed.
This may be due to an update in snakemake commands. When I tried to paste the command from the instructions "snakemake --use-conda --create-envs-only" I got the error message "snakemake: error: unrecognized arguments: --create-envs-only"
First I tried replacing with snakemake --use-conda --conda-create-envs-only but I got the following error message asking me to enter the number of cores (Error: you need to specify the maximum number of CPU cores to be used at the same time. If you want to use N cores, say --cores N or -cN. For all cores on your system (be sure that this is appropriate) use --cores all. For no parallelization use --cores 1 or -c1.)
I then tried replacing with "snakemake --use-conda --conda-create-envs-only --cores 24" This seems to run as far as "Building DAG of jobs...
Creating conda environment src/snakefiles/hp.yml...
Downloading and installing remote packages."
Although this is taking rather a long time (> 5 minutes) so there may still be an issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.