Composite of Multiple Signals: tests for selection in meiotically recombinant populations
License: BSD 2-Clause "Simplified" License
Composite of Multiple Signals: tests for selection in meiotically recombinant populations
More detailed documentation can be found at http://broad-cms.readthedocs.org/ This includes installation instructions, usage instructions for the command line tools, and usage of the pipeline infrastructure.
When I run selscan from scans.py in an active virtual environment, it will sometime try to install selscan anyway (which then crashes due to some weird recursion issue, it looks like). A clean venv fixes the problem, but then after calling scans.py once it comes back. Based on the debug message, perhaps something is going wrong hereish?
(cms-env3)hw-uger-1000:/idi/sabeti-scratch/jvitti 16:39:24 1009 $ python cms/cms/scans.py selscan_xpehh /idi/sabeti-scratch/jvitti/sims/nulldefault_constantsize/sel1/sel_sims/sel_0.50/rep383_1.tped /idi/sabeti-scratch/jvitti/scores/nulldefault_constantsize/sel1/xpehh/sel_0.50/rep_383_1_3 /idi/sabeti-scratch/jvitti/sims/nulldefault_constantsize/sel1/sel_sims/sel_0.50/rep383_3.tped --threads 7
2016-10-19 16:41:15,374 - cmd:112:main_argparse - INFO - software version: 2.0.0-beta-4-g708555a, python version: 3.5.1 |Continuum Analytics, Inc.| (default, Dec 7 2015, 11:16:01)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
2016-10-19 16:41:15,375 - cmd:115:main_argparse - INFO - command: cms/cms/scans.py selscan_xpehh inputTped=/idi/sabeti-scratch/jvitti/sims/nulldefault_constantsize/sel1/sel_sims/sel_0.50/rep383_1.tped tmpDirKeep=False truncOk=False tmpDir=/tmp/3396933.1.interactive loglevel=DEBUG threads=7 outFile=/idi/sabeti-scratch/jvitti/scores/nulldefault_constantsize/sel1/xpehh/sel_0.50/rep_383_1_3 gapScale=20000 maf=0.05 inputRefTped=/idi/sabeti-scratch/jvitti/sims/nulldefault_constantsize/sel1/sel_sims/sel_0.50/rep383_3.tped
2016-10-19 16:41:15,375 - cmd:127:main_argparse - DEBUG - using tempDir: /tmp/3396933.1.interactive/tmp-scans-selscan_xpehh-a0c4koji
2016-10-19 16:41:15,375 - init:247:init - DEBUG - CONDA_ENV_PATH found
2016-10-19 16:41:20,804 - init:412:attempt_install - DEBUG - Attempting install...
2016-10-19 16:41:20,804 - init:493:install_package - DEBUG - Creating conda environment and installing package selscan=1.1.0b
2016-10-19 16:41:33,764 - misc:240:run_and_print - INFO - {
"error": "An unexpected error has occurred, please consider sending the\nfollowing traceback to the conda GitHub issue tracker at:\n\n https://github.com/conda/conda/issues\n\nInclude the output of the command 'conda info' in your report.\n\nTraceback (most recent call last):\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/cli/main.py", line 146, in args_func\n args.func(args, p)\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/cli/main_create.py", line 49, in execute\n install.install(args, parser, 'create')\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/cli/install.py", line 424, in install\n plan.execute_actions(actions, index, verbose=not args.quiet)\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/plan.py", line 539, in execute_actions\n inst.execute_instructions(plan, index, verbose)\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/instructions.py", line 149, in execute_instructions\n cmd(state, arg)\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/instructions.py", line 92, in LINK_CMD\n link(state['prefix'], arg, index=state['index'])\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/instructions.py", line 88, in link\n install.link(pkgs_dir, prefix, dist, lt, index=index)\n File "/home/unix/vitti/miniconda3/lib/python3.5/site-packages/conda/install.py", line 624, in link\n os.makedirs(dst_dir)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 231, in makedirs\n makedirs(head, mode, exist_ok)\n File "/home/unix/vitti/miniconda3/lib/python3.5/os.py", line 241, in makedirs\n mkdir(name, mode)\nOSError: [Errno 36] File name too long: '/idi/sabeti-scratch/jvitti/total 120M\ndrwxrwxr-x 2 vitti 6.1K Oct 19 15:56 \x1b[0m\x1b[38;5;27m.\x1b[0m/\ndrwxrwxr-x 16 vitti 323 Oct 19 15:21 \x1b[38;5;27m..\x1b[0m/\n-rwxr-xr-x 2 vitti 5.8K Sep 18 2015 \x1b[38;5;34m.gcc-post-link.sh\x1b[0m\n-rwxr-xr-x 2 vitti 427 Sep 18 2015 \x1b[38;5;34m.gcc-pre-unlink.sh\x1b[0m_\n-rwxrwxr-x 3 vitti 9.8K May 31 17:24 \x1b[38;5;34m.openssl-libcrypto-fix\x1b[0m_\n-rw-rw-r-- 3 vitti 213 May 31 17:24 \x1b[44;38;5;15m.openssl-post-link.sh\x1b[0m\nlrwxrwxrwx 1 vitti 8 Oct 19 15:16 \x1b[38;5;51m2to3\x1b[0m -> \x1b[38;5;34m2to3-3.5\x1b[0m_\n-rwxrwxr-x 1 vitti 130 Oct 19 15:17 \x1b[38;5;34m2to3-3.5\x1b[0m_\n-rwxr-xr-x 1 vitti 68K Oct 19 15:17 \x1b[38;5;34mace2sam\x1b[0m_\nlrwxrwxrwx 1 vitti 40 Oct 19 15:56 \x1b[38;5;51mactivate\x1b[0m -> \x1b[38;5;34m'\n",
"error_type": "UnexpectedError"
}Traceback (most recent call last):
File "cms/cms/scans.py", line 580, in
util.cmd.main_argparse(commands, doc)
File "/idi/sabeti-scratch/jvitti/cms/cms/util/cmd.py", line 130, in main_argparse
ret = args.func_main(args)
File "cms/cms/scans.py", line 365, in main_selscan_xpehh
gap_scale = args.gapScale
File "/idi/sabeti-scratch/jvitti/cms/cms/tools/selscan.py", line 530, in execute_xpehh
toolCmd = [self.install_and_get_path()]
File "/idi/sabeti-scratch/jvitti/cms/cms/tools/init.py", line 109, in install_and_get_path
raise NameError("unsuccessful in installing " + type(self).name)
NameError: unsuccessful in installing SelscanTool
Currently, when scans.py encounters two rows with the same position, it merges them together by default (i.e., assuming consistent ancestral coding, it collapses all derived alleles together as 1 via a bitwise procedure).
We want this procedure to take place iff the user specifies --considerMultiAllelic, and otherwise to skip these duplicate sites altogether (in 1KG P3 v5a dataset, ~9k such sites, about half of which are SNPs).
Hi,
thanks for you job to CMS.i wang to caculate the composite of multiple signals (CMS),when i use the composite.py, this wrong informations come ourt:FileNotFoundError: [Errno 2] No such file or directory: '/gpfs/home/zhengws/tools/cms/cms/combine/combine_scores_gw': '/gpfs/home/zhengws/tools/cms/cms/combine/combine_scores_gw'. i have install the cms.but i can not finad the file named combine_scores_gw,how can i deal with this wrong.
Looking forward to your reply。
Thanks again, work smoothly, be happy!
selscan_ihs now gives left and right ancestral/derived iHH scores as part of output (which is awesome!) but selscan_norm_ihs does not anticipate this, and throws the following error:
ERROR: line 2 has 10 columns, but expected 6 columns.
(these scores themselves do not need to be normalized, selscan_norm_ihs just needs to ignore them)
We had been struggling for a while with an error originated by the wrong assesment of the header by VCF reader. The error was originated by vcf_reader.py at line 83:
if line.find("CHROM", 1) > 0:
We solved the issue replacing CHROM by #CHROM
if line.find("#CHROM", 1) > 0:
We suggest this modification because bcftools, while annotating VCF files, inserts a line like the one below and is not trivial to spot the cause of the error that pops up on selscan_file_conversion.
(##bcftools_annotateCommand=annotate -a 1000G_Phase3_chr22.gz -h ancestral_allele.hdr -c CHROM,POS,AA 1000G_Phase3_chr22_noREF.vcf.gz)
Best Regards,
Output made from selscan with the --ihs-detail flag has ten columns. The old selscan normalization function throws an error at this, expecting six (normal, undetailed output). According to the selscan page (https://github.com/szpiech/selscan), norm v1.0.5 can handle this, but this version does not appear to be integrated into scans.py.
only the process promt "calculating summary statistics for 1 populations..." is printed
Hello,
I am running macOS 10.12.3 with python 2.7. I downloaded cms via instructions listed on http://broad-cms.readthedocs.io/en/latest/install.html. Installation did not prompt any errors, and I was able to establish my environment.
when running scans.py selscan_file_conversion:
(cms-env2) MacBook:cms taylorreiter$ python scans.py selscan_file_conversion -h
I receive the following error:
Traceback (most recent call last): File "scans.py", line 15, in <module> import tools.selscan File "/Users/taylorreiter/Desktop/1000 genomes correlates/vcf_files/cms/cms/tools/__init__.py", line 15, in <module> import util.file File "/Users/taylorreiter/Desktop/1000 genomes correlates/vcf_files/cms/cms/util/file.py", line 10, in <module> import util.cmd File "/Users/taylorreiter/Desktop/1000 genomes correlates/vcf_files/cms/cms/util/cmd.py", line 10, in <module> __version__ = util.version.get_version() File "/Users/taylorreiter/Desktop/1000 genomes correlates/vcf_files/cms/cms/util/version.py", line 117, in get_version write_release_version(from_git) File "/Users/taylorreiter/Desktop/1000 genomes correlates/vcf_files/cms/cms/util/version.py", line 60, in write_release_version with open(release_file(), 'wt') as outf: IOError: [Errno 13] Permission denied: '/Users/taylorreiter/Desktop/1000 genomes correlates/vcf_files/cms/cms/VERSION'
$ ls -lh
returns
total 320 -rw-r--r-- 1 root staff 141B Feb 3 10:53 FileConverter.py -rwxr-xr-x 1 root staff 257B Feb 3 10:53 cms_exceptions.py -rw-r--r-- 1 root staff 20K Feb 3 10:53 cms_modeller.py drwxr-xr-x 15 root staff 510B Feb 3 10:53 combine -rw-r--r-- 1 root staff 16K Feb 3 10:53 composite.py drwxr-xr-x 6 root staff 204B Feb 3 10:53 dists -rw-r--r-- 1 root staff 22K Feb 3 10:53 likes_from_model.py -rwxr-xr-x 1 root staff 171B Feb 3 10:53 main.py drwxr-xr-x 16 root staff 544B Feb 3 10:53 model drwxr-xr-x 6 root staff 204B Feb 3 10:53 power -rw-r--r-- 1 root staff 51K Feb 3 10:53 power.py -rw-r--r-- 1 root staff 712B Feb 3 10:53 run_traj.py -rwxr-xr-x 1 root staff 26K Feb 3 10:53 scans.py -rwxr-xr-x 1 root staff 101B Feb 3 10:53 selection.py drwxr-xr-x 5 root staff 170B Feb 3 10:53 test drwxr-xr-x 4 root staff 136B Feb 3 10:53 tools drwxr-xr-x 15 root staff 510B Feb 3 10:53 util
I'm not sure where to begin to resolve this issue. Any help would be greatly appreciated.
Issue with installation (pip -r install requirements.txt): Biopython fails because it can't find Numpy even though it's installed through Anacondas.
When I try to install the software via conda (v4.2.12) I get the following error message:
NoPackagesFoundError: Package missing in current linux-64 channels:
- python-dateutil 2.3*
```.
Can I user a later version of dateutil?
Correlation between selscan/sweep iHS values matches that shown in Hernandez et al. 2014; we are ready to move forward with integrating selscan into the CMS 2.0 pipeline. We want to percolate up to the CMS 2.0 user some of selscan's normalization options (e.g. --bins, --crit-percent, --crit-val).
Within cms/tools/selscan.py, alter process_vcf_into_selscan_tped() so it will correctly act on allosomal VCF files. Of note is that within the genotypes for chromosome X, some sample records will be singletons (i.e. men, ex. "0"), while some will be phased pairs (i.e. women, ex. "0|1"). This will require adjusting the creation of the index list used to access the genotypes by position, or creating an alternate, slower, code path that is more flexible.
The python module imp will be deprecated in favor of importlib. Dynamic loading in scans.py should be updated to use importlib. Example usage:
hw-uger-1080:/idi/sabeti-scratch/jvitti/test_github/cms 16:17:21 376 $ conda env create -f=conda-environment_py2.yaml -n cms-env
Using Anaconda Cloud api site https://api.anaconda.org
Using Anaconda Cloud api site https://api.anaconda.org
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata ...........
Solving package specifications: ..........
Fetching packages ...
ncurses-5.9-8. 100% |#################################################################################################################################################################| Time: 0:00:00 2.22 MB/s
python-2.7.12- 100% |#################################################################################################################################################################| Time: 0:00:01 10.49 MB/s
alabaster-0.7. 100% |#################################################################################################################################################################| Time: 0:00:00 4.05 MB/s
boltons-16.4.1 100% |#################################################################################################################################################################| Time: 0:00:00 1.66 MB/s
docutils-0.12- 100% |#################################################################################################################################################################| Time: 0:00:00 1.63 MB/s
ecdsa-0.13-py2 100% |#################################################################################################################################################################| Time: 0:00:00 643.88 kB/s
imagesize-0.7. 100% |#################################################################################################################################################################| Time: 0:00:00 606.38 kB/s
markupsafe-0.2 100% |#################################################################################################################################################################| Time: 0:00:00 1.17 MB/s
numpy-1.11.0-p 100% |#################################################################################################################################################################| Time: 0:00:01 4.95 MB/s
peewee-2.8.0-p 100% |#################################################################################################################################################################| Time: 0:00:00 1.55 MB/s
pycrypto-2.6.1 100% |#################################################################################################################################################################| Time: 0:00:00 1.77 MB/s
pygments-2.1.3 100% |#################################################################################################################################################################| Time: 0:00:00 2.86 MB/s
pytz-2016.4-py 100% |#################################################################################################################################################################| Time: 0:00:00 1.09 MB/s
setuptools-23. 100% |#################################################################################################################################################################| Time: 0:00:00 1.51 MB/s
six-1.10.0-py2 100% |#################################################################################################################################################################| Time: 0:00:00 595.35 kB/s
snowballstemme 100% |#################################################################################################################################################################| Time: 0:00:00 687.56 kB/s
sphinx_rtd_the 100% |#################################################################################################################################################################| Time: 0:00:00 2.08 MB/s
wheel-0.29.0-p 100% |#################################################################################################################################################################| Time: 0:00:00 885.64 kB/s
babel-2.3.3-py 100% |#################################################################################################################################################################| Time: 0:00:00 6.01 MB/s
biopython-1.67 100% |#################################################################################################################################################################| Time: 0:00:00 4.32 MB/s
jinja2-2.8-py2 100% |#################################################################################################################################################################| Time: 0:00:00 1.22 MB/s
paramiko-1.16. 100% |#################################################################################################################################################################| Time: 0:00:00 1.28 MB/s
pip-8.1.2-py27 100% |#################################################################################################################################################################| Time: 0:00:00 3.12 MB/s
pysam-0.9.1-py 100% |#################################################################################################################################################################| Time: 0:00:01 4.02 MB/s
python-dateuti 100% |#################################################################################################################################################################| Time: 0:00:00 994.95 kB/s
scipy-0.17.0-n 100% |#################################################################################################################################################################| Time: 0:00:22 1.41 MB/s
pandas-0.18.1- 100% |#################################################################################################################################################################| Time: 0:00:01 11.43 MB/s
Extracting packages ...
[ COMPLETE ]|####################################################################################################################################################################################| 100%
Linking packages ...
An unexpected error has occurred, please consider sending the | 4%
following traceback to the conda GitHub issue tracker at:https://github.com/conda/conda/issuesInclude the output of the command 'conda info' in your report.
Traceback (most recent call last):
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/bin/conda-env", line 6, in
sys.exit(conda_env.cli.main.main())
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda_env/cli/main.py", line 68, in main
return args_func(args, parser)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda/cli/main.py", line 130, in args_func
exit_code = args.func(args, p)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda_env/cli/main_create.py", line 99, in execute
installer.install(prefix, pkg_specs, args, env)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda_env/installers/conda.py", line 20, in install
plan.execute_actions(actions, index, verbose=not args.quiet)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda/plan.py", line 595, in execute_actions
inst.execute_instructions(plan, index, verbose)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda/instructions.py", line 136, in execute_instructions
cmd(state, arg)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda/instructions.py", line 79, in LINK_CMD
link(state['prefix'], dist, lt, index=state['index'], shortcuts=shortcuts)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda/install.py", line 1042, in link
with Locked(prefix), Locked(pkgs_dir):
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/site-packages/conda/lock.py", line 60, in enter
os.makedirs(self.lock_path)
File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/lib/python3.5/os.py", line 241, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/broad/software/free/Linux/redhat_6_x86_64/pkgs/anaconda3_4.0.0/pkgs/.conda_lock-3985'
I'm running into a new bug when I try to use selscan_file_conversion. I give e.g. the following command:
(cms-env3) hw-uger-1003:/idi/sabeti-scratch/jvitti 14:17:50 1018 $ python /idi/sabeti-scratch/jvitti/cms/cms/scans.py selscan_file_conversion /idi/sabeti-scratch/jvitti/cms_venv/dry_run/vcfs/YRI-overlap-IRN_chr1.recode.vcf.gz /idi/sabeti-scratch/ilya/gsvn/Data/Ilya_Data/genmaps/hm2/genetic_map_GRCh37_chr1.txt YRI-overlap-IRN_chr1 /idi/sabeti-scratch/jvitti/cms_venv/dry_run/tpeds/ 1
and this is the output:
2016-08-31 14:17:58,879 - cmd:112:main_argparse - INFO - software version: 5a22200, python version: 3.5.1 |Continuum Analytics, Inc.| (default, Dec 7 2015, 11:16:01)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
2016-08-31 14:17:58,879 - cmd:115:main_argparse - INFO - command: /idi/sabeti-scratch/jvitti/cms/cms/scans.py selscan_file_conversion rescaleGeneticDistance=False chromosomeNum=1 ploidy=2 codingFunctionClassFile=None tmpDir=/tmp/2972228.1.interactive loglevel=DEBUG considerMultiAllelic=False filterPops=None genMap=/idi/sabeti-scratch/ilya/gsvn/Data/Ilya_Data/genmaps/hm2/genetic_map_GRCh37_chr1.txt includeLowQualAncestral=False outPrefix=YRI-overlap-IRN_chr1 outLocation=/idi/sabeti-scratch/jvitti/cms_venv/dry_run/tpeds/ inputVCF=/idi/sabeti-scratch/jvitti/cms_venv/dry_run/vcfs/YRI-overlap-IRN_chr1.recode.vcf.gz tmpDirKeep=False endBp=None sampleMembershipFile=None filterSuperPops=None multiAllelicMergeFunction=OR startBp=0
2016-08-31 14:17:58,880 - cmd:127:main_argparse - DEBUG - using tempDir: /tmp/2972228.1.interactive/tmp-scans-selscan_file_conversion-mck2k3jq
Traceback (most recent call last):
File "/idi/sabeti-scratch/jvitti/cms/cms/scans.py", line 580, in
util.cmd.main_argparse(commands, doc)
File "/idi/sabeti-scratch/jvitti/cms/cms/util/cmd.py", line 130, in main_argparse
ret = args.func_main(args)
File "/idi/sabeti-scratch/jvitti/cms/cms/scans.py", line 136, in main_selscan_file_conversion
multi_alleli_merge_function = args.multiAllelicMergeFunction)
File "/idi/sabeti-scratch/jvitti/cms/cms/tools/selscan.py", line 101, in process_vcf_into_selscan_tped
processor = VCFReader(vcf_file)
File "/idi/sabeti-scratch/jvitti/cms/cms/util/vcf_reader.py", line 31, in init
self.sample_names = self.read_sample_names()
File "/idi/sabeti-scratch/jvitti/cms/cms/util/vcf_reader.py", line 93, in read_sample_names
headerRowString = self.sample_header_line()
File "/idi/sabeti-scratch/jvitti/cms/cms/util/vcf_reader.py", line 81, in sample_header_line
elem = headerIter.next()
AttributeError: 'pysam.ctabix.GZIteratorHead' object has no attribute 'next'I've tried cloning the repository de novo, and have re-created the input VCFs (including bgzip and tabix) a few times, and can't seem to resolve the problem. Could this be to do with pysam version? @tomkinsc can you provide any insight?
This command on the Broad server python /idi/sabeti-scratch/jvitti/cms/cms/scans.py selscan_file_conversion /idi/sabeti-scratch/jvitti/1kg_vcfs/regional/snps_1kg_chrX_BEB_148764000_158764000.recode.vcf.gz /idi/sabeti-scratch/ilya/gsvn/Data/Ilya_Data/genmaps/hm2/genetic_map_GRCh37_chrX.txt snps_1kg_chrX_BEB_148764000_158764000 /idi/sabeti-scratch/jvitti/1kg_tpeds/regional/ X gives me the following error:
Traceback (most recent call last):
File "/idi/sabeti-scratch/jvitti/cms/cms/scans.py", line 580, in
util.cmd.main_argparse(commands, doc)
File "/idi/sabeti-scratch/jvitti/cms/cms/util/cmd.py", line 130, in main_argparse
ret = args.func_main(args)
File "/idi/sabeti-scratch/jvitti/cms/cms/scans.py", line 136, in main_selscan_file_conversion
multi_alleli_merge_function = args.multiAllelicMergeFunction)
File "/idi/sabeti-scratch/jvitti/cms/cms/tools/selscan.py", line 174, in process_vcf_into_selscan_tped
of1l.write(lineToWrite1)
File "/home/unix/vitti/miniconda3/envs/cms-env3/lib/python3.5/gzip.py", line 258, in write
data = memoryview(data)
TypeError: memoryview: a bytes-like object is required, not 'str'
Similar commands on autosomal files perform fine. The source file is bgzipped and Tabix-indexed.
I'm running XPEHH calculations for a bunch of different population pairs. Using the same script, a bunch of these have gotten started running, while the latter half fails with an unexpected error:
2017-01-28 09:48:28,476 - cmd:112:main_argparse - INFO - software version: 2.0.0-beta-181-ge43cb00-dirty, python version: 3.5.2 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
2017-01-28 09:48:28,477 - cmd:115:main_argparse - INFO - command: /n/home08/jvitti/cms/cms/scans.py selscan_xpehh tmpDirKeep=False inputRefTped=/n/regal/sabeti_lab/jvitti/clear-synth/1kg_tpeds/chr1_strictMask_PJL_clear-synth-20161227.tped.gz maf=0.05 threads=8 inputTped=/n/regal/sabeti_lab/jvitti/clear-synth/1kg_tpeds/chr1_strictMask_FIN_clear-synth-20161227.tped.gz outFile=/n/regal/sabeti_lab/jvitti/clear-synth/1kg_scores/xpehh/chr1_strictMask_FIN_clear-synth-20161227_vs_PJL tmpDir=/tmp gapScale=20000 loglevel=DEBUG truncOk=False
2017-01-28 09:48:28,478 - cmd:127:main_argparse - DEBUG - using tempDir: /tmp/tmp-scans-selscan_xpehh-semx0iw6
Traceback (most recent call last):
File "/n/home08/jvitti/cms/cms/scans.py", line 580, in
util.cmd.main_argparse(commands, doc)
File "/n/home08/jvitti/cms/cms/util/cmd.py", line 130, in main_argparse
ret = args.func_main(args)
File "/n/home08/jvitti/cms/cms/scans.py", line 365, in main_selscan_xpehh
gap_scale = args.gapScale
File "/n/home08/jvitti/cms/cms/tools/selscan.py", line 530, in execute_xpehh
toolCmd = [self.install_and_get_path()]
File "/n/home08/jvitti/cms/cms/tools/init.py", line 124, in install_and_get_path
self.install()
File "/n/home08/jvitti/cms/cms/tools/init.py", line 101, in install
m.attempt_install()
File "/n/home08/jvitti/cms/cms/tools/init.py", line 145, in attempt_install
self._attempt_install()
File "/n/home08/jvitti/cms/cms/tools/init.py", line 441, in _attempt_install
pkg_version = self.verify_install()
File "/n/home08/jvitti/cms/cms/tools/init.py", line 386, in verify_install
installed_version = self.get_installed_version()
File "/n/home08/jvitti/cms/cms/tools/init.py", line 480, in get_installed_version
matches = package_info_re.match(installed_package_string)
TypeError: expected string or bytes-like object
srun: error: holy2a18202: task 0: Exited with exit code 1
And I now appear to get the same error when I run scans.py from the command line. I don't have this issue on the Broad cluster. I tried rolling back to conda 4.0.10 (which is what's installed on the Broad) but ran into another unexpected error:
(cms-env3) [jvitti@holy2a18308 /]$ conda install -q -y conda==4.0.10
Fetching package metadata ...........InstallError: Install error: Error: 'conda' can only be installed into the root environment
I can always rsync the data to the Broad server and run the calculations there, but it'd be great to understand why this (apparently) stopped working -- spooky.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.