chapmanb / cloudbiolinux Goto Github PK

View Code? Open in Web Editor NEW

255.0 255.0 160.0 8.13 MB

CloudBioLinux: configure virtual (or real) machines with tools for biological analyses

Home Page: http://cloudbiolinux.org

License: MIT License

Python 92.48% Shell 4.53% Ruby 2.39% HTML 0.59%

cloudbiolinux's People

Contributors

Stargazers

Watchers

Forkers

scilifelab pjotrp afgane intronic chrihart sterlesser aryantaheri hjanime jcvi-cloud seanward pajanne iracooke xychua jamesgmorgan taoito hmenager jmchilton tadmor csw pombredanne i000 kevyin sebhtml nldowell clsung jasonbishop bishudas janusz005 traksewt arq5x wijerasa claresloggett effect tanglingfung bksriharsha andersje porterjamesj changpingc mariogiov hrez jianwuwang adaaouak xiaojay caddymob heuermh ny-shao tistaharahap nuada glebkuznetsov kern3020 rajuniit clearinterface vduvvuri imclab rmlawton petertonoli msgendev yothin lpantano kccg colorshifter genemechanic brainiarc7 abdo3a ashwinkalbhor bcil honglongwu ishaqsharief28 nathan-albrighton scatreux mauriciocarneiro rbloom5 saadatqadri kdaily rtvt123 uom-resplat-devops nishesh18 averagehat yli845 churchlab 7uv joemphilips elkingtonmcb aicidnn jn7163 marcelcaraciolo sucharitha93 mtbkeith humanner druvus bennyyu686 geetduggal naumenko-sa fredus14 vladsavelyev cbrueffer sch-sdgs yxartemis teslaa22 alenzhao

cloudbiolinux's Issues

centos 6.4 nginx build pauses

when installing nginx, installer pauses:

[localhost] sudo: cd /usr; stow nginx
[localhost] local: rm -rf /home/jeremy/tmp/cloudbiolinux
[localhost] put: /tmp/tmptIywJm -> /usr/nginx/conf/nginx.conf

at the 'put' until a carriage return is pressed...

Interest in Libvirt/KVM VM version?

I would like to be able to use Cloudbiolinux on my virtualization hosts running libvirt/KVM. Would you be interested in adding such a feature to the main distribution, if I wrote it?

ubuntu 13.04 fails on pydoop install

This would appear to be due to incorrect java home -- it's setting ot to /usr/lib/jvm/java-6-openjdk, when on this 64-bit ubuntu 13.04 box, it's actually /usr/lib/jvm/java-6-openjdk-amd64/

while a person can get around this by manually setting it in /etc/profile or somesuch, it sure would be nice if CBL auto-detected and set this :)

snippet:

DBG [fabfile.py]: Importing module cloudbio.custom.distributed
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.distributed.install_pydoop
[localhost] local: python -c 'import pydoop'
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named pydoop

Warning: local() encountered an error (return code 1) while executing 'python -c 'import pydoop''

[localhost] local: sudo bash -c "export JAVA_HOME=/usr/lib/jvm/java-6-openjdk && pip install pydoop"
Downloading/unpacking pydoop
Downloading pydoop-0.10.0_rc1.tar.gz (1.0MB): 1.0MB downloaded
Running setup.py egg_info for package pydoop
Traceback (most recent call last):
File "", line 16, in
File "/tmp/pip-build-root/pydoop/setup.py", line 46, in
import pydoop
File "pydoop/init.py", line 41, in
_HADOOP_INFO = _PATH_FINDER.find() # fill the cache ASAP
File "pydoop/hadoop_utils.py", line 419, in find
info[a] = getattr(self, a)()
File "pydoop/hadoop_utils.py", line 323, in hadoop_version_info
self.hadoop_version(hadoop_home)
File "pydoop/hadoop_utils.py", line 312, in hadoop_version
raise RuntimeError(err or out)
RuntimeError: /usr/lib/hadoop-0.20/bin/hadoop: line 470: /usr/lib/jvm/java-6-openjdk/bin/java: No such file or directory

Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 16, in

File "/tmp/pip-build-root/pydoop/setup.py", line 46, in

import pydoop

File "pydoop/init.py", line 41, in

_HADOOP_INFO = _PATH_FINDER.find()  # fill the cache ASAP

File "pydoop/hadoop_utils.py", line 419, in find

info[a] = getattr(self, a)()

File "pydoop/hadoop_utils.py", line 323, in hadoop_version_info

self.hadoop_version(hadoop_home)

File "pydoop/hadoop_utils.py", line 312, in hadoop_version

raise RuntimeError(err or out)

RuntimeError: /usr/lib/hadoop-0.20/bin/hadoop: line 470: /usr/lib/jvm/java-6-openjdk/bin/java: No such file or directory

Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-root/pydoop
Storing complete log in /home/bioinformatics/.pip/pip.log

Fatal error: local() encountered an error (return code 1) while executing 'sudo bash -c "export JAVA_HOME=/usr/lib/jvm/java-6-openjdk && pip install pydoop"'

Aborting.
Disconnecting from localhost... done.
bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ ls /usr/lib/jvm/java-6-openjdk/bin/java
ls: cannot access /usr/lib/jvm/java-6-openjdk/bin/java: No such file or directory

bio-linux-mira : Depends: mira-assembler but it is not going to be installed on Ubuntu 13.04

Ran into this when installing cloudbiolinux to a fresh Ubuntu Server 13.04

$ fab -f fabfile.py -H localhost install_biolinux
...
INFO: Package install progress: 60/360
[localhost] sudo: apt-get -y --force-yes install bio-linux-dust bio-linux-lucy bio-linux-mira bio-linux-mira-3rd-party bio-linux-tablet bio-linux-bldp-files bio-linux-genquery bio-linux-keyring bio-linux-sampledata bio-linux-taxinspector bio-linux-xcut rasmol texlive-latex-base texlive-latex-extra texlive-latex-recommended erlang erlang-base erlang-dev erlang-examples erlang-manpages erlang-mode hadoop-0.20 hadoop-0.20-pipes hadoop-0.20-source hadoop-hive hadoop-pig libhdfs0-dev c2hs cabal-install ghc6
[localhost] out: sudo password:
...
[localhost] out: bio-linux-keyring is already the newest version.
[localhost] out: Some packages could not be installed. This may mean that you have
[localhost] out: requested an impossible situation or if you are using the unstable
[localhost] out: distribution that some required packages have not yet been created
[localhost] out: or been moved out of Incoming.
[localhost] out: The following information may help to resolve the situation:
[localhost] out: 
[localhost] out: The following packages have unmet dependencies:
[localhost] out:  bio-linux-mira : Depends: mira-assembler but it is not going to be installed
[localhost] out: E: Unable to correct problems, you have held broken packages.

Do I just need to add mira-assembler to config/packages.yaml?

package conflict on Debian

I see a conflict error:

[192.168.2.29] out: dpkg: error processing /var/cache/apt/archives/bio-linux-emboss_6.3bl1-8_all.deb (--unpack):
[192.168.2.29] out: trying to overwrite '/usr/share/EMBOSS/data/REBASE/embossre.equ', which is also in package emboss-data 6.4.0-4

Pakcages seem to conflict on at least one file.

Package list has some dependencies to Oneiric release.

Assuming I choose lucid release, packages such as gnome-session-fallback, unity-2d, and libmpich2-3 won't be found in the repository. So config/packages.yaml should be aware of the chosen release.

Good job though :)

CORTEX has new version

currently cloudbio/custom/bio_nextgen.py hardcodes a check for version 1.0.5.18 of CORTEX -- but 1.0.5.20 is out, and they've removed 1.0.5.18 from the latest/ directory.

It seems like this would a perfect place for them to put in a symlink, so we could just get http://sourceforge.net/projects/cortexassembler/files/cortex_var/latest/latest.tar.gz and hope for the best, eh?

updating version to be 1.0.5.20 in cloudbio/custom/bio_nextgen.py allows the build to proceed.

distribute in python_libs fails to install

on running fab -f fabfile.py -H localhost install_libraries:python on centos 6.4, I get a compile error on setuptools. Manually running "pip install --upgrade distribute" also fails, but running "sudo pip install --upgrade distribute" DOES succeed.

[localhost] local: pip install --upgrade distribute
Downloading/unpacking distribute
Downloading distribute-0.7.3.zip (145kB): 145kB downloaded
Running setup.py egg_info for package distribute

Downloading/unpacking setuptools>=0.7 (from distribute)
Downloading setuptools-0.9.8.tar.gz (766kB): 766kB downloaded
Running setup.py egg_info for package setuptools

Installing collected packages: distribute, setuptools
Running setup.py install for distribute

Found existing installation: setuptools 0.6c11
Uninstalling setuptools:
Successfully uninstalled setuptools
Running setup.py install for setuptools
error: could not delete '/usr/anaconda/lib/python2.7/site-packages/setuptools/_backport/init.py': Permission denied
Complete output from command /usr/anaconda/bin/python -c "import setuptools;file='/tmp/pip-build-cbl/setuptools/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-pbw1NF-record/install-record.txt --single-version-externally-managed:
running install

running build

running build_py

creating build

creating build/lib

copying pkg_resources.py -> build/lib

copying easy_install.py -> build/lib

creating build/lib/_markerlib

copying _markerlib/init.py -> build/lib/_markerlib

copying _markerlib/markers.py -> build/lib/_markerlib

creating build/lib/setuptools

copying setuptools/py27compat.py -> build/lib/setuptools

copying setuptools/init.py -> build/lib/setuptools

copying setuptools/py24compat.py -> build/lib/setuptools

copying setuptools/site-patch.py -> build/lib/setuptools

copying setuptools/ssl_support.py -> build/lib/setuptools

copying setuptools/archive_util.py -> build/lib/setuptools

copying setuptools/package_index.py -> build/lib/setuptools

copying setuptools/sandbox.py -> build/lib/setuptools

copying setuptools/extension.py -> build/lib/setuptools

copying setuptools/script template.py -> build/lib/setuptools

copying setuptools/compat.py -> build/lib/setuptools

copying setuptools/script template (dev).py -> build/lib/setuptools

copying setuptools/depends.py -> build/lib/setuptools

copying setuptools/dist.py -> build/lib/setuptools

creating build/lib/setuptools/tests

copying setuptools/tests/init.py -> build/lib/setuptools/tests

copying setuptools/tests/test_build_ext.py -> build/lib/setuptools/tests

copying setuptools/tests/test_test.py -> build/lib/setuptools/tests

copying setuptools/tests/test_upload_docs.py -> build/lib/setuptools/tests

copying setuptools/tests/py26compat.py -> build/lib/setuptools/tests

copying setuptools/tests/test_easy_install.py -> build/lib/setuptools/tests

copying setuptools/tests/doctest.py -> build/lib/setuptools/tests

copying setuptools/tests/test_resources.py -> build/lib/setuptools/tests

copying setuptools/tests/test_markerlib.py -> build/lib/setuptools/tests

copying setuptools/tests/test_packageindex.py -> build/lib/setuptools/tests

copying setuptools/tests/test_dist_info.py -> build/lib/setuptools/tests

copying setuptools/tests/server.py -> build/lib/setuptools/tests

copying setuptools/tests/test_bdist_egg.py -> build/lib/setuptools/tests

copying setuptools/tests/test_sandbox.py -> build/lib/setuptools/tests

copying setuptools/tests/test_develop.py -> build/lib/setuptools/tests

copying setuptools/tests/test_sdist.py -> build/lib/setuptools/tests

copying setuptools/tests/test_egg_info.py -> build/lib/setuptools/tests

creating build/lib/setuptools/command

copying setuptools/command/install.py -> build/lib/setuptools/command

copying setuptools/command/init.py -> build/lib/setuptools/command

copying setuptools/command/bdist_wininst.py -> build/lib/setuptools/command

copying setuptools/command/easy_install.py -> build/lib/setuptools/command

copying setuptools/command/install_lib.py -> build/lib/setuptools/command

copying setuptools/command/rotate.py -> build/lib/setuptools/command

copying setuptools/command/test.py -> build/lib/setuptools/command

copying setuptools/command/sdist.py -> build/lib/setuptools/command

copying setuptools/command/saveopts.py -> build/lib/setuptools/command

copying setuptools/command/bdist_rpm.py -> build/lib/setuptools/command

copying setuptools/command/develop.py -> build/lib/setuptools/command

copying setuptools/command/upload_docs.py -> build/lib/setuptools/command

copying setuptools/command/alias.py -> build/lib/setuptools/command

copying setuptools/command/egg_info.py -> build/lib/setuptools/command

copying setuptools/command/register.py -> build/lib/setuptools/command

copying setuptools/command/install_scripts.py -> build/lib/setuptools/command

copying setuptools/command/upload.py -> build/lib/setuptools/command

copying setuptools/command/bdist_egg.py -> build/lib/setuptools/command

copying setuptools/command/setopt.py -> build/lib/setuptools/command

copying setuptools/command/build_py.py -> build/lib/setuptools/command

copying setuptools/command/install_egg_info.py -> build/lib/setuptools/command

copying setuptools/command/build_ext.py -> build/lib/setuptools/command

creating build/lib/setuptools/_backport

copying setuptools/_backport/init.py -> build/lib/setuptools/_backport

creating build/lib/setuptools/_backport/hashlib

copying setuptools/_backport/hashlib/init.py -> build/lib/setuptools/_backport/hashlib

copying setuptools/_backport/hashlib/_sha512.py -> build/lib/setuptools/_backport/hashlib

copying setuptools/_backport/hashlib/_sha.py -> build/lib/setuptools/_backport/hashlib

copying setuptools/_backport/hashlib/_sha256.py -> build/lib/setuptools/_backport/hashlib

running install_lib

copying build/lib/_markerlib/init.py -> /usr/anaconda/lib/python2.7/site-packages/_markerlib

copying build/lib/_markerlib/markers.py -> /usr/anaconda/lib/python2.7/site-packages/_markerlib

copying build/lib/pkg_resources.py -> /usr/anaconda/lib/python2.7/site-packages

copying build/lib/easy_install.py -> /usr/anaconda/lib/python2.7/site-packages

copying build/lib/setuptools/py27compat.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/init.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/py24compat.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/tests/init.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_build_ext.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_test.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_upload_docs.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/py26compat.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_easy_install.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/doctest.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_resources.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_markerlib.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_packageindex.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_dist_info.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/server.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_bdist_egg.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_sandbox.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_develop.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_sdist.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/tests/test_egg_info.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/tests

copying build/lib/setuptools/site-patch.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/ssl_support.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/archive_util.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/package_index.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/sandbox.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/extension.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/command/install.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/init.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/bdist_wininst.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/easy_install.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/install_lib.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/rotate.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/test.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/sdist.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/saveopts.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/bdist_rpm.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/develop.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/upload_docs.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/alias.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/egg_info.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/register.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/install_scripts.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/upload.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/bdist_egg.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/setopt.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/build_py.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/install_egg_info.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/command/build_ext.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/command

copying build/lib/setuptools/script template.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/compat.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/script template (dev).py -> /usr/anaconda/lib/python2.7/site-packages/setuptools

copying build/lib/setuptools/_backport/init.py -> /usr/anaconda/lib/python2.7/site-packages/setuptools/_backport

error: could not delete '/usr/anaconda/lib/python2.7/site-packages/setuptools/_backport/init.py': Permission denied

Rolling back uninstall of setuptools
Command /usr/anaconda/bin/python -c "import setuptools;file='/tmp/pip-build-cbl/setuptools/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-pbw1NF-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-cbl/setuptools
Storing complete log in /home/cbl/.pip/pip.log

Fatal error: local() encountered an error (return code 1) while executing 'pip install --upgrade distribute'

Aborting.
[cbl@co-cbl cloudbiolinux]$

gemini build fails on ubuntu 13.04

It looks to me to be a $PATHing issue -- perhaps ensuring that gemini_install.py is called with a full path will resolve this?

INFO: Custom install for 'gemini' start time: 2013-08-21 20:01:22.369799
DBG [fabfile.py]: Importing module cloudbio.custom.bio_nextgen
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.bio_nextgen.install_gemini
[localhost] local: echo $HOME
[localhost] local: wget --no-check-certificate https://raw.github.com/arq5x/gemini/master/gemini/scripts/gemini_install.py
--2013-08-21 15:01:22-- https://raw.github.com/arq5x/gemini/master/gemini/scripts/gemini_install.py
Resolving raw.github.com (raw.github.com)... 199.27.74.133
Connecting to raw.github.com (raw.github.com)|199.27.74.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10299 (10K) [text/plain]
Saving to: ‘gemini_install.py.4’

100%[======================================>] 10,299 --.-K/s in 0s

2013-08-21 15:01:22 (35.6 MB/s) - ‘gemini_install.py.4’ saved [10299/10299]

[localhost] local: /usr/anaconda/bin/python gemini_install.py /usr /usr/local/share/gemini
Checking required dependencies...
git found
wget found
curl found
Installing isolated base python installation
Installing gemini...
Traceback (most recent call last):
File "gemini_install.py", line 229, in
main(parser.parse_args())
File "gemini_install.py", line 43, in main
install_conda_pkgs(anaconda)
File "gemini_install.py", line 96, in install_conda_pkgs
subprocess.check_call([anaconda["conda"], "install", "--yes"] + pkgs)
File "/usr/anaconda/lib/python2.7/subprocess.py", line 537, in check_call
retcode = call(_popenargs, *_kwargs)
File "/usr/anaconda/lib/python2.7/subprocess.py", line 524, in call
return Popen(_popenargs, *_kwargs).wait()
File "/usr/anaconda/lib/python2.7/subprocess.py", line 711, in init
errread, errwrite)
File "/usr/anaconda/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Fatal error: local() encountered an error (return code 1) while executing '/usr/anaconda/bin/python gemini_install.py /usr /usr/local/share/gemini'

Aborting.

install fails on ubuntu 13.04 amazon image

Passing the galaxy working directory, into the install below, doesn't seem to work quite right. Will submit pull request which fixes this momentarily.

DBG [fabfile.py]: Importing module cloudbio.custom.distributed
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.distributed.install_seal
[localhost] local: /usr/anaconda/bin/python -c 'import bl.mr.seq.seqal'
Traceback (most recent call last):
File "", line 1, in
File "/usr/anaconda/lib/python2.7/site-packages/bl/mr/seq/seqal/init.py", line 25, in
from pydoop.pipes import runTask, Factory
File "/usr/anaconda/lib/python2.7/site-packages/pydoop/pipes.py", line 29, in
pp = pydoop.import_version_specific_module('_pipes')
File "/usr/anaconda/lib/python2.7/site-packages/pydoop/init.py", line 107, in import_version_specific_module
return import_module(complete_mod_name(name))
File "/usr/anaconda/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
ImportError: /usr/anaconda/lib/python2.7/site-packages/pydoop/_pipes_0_20_2_cdh_3_6.so: undefined symbol: BIO_s_mem

Warning: local() encountered an error (return code 1) while executing '/usr/anaconda/bin/python -c 'import bl.mr.seq.seqal''

[localhost] local: /usr/anaconda/bin/python -c 'import pydoop'
[localhost] local: export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 && /usr/anaconda/bin/pip install --pre seal
Requirement already satisfied (use --upgrade to upgrade): seal in /usr/anaconda/lib/python2.7/site-packages
Cleaning up...
INFO: Custom install for 'seal' end time: 2013-08-29 20:06:11.433446; duration: 0:00:01.874966
INFO: Custom install for 'galaxy_webapp' start time: 2013-08-29 20:06:11.434399
DBG [fabfile.py]: Importing module cloudbio.custom.galaxy
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.galaxy.install_galaxy_webapp
DBG [init.py]: Done setting up Galaxy/CloudMan users
WARN [init.py(110)]: Galaxy install dir /mnt/galaxyTools/galaxy-central exists and seems to have a Mercurial repository already there. Galaxy already installed?
[localhost] local: sudo http_proxy=$http_proxy https_proxy=$https_proxy bash -c "chown --recursive galaxy:galaxy /mnt/galaxyTools/galaxy-central"
[localhost] local: sudo http_proxy=$http_proxy https_proxy=$https_proxy bash -c "chown galaxy:galaxy /mnt/galaxyTools"
[localhost] local: sudo http_proxy=$http_proxy https_proxy=$https_proxy bash -c "chown --recursive galaxy:galaxy /mnt/galaxyTools/galaxy-central"
[localhost] local: sudo http_proxy=$http_proxy https_proxy=$https_proxy su - galaxy bash -c "sed -i 's/python .//python -ES .//g' run.sh"
sed: can't read run.sh: No such file or directory

Fatal error: local() encountered an error (return code 2) while executing 'sudo http_proxy=$http_proxy https_proxy=$https_proxy su - galaxy bash -c "sed -i 's/python .//python -ES .//g' run.sh"'

Aborting.
Disconnecting from localhost... done.
ubuntu@ip-10-249-59-84:~/cloudbiolinux$

restarting failed localhost install can spawn GUI for beast, et al

when doing a localhost install, the first install of tassel and beast, for instance, work fine. But if the install fails, and one re-runs the

fab -f ./fabfile.py -i /path/to/my/key -H localhost install_biolinux

beast and tassel and gemini will pop up a window, via X forwarding. ubuntu enables X forwarding by default. The easiest way to keep this from happening that I've found is to do a quick "export DISPLAY=/dev/null" before running the install, but it'd be mighty handy if the install script did it for me, so I didn't have to remember.

bio-blastxmlparser will not install on centos 6.4

Nothing to update
[localhost] local: sudo bash -c "gem install bio-blastxmlparser"
ERROR: Error installing bio-blastxmlparser:
nokogiri requires Ruby version >= 1.9.2.

Fatal error: local() encountered an error (return code 1) while executing 'sudo bash -c "gem install bio-blastxmlparser"'

Aborting.
Disconnecting from localhost... done.

centos 6.4 install attempts to install erroneous RPM packages

erlang-devel is required, but is not available via yum.
boost141 and boost141-devel are required, but are not available via yum
prover9 is required, but is not available via yum

globus-openss should be globus-openssl-module
globus-openssl-devel should be globus-openssl-module-devel

additional packages which should be added:

libuuid-devel #needed for CPAN UUID module

seal install fails on centos 6.4

immediately after pydoop installs, seal fails to complete.

INFO: Custom install for 'seal' start time: 2013-08-07 15:03:25.370697
DBG [fabfile.py]: Importing module cloudbio.custom.distributed
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.distributed.install_seal
[localhost] local: python -c 'import bl.mr.seq.seqal'
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named bl.mr.seq.seqal

Warning: local() encountered an error (return code 1) while executing 'python -c 'import bl.mr.seq.seqal''

[localhost] local: python -c 'import pydoop'
[localhost] local: sudo bash -c "export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/ && pip install seal"
Downloading/unpacking seal

pip can't proceed with requirement 'seal' due to a pre-existing build directory.
location: /tmp/pip-build-root/seal
This is likely due to a previous installation that failed.
pip is being responsible and not assuming it can delete this.
Please delete it and try again.

Cleaning up...
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/req.py", line 1071, in prepare_files
raise e
PreviousBuildDirError:
pip can't proceed with requirement 'seal' due to a pre-existing build directory.
location: /tmp/pip-build-root/seal
This is likely due to a previous installation that failed.
pip is being responsible and not assuming it can delete this.
Please delete it and try again.

Storing complete log in /root/.pip/pip.log

Fatal error: local() encountered an error (return code 2) while executing 'sudo bash -c "export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/ && pip install seal"'

Aborting.
Disconnecting from localhost... done.

hg19-rnaseq.tar.xz

it doesn't work with gtf2fasta (with the hg19.fa downloaded from the amazon cloud), thanks.

beast install requires password to be set on user acount

because of this, even when installing with a pubkey-auth only ubuntu box, it hangs on beast install.

DBG [fabfile.py]: Importing module cloudbio.custom.phylogeny
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.phylogeny.install_beast
[localhost] Login password for 'ubuntu':
Sorry, you can't enter an empty password. Please try again.
[localhost] Login password for 'ubuntu':
Sorry, you can't enter an empty password. Please try again

matplotlib custom build fails on Debian

[[email protected]:2222] out: running build
[[email protected]:2222] out: running build_py
[[email protected]:2222] out: copying lib/matplotlib/mpl-data/matplotlibrc -> build/lib.linux-i686-2.6/matplotlib/[[email protected]:2222] out:
[[email protected]:2222] out: copying lib/matplotlib/mpl-data/matplotlib.conf -> build/lib.linux-i686-2.6/matplotl[[email protected]:2222] out:
[[email protected]:2222] out: running build_ext
[[email protected]:2222] out: building 'matplotlib.ft2font' extension
[[email protected]:2222] out: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPY_ARRAY_UNIQUE_SYMBOL=MPL_ARRAY_API -DPYCXX_ISO_CPP_LIB=1 -I/usr/lib/pymodules/python2.6/numpy/core/include -I/usr/local/include -I/usr/include -I. -I/usr/lib/pymodules/python2.6/numpy/core/include/freetype2 -I/usr/local/include/freetype2 -I/usr/include/freetype2 -I./freetype2 -I/usr/include/python2.6 -c src/ft2font.cpp -o build/te[[email protected]:2222] out: o
(...)
[[email protected]:2222] out: src/ft2font.cpp:109: error: â was not declared in this scope
[[email protected]:2222] out: src/ft2font.cpp:110: error: â was not declared in this scope
[[email protected]:2222] out: /usr/lib/pymodules/python2.6/numpy/core/include/numpy/__multiarray_api.h:1187: warni[[email protected]:2222] out:
[[email protected]:2222] out: error: command 'gcc' failed with exit status 1
[[email protected]:2222] out:

Fatal error: run() encountered an error (return code 1) while executing 'python setup.py build'

AWS ubuntu13.04: install_data_s3 install fails on mkdir -p /mnt/biodata/genomes

install runs fine if I manually create /mnt/biodata/genomes, and chown it to ubuntu before restarting it. My quick grep of the code didn't reveal unto me the source of the issue, but the log follows below:

[localhost] local: echo $HOME
[localhost] local: uname -m
INFO: Now, testing connection to host...
INFO: Connection to host appears to work!
DBG [utils.py]: Expand paths
INFO: List of genomes to get (from the config file at '/home/ubuntu/cloudbiolinux/config/biodata.yaml'): phiX174, Human (hg19), Human (GRCh37), D melangogaster (dm3), Mouse (mm9), Arabidopsis thaliana (TAIR9), X tropicalis (xenTro2), S cerevisiae (sacCer2), C elegans (WS210)
[localhost] local: mkdir -p /mnt/biodata/genomes

Warning: local() encountered an error (return code 1) while executing 'mkdir -p /mnt/biodata/genomes'

[localhost] local: mkdir -p /mnt/biodata/genomes/phiX174/phix
mkdir: cannot create directory ‘/mnt/biodata’: Permission denied

Fatal error: local() encountered an error (return code 1) while executing 'mkdir -p /mnt/biodata/genomes/phiX174/phix'

Aborting.

ubuntu 13.04 build fails at virtualenv-burrito.sh

...
libcurl4-openssl-dev is already the newest version.
libssl-dev is already the newest version.
libxml2-dev is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.
[localhost] local: sudo bash -c "apt-get clean"
[localhost] local: echo $HOME
[localhost] local: mkdir -p '/home/bioinformatics/tmp/cloudbiolinux'
[localhost] local: curl -s https://raw.github.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | $SHELL
[localhost] local: echo '. $HOME/.venvburrito/startup.sh' >> "$(echo ~/.bashrc)"
[localhost] local: source ~/.bashrc
/bin/sh: 1: source: not found

Fatal error: local() encountered an error (return code 127) while executing 'source ~/.bashrc'

Aborting.
Disconnecting from localhost... done.

CloudMan test

Brad,

Have you tried cloudman on top of cloudbiolinux?
There are some steps related to cloudman installation but it seems the outcome is not fully functional.

Going to try mi-deployment, apparently it is not adapted for a private cloud environment.

Cheers,
Aryan

ubuntu 13.04 install fails on FreeC

[localhost] local: sudo bash -c "cp -rf ./duppy /usr/bin"
[localhost] local: sudo bash -c "cp -rf ./jumpy /usr/bin"
[localhost] local: sudo bash -c "cp -rf ./delly /usr/bin"
[localhost] local: rm -rf /home/bioinformatics/tmp/cloudbiolinux
INFO: Custom install for 'delly' end time: 2013-08-19 19:54:12.021271; duration: 0:00:11.880527
INFO: Custom install for 'freec' start time: 2013-08-19 19:54:12.023767
DBG [fabfile.py]: Importing module cloudbio.custom.bio_nextgen
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.bio_nextgen.install_freec
[localhost] local: echo $HOME
[localhost] local: mkdir -p '/home/bioinformatics/tmp/cloudbiolinux'
[localhost] local: wget --no-check-certificate -O FREEC_Linux64.tar.gz 'http://bioinfo-out.curie.fr/projects/freec/src/FREEC_Linux64.tar.gz'
--2013-08-19 14:54:12-- http://bioinfo-out.curie.fr/projects/freec/src/FREEC_Linux64.tar.gz
Resolving bioinfo-out.curie.fr (bioinfo-out.curie.fr)... 193.49.205.28
Connecting to bioinfo-out.curie.fr (bioinfo-out.curie.fr)|193.49.205.28|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1443420 (1.4M) [application/x-tar]
Saving to: ‘FREEC_Linux64.tar.gz’

100%[======================================>] 1,443,420 479KB/s in 2.9s

2013-08-19 14:54:15 (479 KB/s) - ‘FREEC_Linux64.tar.gz’ saved [1443420/1443420]

[localhost] local: tar -xzpf FREEC_Linux64.tar.gz
[localhost] local: make
make -f Makefile.freec init
make[1]: Entering directory /home/bioinformatics/tmp/cloudbiolinux' make[1]: Nothing to be done forinit'.
make[1]: Leaving directory /home/bioinformatics/tmp/cloudbiolinux' make -f Makefile.freec depend make[1]: Entering directory/home/bioinformatics/tmp/cloudbiolinux'
make[1]: Leaving directory /home/bioinformatics/tmp/cloudbiolinux' make -f Makefile.freec all make[1]: Entering directory/home/bioinformatics/tmp/cloudbiolinux'
g++ -O3 -g -DPROFILE_TRACE -Wall -m64 -c -o main.o main.cpp
In file included from SNPinGenome.h:10:0,
from SVfinder.h:64,
from main.cpp:24:
ThreadPool.h: In member function ‘bool Thread::waitFinished()’:
ThreadPool.h:101:22: error: ‘usleep’ was not declared in this scope
main.cpp: In function ‘int main(int, char*)’:
main.cpp:198:36: error: ‘access’ was not declared in this scope
main.cpp:217:53: error: ‘access’ was not declared in this scope
make[1]: ** [main.o] Error 1
make[1]: Leaving directory `/home/bioinformatics/tmp/cloudbiolinux'
make: *** [all] Error 2

Fatal error: local() encountered an error (return code 2) while executing 'make'

Aborting.

Make RabbitMQ installation non-interactive

I have not dig why it has happened, but while installing rabbitmq, it said there exist a previous version. So there was this whiptail box, asking for OK. Fab doesn't support it, so simply you should add more parameters to debconf-set-selections.

# echo "rabbitmq-server rabbitmq-server/upgrade_previous note" | debconf-set-selections

ubuntu 13.04 install failing on seal

this is just one machine failing on this...I have cbl successfully installed on another, so I'm not sure if this is a regression, or something isolated to just this machine...

Interestingly, the log seems to indicate that the --pre flag needs to be passed into the install.

INFO: Custom install for 'seal' start time: 2013-08-22 16:59:39.297220
DBG [fabfile.py]: Importing module cloudbio.custom.distributed
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.distributed.install_seal
[localhost] local: /usr/anaconda/bin/python -c 'import bl.mr.seq.seqal'
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named bl.mr.seq.seqal

Warning: local() encountered an error (return code 1) while executing '/usr/anaconda/bin/python -c 'import bl.mr.seq.seqal''

[localhost] local: /usr/anaconda/bin/python -c 'import pydoop'
[localhost] local: export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 && /usr/anaconda/bin/pip install seal
Downloading/unpacking seal
Could not find a version that satisfies the requirement seal (from versions: devel-20120307)
Cleaning up...
No distributions matching the version for seal
Storing complete log in /tmp/tmpe7O2t1

Fatal error: local() encountered an error (return code 1) while executing 'export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 && /usr/anaconda/bin/pip install seal'

Aborting.
Disconnecting from localhost... done.

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ cat /tmp/tmpe7O2t1

/usr/anaconda/bin/pip run on Thu Aug 22 11:59:40 2013
Downloading/unpacking seal

Getting page https://pypi.python.org/simple/seal/
URLs to search for versions for seal:

https://pypi.python.org/simple/seal/
Analyzing links from page https://pypi.python.org/simple/seal/
Found link https://pypi.python.org/packages/source/s/seal/seal-devel-20120307.tar.gz#md5=e69bc390078303547a6088e3e679f57b (from https://pypi.python.org/simple/seal/), version: devel-20120307
Skipping link http://hadoop.apache.org (from https://pypi.python.org/simple/seal/); not a file
Skipping link http://pydoop.sourceforge.net (from https://pypi.python.org/simple/seal/); not a file
Ignoring link https://pypi.python.org/packages/source/s/seal/seal-devel-20120307.tar.gz#md5=e69bc390078303547a6088e3e679f57b (from https://pypi.python.org/simple/seal/), version devel-20120307 is a pre-release (use --pre to allow).
Could not find a version that satisfies the requirement seal (from versions: devel-20120307)

Cleaning up...

Removing temporary dir /tmp/pip_build_bioinformatics...
No distributions matching the version for seal

Exception information:
Traceback (most recent call last):
File "/usr/anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/usr/anaconda/lib/python2.7/site-packages/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/anaconda/lib/python2.7/site-packages/pip/req.py", line 1085, in prepare_files
url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
File "/usr/anaconda/lib/python2.7/site-packages/pip/index.py", line 309, in find_requirement
raise DistributionNotFound('No distributions matching the version for %s' % req)
DistributionNotFound: No distributions matching the version for seal
bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$

Do not crash on install_data for not found dbsnp/rnaseq

While running:

fab -f data_fabfile.py -H biologin -c ~/config/biodata/fabricrc.txt install_data:~config/biodata/biodata.yaml

And having the following information in biodata.yaml:

  - dbkey: dm3
    name: D melangogaster (dm3)
    rnaseq: true

The whole process crashes when it's not found on Broad's FTP repository:

INFO: Processing genome Scerevisiae and putting it to /proj/a2010002/nobackup/roman/genomes/Scerevisiae/sacCer2
INFO: Processing genome Celegans and putting it to /proj/a2010002/nobackup/roman/genomes/Celegans/WS210
[biologin] run: wget ftp://gsapubftp-anonymous:@ftp.broadinstitute.org/bundle/2.3/dm3/dbsnp_137.dm3.vcf.gz
[biologin] out: --2013-02-24 23:48:18--  ftp://gsapubftp-anonymous:*password*@ftp.broadinstitute.org/bundle/2.3/dm3/dbsnp_137.dm3.vcf.gz
[biologin] out:            => “dbsnp_137.dm3.vcf.gz”
[biologin] out: Resolving ftp.broadinstitute.org... 69.173.80.251
[biologin] out: Connecting to ftp.broadinstitute.org|69.173.80.251|:21... connected.
[biologin] out: Logging in as gsapubftp-anonymous ... Logged in!
[biologin] out: ==> SYST ... done.    ==> PWD ... done.
[biologin] out: ==> TYPE I ... done.  ==> CWD (1) /bundle/2.3/dm3 ...
[biologin] out: No such directory “bundle/2.3/dm3”.
[biologin] out:
[biologin] out:


Fatal error: run() received nonzero return code 8 while executing!

Requested: wget ftp://gsapubftp-anonymous:@ftp.broadinstitute.org/bundle/2.3/dm3/dbsnp_137.dm3.vcf.gz
Executed: /bin/bash -i -l -c "cd roman/genomes/Dmelanogaster/dm3/variation && export PATH=\"\$PATH:\"/usr/local/bin\" \" && wget ftp://gsapubftp-anonymous:@ftp.broadinstitute.org/bundle/2.3/dm3/dbsnp_137.dm3.vcf.gz"

Aborting.
Disconnecting from biologin... done.

perl-modules : Breaks: libextutils-cbuilder-perl (< 0.280203) on Ubuntu 12.04

On Ubuntu Server 12.04

$ fab -f fabfile.py -H localhost install_biolinux
...
INFO: Package install progress: 120/362
[localhost] sudo: apt-get -y --force-yes install libreadline-dev libssl-dev libxml2 libxml2-dev libxslt1-dev libxslt1.1 libzip-dev mime-support uuid uuid-dev xz-utils zlib1g-dev zlib1g-dev octave prover9 cpanminus libextutils-autoinstall-perl libextutils-cbuilder-perl libperl-dev parrot parrot-devel perl perl-modules perl-tk python python-dev python-distutils-extra python-pip python-rpy python-setuptools
[localhost] out: sudo password:
[localhost] out: 
...
[localhost] out: perl-modules is already the newest version.
[localhost] out: perl-modules set to manually installed.
[localhost] out: Some packages could not be installed. This may mean that you have
[localhost] out: requested an impossible situation or if you are using the unstable
[localhost] out: distribution that some required packages have not yet been created
[localhost] out: or been moved out of Incoming.
[localhost] out: The following information may help to resolve the situation:
[localhost] out: 
[localhost] out: The following packages have unmet dependencies:
[localhost] out:  perl-modules : Breaks: libextutils-cbuilder-perl (< 0.280203)
[localhost] out: E: Unable to correct problems, you have held broken packages.

If I understand this correctly, libextutils-cbuilder-perl is already included in perl-modules, so it can safely be removed from config/packages.yaml.

debian packages not available

Hi,
I found the following packages, not available in Debian (sid at least) preventing the install:

ugene
mongodb-10gen -> mongodb
clojure
gdm -> gdm3
ubuntu-desktop
unity-2d
freenx-server (this one would need an other repository)

Olivier

Install error on Debian sid

Hi,
I just tried to install cloudbiolinux on a debian sid and it failed:

Does it mean that I can install it only against a stable release (squeeze etc..) ?


...
[192.168.2.44] out: Ign http://downloads-distro.mongodb.org dist/10gen Translation-en  
[192.168.2.44] out: Err http://download.virtualbox.org sid/contrib amd64 Packages  
[192.168.2.44] out:   404  Not found
[192.168.2.44] out: Ign http://download.virtualbox.org sid/contrib Translation-en
[192.168.2.44] out: Err http://cran.stat.ucla.edu sid-cran/ Packages
[192.168.2.44] out:   404  Not Found
[192.168.2.44] out: Ign http://cran.stat.ucla.edu sid-cran/ Translation-en
[192.168.2.44] out: Fetched 123 kB in 3s (41.1 kB/s)
[192.168.2.44] out: W: GPG error: http://nebc.nerc.ac.uk unstable Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B1A598E8128B92BD
[192.168.2.44] out: W: Failed to fetch http://download.virtualbox.org/virtualbox/debian/dists/sid/contrib/binary-amd64/Packages  404  Not found
[192.168.2.44] out: W: Failed to fetch http://cran.stat.ucla.edu/bin/linux/debian/sid-cran/Packages  404  Not Found

[192.168.2.44] out: E: Some index files failed to download. They have been ignored, or old ones used instead.

bx_python install fails on ubuntu 13.04

[localhost] local: rm -rf /home/bioinformatics/tmp/cloudbiolinux
INFO: Custom install for 'tracer' end time: 2013-08-14 13:38:20.220218; duration: 0:00:10.961804
INFO: Custom install for 'bx-python' start time: 2013-08-14 13:38:20.220481
DBG [fabfile.py]: Importing module cloudbio.custom.python
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.python.install_bx_python
[localhost] local: python -c 'import bx'
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named bx

Warning: local() encountered an error (return code 1) while executing 'python -c 'import bx''

[localhost] local: pip install --upgrade distribute
Downloading/unpacking distribute from https://pypi.python.org/packages/source/d/distribute/distribute-0.7.3.zip#md5=c6c59594a7b180af57af8a0cc0cf5b4a
Downloading distribute-0.7.3.zip (145kB): 145kB downloaded
Running setup.py egg_info for package distribute

Downloading/unpacking setuptools>=0.7 (from distribute)
Downloading setuptools-0.9.8.tar.gz (766kB): 766kB downloaded
Running setup.py egg_info for package setuptools

Installing collected packages: distribute, setuptools
Found existing installation: distribute 0.6.45
Uninstalling distribute:
Successfully uninstalled distribute
Running setup.py install for distribute

Found existing installation: distribute 0.6.45
Can't uninstall 'distribute'. No files were found to uninstall.
Running setup.py install for setuptools

Installing easy_install script to /usr/anaconda/bin
Installing easy_install-2.7 script to /usr/anaconda/bin

Successfully installed distribute setuptools
Cleaning up...
[localhost] local: pip install --upgrade https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2
Downloading/unpacking https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2
Downloading tip.tar.bz2 (1.9MB): 1.9MB downloaded
Running setup.py egg_info for package from https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz
Extracting in /tmp/tmpWY77H0
Now working in /tmp/tmpWY77H0/distribute-0.6.35
Building a Distribute egg in /tmp/pip-HQ601F-build
/tmp/pip-HQ601F-build/distribute-0.6.35-py2.7.egg
/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe'
warnings.warn(msg)
/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'
warnings.warn(msg)
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help

error: invalid command 'egg_info'
Complete output from command python setup.py egg_info:
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz

Extracting in /tmp/tmpWY77H0

Now working in /tmp/tmpWY77H0/distribute-0.6.35

Building a Distribute egg in /tmp/pip-HQ601F-build

/tmp/pip-HQ601F-build/distribute-0.6.35-py2.7.egg

/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe'

warnings.warn(msg)

/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'

warnings.warn(msg)

usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

or: -c --help [cmd1 cmd2 ...]

or: -c --help-commands

or: -c cmd --help

error: invalid command 'egg_info'

Command python setup.py egg_info failed with error code 1 in /tmp/pip-HQ601F-build
Storing complete log in /home/bioinformatics/.pip/pip.log

Fatal error: local() encountered an error (return code 1) while executing 'pip install --upgrade https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2'

Aborting.

CONTENTS of /home/bioinformatics/.pip/pip.log:

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ cat /home/bioinformatics/.pip/pip.log

/usr/anaconda/bin/pip run on Wed Aug 14 08:38:31 2013
Downloading/unpacking https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2

Downloading from URL https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2
Running setup.py egg_info for package from https://bitbucket.org/james_taylor/bx-python/get/tip.tar.bz2

Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz

Extracting in /tmp/tmpWY77H0

Now working in /tmp/tmpWY77H0/distribute-0.6.35

Building a Distribute egg in /tmp/pip-HQ601F-build

/tmp/pip-HQ601F-build/distribute-0.6.35-py2.7.egg

/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe'

  warnings.warn(msg)

/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'

  warnings.warn(msg)

usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

   or: -c --help [cmd1 cmd2 ...]

   or: -c --help-commands

   or: -c cmd --help



error: invalid command 'egg_info'

Complete output from command python setup.py egg_info:

Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz

Extracting in /tmp/tmpWY77H0

Now working in /tmp/tmpWY77H0/distribute-0.6.35

Building a Distribute egg in /tmp/pip-HQ601F-build

/tmp/pip-HQ601F-build/distribute-0.6.35-py2.7.egg

/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe'

warnings.warn(msg)

/usr/anaconda/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'

warnings.warn(msg)

usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

or: -c --help [cmd1 cmd2 ...]

or: -c --help-commands

or: -c cmd --help

error: invalid command 'egg_info'

Command python setup.py egg_info failed with error code 1 in /tmp/pip-HQ601F-build

Exception information:
Traceback (most recent call last):
File "/usr/anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/usr/anaconda/lib/python2.7/site-packages/pip/commands/install.py", line 266, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/anaconda/lib/python2.7/site-packages/pip/req.py", line 1057, in prepare_files
req_to_install.run_egg_info()
File "/usr/anaconda/lib/python2.7/site-packages/pip/req.py", line 236, in run_egg_info
command_desc='python setup.py egg_info')
File "/usr/anaconda/lib/python2.7/site-packages/pip/util.py", line 662, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip-HQ601F-build

AWS ubuntu13.04: install fails on seal

running fab -f ./fabfile.py -H localhost install_custom:seal generates a failure:

make: Leaving directory `/tmp/pip_build_ubuntu/seal/bl/lib/seq/aligner/bwa/libbwa'

Buildfile: /tmp/pip_build_ubuntu/seal/build.xml

 [echo] debug.build: false

version_check:

find_hadoop:

BUILD FAILED

/tmp/pip_build_ubuntu/seal/build.xml:71: exec returned: 1

Total time: 1 second

error: Could not build Java components

Cleaning up...
Command /usr/local/anaconda/bin/python -c "import setuptools;file='/tmp/pip_build_ubuntu/seal/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-GxgqL9-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_ubuntu/seal
Traceback (most recent call last):
File "/usr/local/anaconda/bin/pip", line 6, in
sys.exit(main())
File "/usr/local/anaconda/lib/python2.7/site-packages/pip/init.py", line 161, in main
return command.main(args[1:], options)
File "/usr/local/anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 169, in main
text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 25: ordinal not in range(128)

Fatal error: local() encountered an error (return code 1) while executing 'export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 && /usr/local/anaconda/bin/pip install --pre seal'

Aborting.

404 when downloading Vagrant biolinux_version.box

I'm trying to follow the instructions at https://github.com/chapmanb/cloudbiolinux#vagrant-virtualbox but I can't download the Vagrant base box:

$ vagrant box add biolinux_version https://s3.amazonaws.com/chapmanb/biolinux_version.box
[vagrant] Downloading with Vagrant::Downloaders::HTTP...
[vagrant] Downloading box: https://s3.amazonaws.com/chapmanb/biolinux_version.box
Bad status code: 404

Please verify that the box exists and is accessible. Also verify that
this computer is properly connected to the internet.
$ curl https://s3.amazonaws.com/chapmanb/biolinux_version.box
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message>    <Key>biolinux_version.box</Key><RequestId>720078D3AEAC656D</RequestId><HostId>sZUD/6RnPd1TNvIgzsScDRY/k2MkFWcGiM3/mOfIL6x2RW2wwEtFM3i6eMrxU/2L</HostId></Error>

sad news: miniconda-1.6.2-Linux-x86_64.sh no longer available

Ubuntu 13.04 install fails when trying to access http://repo.continuum.io/miniconda/Miniconda-1.6.2-Linux-x86_64.sh

a manual check of that repo shows that the aforementioned package is unavailable. The choices are either to use python3 + Miniconda v1.6.2 or python2 + Miniconda-1.9.1

ruby-ensembl-api install fails on centos 6.4

Needs ruby-1.9.3, which centos 6.4 does not have.

[localhost] local: sudo bash -c "gem install ruby-ensembl-api"
ERROR: Error installing ruby-ensembl-api:
activesupport requires Ruby version >= 1.9.3.

Fatal error: local() encountered an error (return code 1) while executing 'sudo bash -c "gem install ruby-ensembl-api"'

Aborting.
Disconnecting from localhost... done.

centos6.4 install fails on scikit install.

the latest version I can find is
scikit-learn-0.13.1-np17py27_0.tar.bz2

but the code is trying to get
http://repo.continuum.io/pkgs/free/linux-64/scikit-learn-0.14.1-np17py27_0.tar.bz2'

text of error:

Bio::GFF3::LowLevel is up to date. (1.6)
DBG [config.py]: Using config file /home/cbl/sandbox/cloudbiolinux/cloudbio/../config/python-libs.yaml
[localhost] local: conda install --yes boto

All requested packages already installed.

packages in environment at /usr/anaconda:

boto 2.9.9 py27_0
[localhost] local: conda install --yes biopython

All requested packages already installed.

packages in environment at /usr/anaconda:

biopython 1.61 np17py27_0
[localhost] local: conda install --yes cython

All requested packages already installed.

packages in environment at /usr/anaconda:

cython 0.19.1 py27_0
[localhost] local: conda install --yes h5py

All requested packages already installed.

packages in environment at /usr/anaconda:

h5py 2.1.1 np17py27_0
[localhost] local: conda install --yes ipython

All requested packages already installed.

packages in environment at /usr/anaconda:

ipython 0.13.2 py27_0
[localhost] local: conda install --yes matplotlib

All requested packages already installed.

packages in environment at /usr/anaconda:

matplotlib 1.2.1 np17py27_1
[localhost] local: conda install --yes networkx

All requested packages already installed.

packages in environment at /usr/anaconda:

networkx 1.8 py27_0
[localhost] local: conda install --yes nose

All requested packages already installed.

packages in environment at /usr/anaconda:

nose 1.3.0 py27_0
[localhost] local: conda install --yes numpy

All requested packages already installed.

packages in environment at /usr/anaconda:

numpy 1.7.1 py27_0
[localhost] local: conda install --yes pandas

Package plan for installation in environment /usr/anaconda:

The following packages will be downloaded:

package	build
pandas-0.12.0	np17py27_0 6.9 MB

The following packages will be UN-linked:

package	build
pandas-0.11.0	np17py27_1

The following packages will be linked:

package	build
pandas-0.12.0	np17py27_0

Fetching packages ...
pandas-0.12.0-np17py27_0.tar.bz2 100% |#################################################################################################################################| Time: 0:00:03 1.96 MB/s
Extracting packages ...
[ COMPLETE ] |#####################################################################################################################################################################| 100%
Unlinking packages ...
[ COMPLETE ] |#####################################################################################################################################################################| 100%
Linking packages ...
[ COMPLETE ] |#####################################################################################################################################################################| 100%
[localhost] local: conda install --yes pysam

All requested packages already installed.

packages in environment at /usr/anaconda:

pysam 0.6 py27_0
[localhost] local: conda install --yes pyyaml

All requested packages already installed.

packages in environment at /usr/anaconda:

pyyaml 3.10 py27_0
[localhost] local: conda install --yes pyzmq

All requested packages already installed.

packages in environment at /usr/anaconda:

pyzmq 2.2.0.1 py27_1
[localhost] local: conda install --yes scikit-learn

Package plan for installation in environment /usr/anaconda:

The following packages will be downloaded:

package	build
scikit-learn-0.14.1	np17py27_0 6.2 MB

The following packages will be UN-linked:

package	build
scikit-learn-0.13.1	np17py27_0

The following packages will be linked:

package	build
scikit-learn-0.14.1	np17py27_0

Fetching packages ...
Error: Could not locate 'http://repo.continuum.io/pkgs/free/linux-64
/scikit-learn-0.14.1-np17py27_0.tar.bz2'

Fatal error: local() encountered an error (return code 1) while executing 'conda install --yes scikit-learn'

Aborting.
Disconnecting from localhost... done.

rpy2-bioconductor-extensions fails install on centos 6.4

[localhost] local: sudo bash -c "pip install --upgrade rpy2"
Requirement already up-to-date: rpy2 in /usr/lib64/python2.6/site-packages
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade rpy2-bioconductor-extensions"
Downloading/unpacking rpy2-bioconductor-extensions
Could not find a version that satisfies the requirement rpy2-bioconductor-extensions (from versions: 0.2.4-dev, 0.2.5-dev, 0.2.3-dev, 0.2-dev)
Cleaning up...
No distributions matching the version for rpy2-bioconductor-extensions
Storing complete log in /root/.pip/pip.log

Fatal error: local() encountered an error (return code 1) while executing 'sudo bash -c "pip install --upgrade rpy2-bioconductor-extensions"'

Aborting.
Disconnecting from localhost... done.

contents of /root/.pip/pip.log:

[jeremy@centos-try2 cloudbiolinux]$ sudo cat /root/.pip/pip.log

/usr/bin/pip run on Wed Aug 7 11:00:16 2013
Downloading/unpacking rpy2-bioconductor-extensions

Getting page https://pypi.python.org/simple/rpy2-bioconductor-extensions/
URLs to search for versions for rpy2-bioconductor-extensions:

https://pypi.python.org/simple/rpy2-bioconductor-extensions/
Analyzing links from page https://pypi.python.org/simple/rpy2-bioconductor-extensions/
Found link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2.4-dev.tar.gz#md5=111d3afaafe7b7aa4f4a5f0d0e4186b2 (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version: 0.2.4-dev
Found link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2.5-dev.tar.gz#md5=ea81c5c92c3b9de1b024ff7bfec5156a (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version: 0.2.5-dev
Found link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2.3-dev.tar.gz#md5=164d8084f4c512d079ac8f785c0fa696 (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version: 0.2.3-dev
Found link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2-dev.tar.gz#md5=22e3fa85cb3ff7e668694e7280b51eed (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version: 0.2-dev
Skipping link http://packages.python.org/rpy2-bioconductor-extensions (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/); not a file
Ignoring link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2.4-dev.tar.gz#md5=111d3afaafe7b7aa4f4a5f0d0e4186b2 (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version 0.2.4-dev is a pre-release (use --pre to allow).
Ignoring link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2.5-dev.tar.gz#md5=ea81c5c92c3b9de1b024ff7bfec5156a (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version 0.2.5-dev is a pre-release (use --pre to allow).
Ignoring link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2.3-dev.tar.gz#md5=164d8084f4c512d079ac8f785c0fa696 (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version 0.2.3-dev is a pre-release (use --pre to allow).
Ignoring link https://pypi.python.org/packages/source/r/rpy2-bioconductor-extensions/rpy2-bioconductor-extensions-0.2-dev.tar.gz#md5=22e3fa85cb3ff7e668694e7280b51eed (from https://pypi.python.org/simple/rpy2-bioconductor-extensions/), version 0.2-dev is a pre-release (use --pre to allow).
Could not find a version that satisfies the requirement rpy2-bioconductor-extensions (from versions: 0.2.4-dev, 0.2.5-dev, 0.2.3-dev, 0.2-dev)

Cleaning up...

Removing temporary dir /tmp/pip-build-root...
No distributions matching the version for rpy2-bioconductor-extensions

Exception information:
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/req.py", line 1077, in prepare_files
url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/index.py", line 306, in find_requirement
raise DistributionNotFound('No distributions matching the version for %s' % req)
DistributionNotFound: No distributions matching the version for rpy2-bioconductor-extensions
[jeremy@centos-try2 cloudbiolinux]$

ubuntu 13.04 fails on pybedtools install

Apparently, pybedtools needs cython to continue, but pybedtools is installed BEFORE cython.

moving the cython requirement in config/python-libs.yaml to be before pybedtools appears to resolve the issue.

build failure:

[localhost] local: sudo bash -c "pip install --upgrade Pyrex"
Requirement already up-to-date: Pyrex in /usr/local/lib/python2.7/dist-packages
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade Twisted"
Requirement already up-to-date: Twisted in /usr/local/lib/python2.7/dist-packages
Requirement already up-to-date: zope.interface>=3.6.0 in /usr/lib/python2.7/dist-packages (from Twisted)
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade cutadapt"
Requirement already up-to-date: cutadapt in /usr/local/lib/python2.7/dist-packages
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade ruffus"
Requirement already up-to-date: ruffus in /usr/local/lib/python2.7/dist-packages
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade DendroPy"
Requirement already up-to-date: DendroPy in /usr/local/lib/python2.7/dist-packages
Requirement already up-to-date: setuptools in /usr/local/lib/python2.7/dist-packages (from DendroPy)
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade HTSeq"
Requirement already up-to-date: HTSeq in /usr/local/lib/python2.7/dist-packages
Cleaning up...
[localhost] local: sudo bash -c "pip install --upgrade pybedtools"
Downloading/unpacking pybedtools
Downloading pybedtools-0.6.2.tar.gz (18.1MB): 18.1MB downloaded
Running setup.py egg_info for package pybedtools

    ==================================================

    Please install Cython (http://cython.org/),
    which is required to build pybedtools. Usually
    you can do:

        pip install -U cython

    or

        easy_install -U cython

    ==================================================

Complete output from command python setup.py egg_info:


==================================================



Please install Cython (http://cython.org/),

which is required to build pybedtools. Usually

you can do:



    pip install -U cython



or



    easy_install -U cython



==================================================

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/pybedtools
Storing complete log in /home/bioinformatics/.pip/pip.log

Fatal error: local() encountered an error (return code 1) while executing 'sudo bash -c "pip install --upgrade pybedtools"'

Aborting.

bio-linux-assembly-conversion-tools failure on Debian

bio-linux-assembly-conversion-tools fails to install on Debian:

failure on dependencies: bio-linux-staden -> ia32-libs -> ia32-libs-i386

I try to install on 64 bits server.

localhost build pauses during anaconda install

The anaconda install script displays a big 'ol readme, which sits waiting for the person installing to hit enter...it'd be nice if it'd just continue on its own at that point :)

star compile fails on ubuntu 13.04

after doing a git pull, and a running install_biolinux:custom, I'm getting this:

DBG [fabfile.py]: Importing module cloudbio.custom.bio_nextgen
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.bio_nextgen.install_snap
INFO: Custom install for 'snap' end time: 2013-08-14 14:23:49.769315; duration: 0:00:00.007779
INFO: Custom install for 'star' start time: 2013-08-14 14:23:49.769535
DBG [fabfile.py]: Importing module cloudbio.custom.bio_nextgen
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.bio_nextgen.install_star
[localhost] local: echo $HOME
[localhost] local: tar -xzpf STAR_2.3.0e.tgz
[localhost] local: make
g++ -c -O3 -Wall -Wextra -fopenmp -D'SVN_VERSION_COMPILED="STAR_2.3.0e_r291"' -D'COMPILATION_TIME_PLACE="Wed Aug 14 09:23:49 CDT 2013 :/home/bioinformatics/tmp/cloudbiolinux/STAR_2.3.0e"' Genome.cpp
Genome.cpp: In member function ‘void Genome::genomeLoad(Parameters_)’:
Genome.cpp:157:21: error: ‘sleep’ was not declared in this scope
make: *_* [Genome.o] Error 1

Fatal error: local() encountered an error (return code 2) while executing 'make'

Aborting.
bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$

ValueError: zero length field name in format, bio_nextgen.py line 204

Brad,

Sorry dude, I'm going through the code, so I might bug you a lot :)

The version field is missing.
File: cloudbio/custom/bio_nextgen.py
Line: 204
"plinkseq-{}-x86_64.tar.gz".format(version) --> "plinkseq-{ver}-x86_64.tar.gz".format(ver=version)

centos64 install fails on leiningen

the leiningen install script uses HTTP_CLIENT before defining it.

Also, the download URL for leiningen ( https://leiningen.s3.amazonaws.com/downloads/leiningen-2.3.0-standalone.jar ) is currently returning a permission denied message:

AccessDeniedAccess DeniedCA6AB9AFD2F7134Fo1QysKTl9TVyV6Iuddj4bDusZ+O9H6IwEcMYBpo7EWEQ4RmZvGs0rk7S0/tq1ohK

dbsnp/rnaseq flags in biodata.yaml... don't die if not found

Hi Brad,

I might fix this one soon, but I'm leaving it here so that you know. PhiX does not have dbsnp nor rnaseq so it fails there halting the whole install_data process... instead of failing, a "this genome does not have this property" would be good to have:

$ fab -f data_fabfile.py -H biologin -c ~/config/fabricrc.txt install_data:~config/biodata/biodata.yaml
(...)
[biologin] run: mkdir -p genomes/phiX174/phix/variation
Traceback (most recent call last):
  File "roman/.virtualenvs/py2.7/lib/python2.7/site-packages/fabric/main.py", line 736, in main
    *args, **kwargs
  File "roman/.virtualenvs/py2.7/lib/python2.7/site-packages/fabric/tasks.py", line 314, in execute
    multiprocessing
  File "roman/.virtualenvs/py2.7/lib/python2.7/site-packages/fabric/tasks.py", line 211, in _execute
    return task.run(*args, **kwargs)
  File "roman/.virtualenvs/py2.7/lib/python2.7/site-packages/fabric/tasks.py", line 121, in run
    return self.wrapped(*args, **kwargs)
  File "roman/dev/cloudbiolinux/data_fabfile.py", line 61, in install_data
    genomes.install_data(config_source)
  File "roman/dev/cloudbiolinux/cloudbio/biodata/genomes.py", line 248, in install_data
    _install_additional_data(genomes, genome_indexes, config)
  File "roman/dev/cloudbiolinux/cloudbio/biodata/genomes.py", line 273, in _install_additional_data
    download_dbsnp(genomes, BROAD_BUNDLE_VERSION, DBSNP_VERSION)
  File "roman/dev/cloudbiolinux/cloudbio/biodata/dbsnp.py", line 34, in download_dbsnp
    _download_broad_bundle(manager.dl_name, bundle_version, dl_name, dl_ext)
AttributeError: NCBIRest instance has no attribute 'dl_name'

@b97pla

None type not iterable in s3_multipart_upload.py

My multi part upload did not complete due to the following error.

Traceback (most recent call last):
  File "/root/s3-backup//s3_multipart_upload.py", line 167, in <module>
    main(*args, **kwargs)
  File "/root/s3-backup//s3_multipart_upload.py", line 56, in main
    cores)
  File "/root/s3-backup//s3_multipart_upload.py", line 133, in _multipart_upload
    mp.complete_upload()
  File "/usr/lib/python2.6/site-packages/boto/s3/multipart.py", line 285, in complete_upload
    xml = self.to_xml()
  File "/usr/lib/python2.6/site-packages/boto/s3/multipart.py", line 150, in to_xml
    for part in self:
  File "/usr/lib/python2.6/site-packages/boto/s3/multipart.py", line 117, in part_lister
    for part in parts:
TypeError: 'NoneType' object is not iterable

when installing on centos 6.4, R scripts have a pathing error

in cloudbio/libraries.py , the variable outfile is set to "install_packages.R"
however, in a localhost install on centos 6.4, the outfile isn't being found.
forcing outfile to be "/tmp/install_packages.R" in the source file above appears to have resolved the issue.

ucsc_tools cannot stat ‘setupnx.sh’: No such file or directory

I have seen a few instances of installation problems such as this one on Ubuntu Server 13.04

INFO: Custom install for 'ucsc_tools' end time: 2013-08-02 21:47:34.389027; duration: 0:00:15.919216
DBG [config.py]: Using config file /home/ubuntu/cloudbiolinux/cloudbio/../config/chef_recipes.yaml
INFO: Reading /home/ubuntu/cloudbiolinux/cloudbio/../config/chef_recipes.yaml
DBG [shared.py]: Packages to install: 
DBG [config.py]: Using config file /home/ubuntu/cloudbiolinux/cloudbio/../config/puppet_classes.yaml
INFO: Reading /home/ubuntu/cloudbiolinux/cloudbio/../config/puppet_classes.yaml
DBG [shared.py]: Packages to install: 
[localhost] put: /home/ubuntu/cloudbiolinux/cloudbio/../config/../installed_files/setupnx.sh -> setupnx.sh
[localhost] local: sudo bash -c "mv setupnx.sh /usr/bin/setupnx.sh"
mv: cannot stat ‘setupnx.sh’: No such file or directory

Fatal error: local() encountered an error (return code 1) while executing 'sudo bash -c "mv setupnx.sh /usr/bin/setupnx.sh"'

Aborting.
Disconnecting from localhost... done.

Modifying the script to write to /tmp/setupnx.sh seems to help, it that the right approach or is there something else going on?

varianttools fail to install on centos 6.4

[jeremy@centos-try2 cloudbiolinux]$ sudo cat /root/.pip/pip.log

/usr/bin/pip run on Tue Aug 6 14:59:13 2013
Unpacking /home/jeremy/tmp/cloudbiolinux/variant_tools-1.0.6

Running setup.py egg_info for package from file:///home/jeremy/tmp/cloudbiolinux/variant_tools-1.0.6

Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/tmp/pip-MYTxn9-build/setup.py", line 45, in <module>

    from source import VTOOLS_VERSION

  File "source/__init__.py", line 30, in <module>

    if (ver.major == 2 and (ver.minor, ver.micro) < (7, 1)) or (ver.major == 3 and (ver.minor, ver.micro) < (2, 0)):

AttributeError: 'tuple' object has no attribute 'major'

Complete output from command python setup.py egg_info:

Traceback (most recent call last):

File "", line 16, in

File "/tmp/pip-MYTxn9-build/setup.py", line 45, in

from source import VTOOLS_VERSION

File "source/init.py", line 30, in

if (ver.major == 2 and (ver.minor, ver.micro) < (7, 1)) or (ver.major == 3 and (ver.minor, ver.micro) < (2, 0)):

AttributeError: 'tuple' object has no attribute 'major'

Cleaning up...

Command python setup.py egg_info failed with error code 1 in /tmp/pip-MYTxn9-build

Exception information:
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/req.py", line 1124, in prepare_files
req_to_install.run_egg_info()
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/req.py", line 257, in run_egg_info
command_desc='python setup.py egg_info')
File "/usr/lib/python2.6/site-packages/pip-1.4-py2.6.egg/pip/util.py", line 670, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip-MYTxn9-build

ubuntu 13.04 install failing on rpy

I've got this going on on all of my VM installs now...

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ cat /tmp/tmpN9ZdOp

/usr/anaconda/bin/pip run on Sun Aug 25 14:54:54 2013
Unpacking /home/bioinformatics/tmp/cloudbiolinux/rpy-1.0.3

Running setup.py egg_info for package from file:///home/bioinformatics/tmp/cloudbiolinux/rpy-1.0.3

RHOMES= []

DEBUG= True

Setting RHOMES to  ['/usr/lib/R']

### Using R verion 3.0.1 installed at /usr/lib/R ###

RHOME= /usr/lib/R

copying src/rpymodule.c -> src/rpymodule3001.c
copying src/R_eval.c -> src/R_eval3001.c
copying src/io.c -> src/io3001.c
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

   or: -c --help [cmd1 cmd2 ...]

   or: -c --help-commands

   or: -c cmd --help



error: invalid command 'egg_info'

Complete output from command python setup.py egg_info:

RHOMES= []

DEBUG= True

Setting RHOMES to ['/usr/lib/R']

Using R verion 3.0.1 installed at /usr/lib/R

RHOME= /usr/lib/R

copying src/rpymodule.c -> src/rpymodule3001.c

copying src/R_eval.c -> src/R_eval3001.c

copying src/io.c -> src/io3001.c

usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

or: -c --help [cmd1 cmd2 ...]

or: -c --help-commands

or: -c cmd --help

error: invalid command 'egg_info'

Cleaning up...

Removing temporary dir /tmp/pip_build_bioinformatics...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-F0YM3k-build

Exception information:
Traceback (most recent call last):
File "/usr/anaconda/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/usr/anaconda/lib/python2.7/site-packages/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/anaconda/lib/python2.7/site-packages/pip/req.py", line 1134, in prepare_files
req_to_install.run_egg_info()
File "/usr/anaconda/lib/python2.7/site-packages/pip/req.py", line 259, in run_egg_info
command_desc='python setup.py egg_info')
File "/usr/anaconda/lib/python2.7/site-packages/pip/util.py", line 670, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip-F0YM3k-build
bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$

wrong url for Debian with watson site

In distribution, there is an error with watson.nci.nig.gov

Debian URL is:

http://watson.nci.nih.gov/cran_mirror/bin/linux/debian/squeeze-cran/

instead of

http://watson.nci.nih.gov/cran_mirror/bin/linux/debian/squeeze/

Scripts show error:

[192.168.2.29] out: W: Failed to fetch http://watson.nci.nih.gov/cran_mirror/bin/linux/debian/squeeze/Packages 404 Not Found

centos 6.4 sudo configuration interferes with customized pip usage

Because CentOS 6.4 uses the secure_path option in /etc/sudoers, attempting to call pip via sudo will always resolve to /usr/bin/pip -- the old version, which causes failures in the varianttools build (among other things).

The following sed command will resolve this for the install:

sed -i -e 's/(secure_path = )(.*)$/\1/usr/anaconda/bin:\2/g' /etc/sudoers

perhaps this could be run automatically before a localhost install goes?

After the install, this will clean out that entry:

sed -i -e 's//usr/anaconda/bin:*//g' /etc/sudoers

(if that's even necessary)

Alternately, if pip is always called with absolute pathname (e.g /usr/anaconda/bin/pip), then the assumptions made by sudo are (for this issue) irrelevant.

I'm continuing to test with this fix, but it appears to have resolved the pip-related build issues.

cloudbiolinx does not build

I am trying to set-up a vagrant cloudbiolunux box from scratch.

However the process fails in multiple places hours into the operation. Some of the errors I was able to fix, but I gave up after the haskell issues (I use a 64bit Ubuntu 12.04 base box).

fab -f $cb/fabfile.py -H vagrant  -c $cb/config/fabricrc.txt install_biolinux:packagelist=$cb/config/main.yaml

Points of failure:

in main.yaml
- the Tracer packages cannot be downloaded
- some haskell tools software compiles bud does not install due to library conflicts resulting in an abort
in cloudbio/galaxy/tools.py
- env does not contain the "galaxy_tools_conf" variable and the default: tools_conf_path = env.get("galaxy_tools_conf", "contrib/cloudman/tools.yaml") is invalid when fab is executed from an vagrant directory.
in cloudbio/custom/shared.py
- two missing imports

from tempfile import NamedTemporaryFile
import urllib

(referenced at line 212 and 215)

galaxy install fails due to hardcoded UID

galaxy_webapp hardcodes user UID to 1001 -- which is the default UID for the first user added to an ubuntu system. Perhaps the code should check to see if 1001 is taken already? alternately, 10001 is less likely to have been taken (but still has the same core issue).

INFO: Custom install for 'galaxy_webapp' start time: 2013-09-19 19:49:50.598744
DBG [fabfile.py]: Importing module cloudbio.custom.galaxy
DBG [fabfile.py]: Looking for custom install function cloudbio.custom.galaxy.install_galaxy_webapp
[localhost] local: sudo http_proxy=$http_proxy https_proxy=$https_proxy bash -c "useradd -d /home/galaxy --create-home --shell /bin/bash -c"Galaxy-required user" --uid 1001 --user-group galaxy"
useradd: UID 1001 is not unique

Fatal error: local() encountered an error (return code 4) while executing 'sudo http_proxy=$http_proxy https_proxy=$https_proxy bash -c "useradd -d /home/galaxy --create-home --shell /bin/bash -c"Galaxy-required user" --uid 1001 --user-group galaxy"'

Aborting.

chapmanb / cloudbiolinux Goto Github PK

cloudbiolinux's People

Contributors

Stargazers

Watchers

Forkers

cloudbiolinux's Issues

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ cat /tmp/tmpe7O2t1

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ cat /home/bioinformatics/.pip/pip.log

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

All requested packages already installed.

packages in environment at /usr/anaconda:

[jeremy@centos-try2 cloudbiolinux]$ sudo cat /root/.pip/pip.log

[jeremy@centos-try2 cloudbiolinux]$ sudo cat /root/.pip/pip.log

bioinformatics@bioinformatics-VirtualBox:~/sandbox/cloudbiolinux$ cat /tmp/tmpN9ZdOp

Using R verion 3.0.1 installed at /usr/lib/R

Recommend Projects

Recommend Topics

Recommend Org