galaxyproject/tools-iuc

Tool Shed repositories maintained by the Intergalactic Utilities Commission

Galaxy Tools maintained by IUC

This repo contains a subset of Galaxy repositories used in the Tool Shed (https://toolshed.g2.bx.psu.edu/).

These repositories are maintained and developed by the Intergalactic Utilities Commission (Github Org)

Pull Requests with dependencies specified as conda-package will be automatically tested and verified using planemo. If the tests pass and the pull request is merged, the tool will be automatically uploaded to the Test- and Main Tool Shed.

Other repositories with Galaxy tools:

• Björn Grüning repo
• Galaxy devteam repo
• Peter Cock's repos:
  • blast repo
  • pico repo
  • mira repo
• ENCODE tools
• Biopython repo
• Galaxy Proteomics repo
• Colibread Galaxy Tools
• Greg von Kuster's repo
• EI repo
• AAFC-MBB Canada repo
• Mark Einon's repo
• National Microbiology Laboratory's repo
• RECETOX's repo
• Workflow4Metabolomics' repo
• Galaxy-M

tools-iuc's Issues

Gemini Error when trying to load a VCF against Data Manager loaded Gemini Data

See Galaxy bug report: https://test.galaxyproject.org/dataset/errors?id=e0efdf8bfb70378a

cmd line for the load includes the --annotation-dir (should be ok before the 'load' command?):

gemini --annotation-dir /galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data load -v "/galaxy/test/files/000/512/dataset_512029.dat" -t snpEff --cores ${GALAXY_SLOTS:-4} "/galaxy/test/files/000/512/dataset_512044.dat"

The directory exists and has the info, but the job fails with Exit Code 1, due to attempting to load several files from the original installed package dir.

CADD scores are being loaded (to skip use:--skip-cadd).
GERP per bp is being loaded (to skip use:--skip-gerp-bp).
Gemini cannot open this annotation file: /galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gwas.bed.gz. 
Have you installed the annotation files?  If so, have they been moved or deleted? Exiting...

where /galaxy/test/jobdir/000/349/349430/(dataset_511255_files) is the the job working directory of the Gemini Data Manager tool run that retrieved the reference data that we are populating.

If we look at the gemini-config.yaml in the installed tool_deps dir, it does have annotation_dir: /galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data, which should be overridden by --annotation-dir.

some quick print statements into the gemini code itself, indicate that the first half of annotation loading might be using the coded .config value instead of the provided --annotation-dir value.

load args Namespace(anno_type='snpEff', annotation_dir=None, command='load_chunk', db='/galaxy/test/files/000/512/dataset_512029.dat.chunk1.db', func=<function loadchunk_fn at 0x7f61c37377d0>, no_genotypes=False, no_load_genotypes=False, offset='10', passonly=False, ped_file=None, skip_cadd=False, skip_gene_tables=False, skip_gerp_bp=False, skip_info_string=False, test_mode=False, vcf='-')
anno_files {'gwas': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gwas.bed.gz', 'gerp_elements': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gerp.elements.bed.gz', 'pfam_domain': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.pfam.ucscgenes.bed.gz', 'conserved': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/29way_pi_lods_elements_12mers.chr_specific.fdr_0.1_with_scores.txt.hg19.merged.bed.gz', 'dbsnp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/dbsnp.138.vcf.gz', 'clinvar': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/clinvar_20140303.vcf.gz', 'esp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ESP6500SI.all.snps_indels.vcf.gz', 'cse': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/cse-hiseq-8_4-2013-02-20.bed.gz', 'cpg_island': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.CpG.bed.gz', 'recomb': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/genetic_map_HapMapII_GRCh37.gz', 'grc': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRC_patch_regions.bed.gz', 'gms': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRCh37-gms-mappability.vcf.gz', 'segdup': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.segdup.bed.gz', '1000g': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ALL.wgs.integrated_phase1_v3.20101123.snps_indels_sv.sites.2012Oct12.vcf.gz', 'dgv': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.dgv.bed.gz', 'encode_dnase1': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/stam.125cells.dnaseI.hg19.bed.gz', 'cytoband': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cytoband.bed.gz', 'rmsk': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.rmsk.bed.gz', 'cosmic': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cosmic.v67.20131024.gz', 'encode_tfbs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/wgEncodeRegTfbsClusteredV2.cell_count.20130213.bed.gz', 'vista_enhancers': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.vista.enhancers.20131108.bed.gz', 'encode_consensus_segs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/encode.6celltypes.consensus.bedg.gz'}
load args Namespace(anno_type='snpEff', annotation_dir=None, command='load_chunk', db='/galaxy/test/files/000/512/dataset_512029.dat.chunk3.db', func=<function loadchunk_fn at 0x7f2d30a2f7d0>, no_genotypes=False, no_load_genotypes=False, offset='28', passonly=False, ped_file=None, skip_cadd=False, skip_gene_tables=False, skip_gerp_bp=False, skip_info_string=False, test_mode=False, vcf='-')
anno_files {'gwas': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gwas.bed.gz', 'gerp_elements': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gerp.elements.bed.gz', 'pfam_domain': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.pfam.ucscgenes.bed.gz', 'conserved': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/29way_pi_lods_elements_12mers.chr_specific.fdr_0.1_with_scores.txt.hg19.merged.bed.gz', 'dbsnp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/dbsnp.138.vcf.gz', 'clinvar': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/clinvar_20140303.vcf.gz', 'esp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ESP6500SI.all.snps_indels.vcf.gz', 'cse': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/cse-hiseq-8_4-2013-02-20.bed.gz', 'cpg_island': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.CpG.bed.gz', 'recomb': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/genetic_map_HapMapII_GRCh37.gz', 'grc': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRC_patch_regions.bed.gz', 'gms': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRCh37-gms-mappability.vcf.gz', 'segdup': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.segdup.bed.gz', '1000g': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ALL.wgs.integrated_phase1_v3.20101123.snps_indels_sv.sites.2012Oct12.vcf.gz', 'dgv': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.dgv.bed.gz', 'encode_dnase1': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/stam.125cells.dnaseI.hg19.bed.gz', 'cytoband': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cytoband.bed.gz', 'rmsk': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.rmsk.bed.gz', 'cosmic': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cosmic.v67.20131024.gz', 'encode_tfbs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/wgEncodeRegTfbsClusteredV2.cell_count.20130213.bed.gz', 'vista_enhancers': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.vista.enhancers.20131108.bed.gz', 'encode_consensus_segs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/encode.6celltypes.consensus.bedg.gz'}
load args Namespace(anno_type='snpEff', annotation_dir=None, command='load_chunk', db='/galaxy/test/files/000/512/dataset_512029.dat.chunk0.db', func=<function loadchunk_fn at 0x7f9bf24e37d0>, no_genotypes=False, no_load_genotypes=False, offset='1', passonly=False, ped_file=None, skip_cadd=False, skip_gene_tables=False, skip_gerp_bp=False, skip_info_string=False, test_mode=False, vcf='-')
anno_files {'gwas': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gwas.bed.gz', 'gerp_elements': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gerp.elements.bed.gz', 'pfam_domain': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.pfam.ucscgenes.bed.gz', 'conserved': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/29way_pi_lods_elements_12mers.chr_specific.fdr_0.1_with_scores.txt.hg19.merged.bed.gz', 'dbsnp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/dbsnp.138.vcf.gz', 'clinvar': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/clinvar_20140303.vcf.gz', 'esp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ESP6500SI.all.snps_indels.vcf.gz', 'cse': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/cse-hiseq-8_4-2013-02-20.bed.gz', 'cpg_island': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.CpG.bed.gz', 'recomb': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/genetic_map_HapMapII_GRCh37.gz', 'grc': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRC_patch_regions.bed.gz', 'gms': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRCh37-gms-mappability.vcf.gz', 'segdup': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.segdup.bed.gz', '1000g': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ALL.wgs.integrated_phase1_v3.20101123.snps_indels_sv.sites.2012Oct12.vcf.gz', 'dgv': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.dgv.bed.gz', 'encode_dnase1': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/stam.125cells.dnaseI.hg19.bed.gz', 'cytoband': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cytoband.bed.gz', 'rmsk': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.rmsk.bed.gz', 'cosmic': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cosmic.v67.20131024.gz', 'encode_tfbs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/wgEncodeRegTfbsClusteredV2.cell_count.20130213.bed.gz', 'vista_enhancers': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.vista.enhancers.20131108.bed.gz', 'encode_consensus_segs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/encode.6celltypes.consensus.bedg.gz'}
load args Namespace(anno_type='snpEff', annotation_dir=None, command='load_chunk', db='/galaxy/test/files/000/512/dataset_512029.dat.chunk2.db', func=<function loadchunk_fn at 0x7f8f17fee7d0>, no_genotypes=False, no_load_genotypes=False, offset='19', passonly=False, ped_file=None, skip_cadd=False, skip_gene_tables=False, skip_gerp_bp=False, skip_info_string=False, test_mode=False, vcf='-')
anno_files {'gwas': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gwas.bed.gz', 'gerp_elements': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.gerp.elements.bed.gz', 'pfam_domain': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.pfam.ucscgenes.bed.gz', 'conserved': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/29way_pi_lods_elements_12mers.chr_specific.fdr_0.1_with_scores.txt.hg19.merged.bed.gz', 'dbsnp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/dbsnp.138.vcf.gz', 'clinvar': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/clinvar_20140303.vcf.gz', 'esp': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ESP6500SI.all.snps_indels.vcf.gz', 'cse': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/cse-hiseq-8_4-2013-02-20.bed.gz', 'cpg_island': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.CpG.bed.gz', 'recomb': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/genetic_map_HapMapII_GRCh37.gz', 'grc': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRC_patch_regions.bed.gz', 'gms': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/GRCh37-gms-mappability.vcf.gz', 'segdup': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.segdup.bed.gz', '1000g': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/ALL.wgs.integrated_phase1_v3.20101123.snps_indels_sv.sites.2012Oct12.vcf.gz', 'dgv': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.dgv.bed.gz', 'encode_dnase1': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/stam.125cells.dnaseI.hg19.bed.gz', 'cytoband': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cytoband.bed.gz', 'rmsk': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.rmsk.bed.gz', 'cosmic': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.cosmic.v67.20131024.gz', 'encode_tfbs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/wgEncodeRegTfbsClusteredV2.cell_count.20130213.bed.gz', 'vista_enhancers': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/hg19.vista.enhancers.20131108.bed.gz', 'encode_consensus_segs': '/galaxy/test/jobdir/000/349/349430/dataset_511255_files/gemini/data/encode.6celltypes.consensus.bedg.gz'}
anno_files {'gwas': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.gwas.bed.gz', 'gerp_elements': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.gerp.elements.bed.gz', 'pfam_domain': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.pfam.ucscgenes.bed.gz', 'conserved': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/29way_pi_lods_elements_12mers.chr_specific.fdr_0.1_with_scores.txt.hg19.merged.bed.gz', 'dbsnp': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/dbsnp.138.vcf.gz', 'clinvar': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/clinvar_20140303.vcf.gz', 'esp': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/ESP6500SI.all.snps_indels.vcf.gz', 'cse': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/cse-hiseq-8_4-2013-02-20.bed.gz', 'cpg_island': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.CpG.bed.gz', 'recomb': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/genetic_map_HapMapII_GRCh37.gz', 'grc': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/GRC_patch_regions.bed.gz', 'gms': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/GRCh37-gms-mappability.vcf.gz', 'gerp_bp': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.gerp.bw', 'segdup': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.segdup.bed.gz', '1000g': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/ALL.wgs.integrated_phase1_v3.20101123.snps_indels_sv.sites.2012Oct12.vcf.gz', 'dgv': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.dgv.bed.gz', 'encode_dnase1': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/stam.125cells.dnaseI.hg19.bed.gz', 'cytoband': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.cytoband.bed.gz', 'rmsk': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.rmsk.bed.gz', 'cosmic': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.cosmic.v67.20131024.gz', 'encode_tfbs': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/wgEncodeRegTfbsClusteredV2.cell_count.20130213.bed.gz', 'vista_enhancers': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/hg19.vista.enhancers.20131108.bed.gz', 'cadd_score': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/whole_genome_SNVs.tsv.compressed.gz', 'encode_consensus_segs': '/galaxy-repl/manageddata/data/gemini/data/hg19/2015-02-16/gemini/data/encode.6celltypes.consensus.bedg.gz'}
Loading with existing bgzip (/galaxy/test/files/000/512/dataset_512029.dat.gz) version of /galaxy/test/files/000/512/dataset_512029.dat.

Not sure if we can fix this on the Galaxy end. It could also be solved upstream.

Bugs from Jenkins+Planemo Testing

rgDifferentialCount

@fubar2

• also need to run through xmllint --pretty 1
• Given rgedgeRpaired_nocamera.xml, should rgedgeRpaired.xml.camera be .camera.xml?

Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/var/lib/jenkins/galaxy/default/test/functional/test_toolbox.py", line 268, in test_tool
    self.do_it( td )
  File "/var/lib/jenkins/galaxy/default/test/functional/test_toolbox.py", line 70, in do_it
    raise e
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 127: ordinal not in range(128)

gatk2_base_recalibrator

@bgruening @nsoranzo

Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/var/lib/jenkins/galaxy/default/test/functional/test_toolbox.py", line 268, in test_tool
    self.do_it( td )
  File "/var/lib/jenkins/galaxy/default/test/functional/test_toolbox.py", line 55, in do_it
    raise e
RunToolException: Error creating a job for these tool inputs - {u'type': u'error', u'data': {u'covariates': u"An invalid option was selected for covariates, u'ReadGroupCovariate', please verify."}}

gatk2_haploytype_caller

RunToolException: Error creating a job for these tool inputs - {u'type': u'error', u'data': {u'dbsnp_rod_bind_type': {u'dbsnp_input_rod': u'History does not include a dataset of the required format / build'}}}

bedtools_closestbed

@bgruening, strange error, IMO. It's bedtools 2.22 not 2.21, but that shouldn't cause a -3 all of a sudden.

JobOutputsError: History item  different than expected, difference (using diff):
( /var/lib/jenkins/workspace/IUC-Github/tools/bedtools/test-data/closestBed_result4.bed v. /tmp/tmpjL76HD/tmp/tmpHWM575closestBed_result4.bed )
--- local_file
+++ history_data
@@ -1,2 +1,2 @@
 chr1   10  20  a1  1   +   chr1    7   8   b1  1   +   -3
-chr1   10  20  a1  1   +   chr1    22  23  b2  2   -   3
+chr1   10  20  a1  1   +   chr1    22  23  b2  2   -   -3

For tests 3, 4

gemini

Just a missing datatype definition here...

gemini_windower, gemini_stats, gemini_roh, gemini_region, gemini_recessive_and_dominant, ....

RunToolException: Error creating a job for these tool inputs - {u'type': u'error', u'data': {u'infile': u'History does not include a dataset of the required format / build'}}

What needs to be done for snapped/snpsift v4.0 to be put on main toolshed?

Automatically pushing repos to sheds

Question for @galaxyproject/iuc:

With planemo and the -r option, we can push the entire repo automatically to various places. I'd like to do this on a regular basis (as a jenkins job, every night or so), and I had a thought that I'd like to put to the IUC for comments.

Originally I'd wanted to push master to the TTS, but then I realised that I wouldn't be able to test things in other branches I'm working on without explicit work on my end to push individual packages/branches.

Instead I'd like to propose:

• push PRs to the TTS automatically so people can test packages that next morning
• push master to the MTS automatically to ensure we're not failing to keep things updated

My justification to automatically pushing master to the to MTS is that when things have been merged into master, they've been explicitly approved as a working package/tool by the IUC (otherwise they shouldn't be in master).

These would all be automated as a jenkins job, I can have it mail the IUC mailing list whenever things are updated if people would like to keep tabs on it.

Thoughts, comments, +/-1s?

update bedtools

bedtools needs an update

• update to version 2.22
• add new features introduced in latest version (e.g. fisher tool)
• add CDATA tag
• add more tests
• -g option should not take a tabular file but should reuse genome.len files from Galaxy (${chromInfo} plus a validator on input file)

Issues discovered from whole git repo push to TTS

Round 1

• package_biopython_1_61 contains a readme.rst (fc79e42)
• package_dill_0.2 has a period (006abbf)
• package_faToTwoBit_35x1 has uppercase letters (same for gfClient/gfServer) (5fa42e4, 2b256a8, 3db286e)
• package_graphclust_0_7_6 has invalid XML (53172da)
• package_tabix had bad yaml (f04f099)
• package_ballgown_1_0_3 is missing a description
• tools/gemini had invalid YAML (21965f9)
• package_samtools_0_1_18 missing description (afa0414)
• package_samtools_0_1_16 missing description (afa0414)
• package_samtools_0_1_2 missing description (afa0414)
• package_ucsc_tools_0_1 missing description (ec17626)
• tool_factory_2 has invalid dependency on package_graphicsmagick_1_3_20 and owner iuc. #90, #89

Round 2

• tool_factory_2: "Could not find repository in toolshed" then "You already own a repository named tool_factory_2, please choose a different name" (#97)
• snpsift_* depend on snpsift_dbnsfp_datatypes repository, which isn't in TTS. (#95)

Other than that we're good. I'll probably handle some of the above tomorrow.

CC @bgruening @martenson

Gemini Load Writes to the database/files dir

This can be solved in the tool by using a symlink, or possibly by using bgzip and tabix converters.

Current behavior as follows on an Input VCF dataset:

$ ls -lah /path/to/files/dataset_1234.dat*
-rw-r--r-- 1 galaxy G-12345  17K Feb 18 09:42 /path/to/files/dataset_1234.dat

Run Tool

Tool running:

$ ls -lah /path/to/files/dataset_1234.dat*
-rw-r--r-- 1 galaxy G-12345  17K Feb 18 09:42 /path/to/files/dataset_1234.dat
-rw-r--r-- 1 galaxy G-12345  24K Feb 26 09:50 /path/to/files/dataset_1234.dat.chunk0.db
-rw-r--r-- 1 galaxy G-12345  24K Feb 26 09:50 /path/to/files/dataset_1234.dat.chunk1.db
-rw-r--r-- 1 galaxy G-12345  24K Feb 26 09:50 /path/to/files/dataset_1234.dat.chunk2.db
-rw-r--r-- 1 galaxy G-12345  24K Feb 26 09:50 /path/to/files/dataset_1234.dat.chunk3.db
-rw-r--r-- 1 galaxy G-12345 3.5K Feb 26 09:50 /path/to/files/dataset_1234.dat.gz
-rw-r--r-- 1 galaxy G-12345   13 Feb 26 09:50 /path/to/files/dataset_1234.dat.gz.gbi

Running a little longer

$ ls -lah /path/to/files/dataset_1234.dat*
-rw-r--r-- 1 galaxy G-12345  17K Feb 18 09:42 /path/to/files/dataset_1234.dat
-rw-r--r-- 1 galaxy G-12345 138M Feb 26 09:51 /path/to/files/dataset_1234.dat.chunk0.db
-rw-r--r-- 1 galaxy G-12345 138M Feb 26 09:51 /path/to/files/dataset_1234.dat.chunk1.db
-rw-r--r-- 1 galaxy G-12345 138M Feb 26 09:51 /path/to/files/dataset_1234.dat.chunk2.db
-rw-r--r-- 1 galaxy G-12345 138M Feb 26 09:51 /path/to/files/dataset_1234.dat.chunk3.db
-rw-r--r-- 1 galaxy G-12345 3.5K Feb 26 09:50 /path/to/files/dataset_1234.dat.gz
-rw-r--r-- 1 galaxy G-12345   13 Feb 26 09:50 /path/to/files/dataset_1234.dat.gz.gbi

Done:

$ ls -lah /path/to/files/dataset_1234.dat*
-rw-r--r-- 1 galaxy G-12345  17K Feb 18 09:42 /path/to/files/dataset_1234.dat
-rw-r--r-- 1 galaxy G-12345 3.5K Feb 26 09:50 /path/to/files/dataset_1234.dat.gz
-rw-r--r-- 1 galaxy G-12345   13 Feb 26 09:50 /path/to/files/dataset_1234.dat.gz.gbi

host visualization plugins

@bgruening had the great idea to host visualization plugins inside this repo (until the ToolShed supports thems).

-> would you be interested (a) in hosting visualization plugins in this repo or (b) do you want to create a separate repo (e.g. vis-iuc) or (c) do you have another idea?

Minimal list of contributors with visualization plugins:

• @saketc
• @carlfeberhard
• @greenify
  [...]

For more infos #71.

scipy installation fails on missing blas

As a part of deepTools installation on Test Galaxy the scipy package needed to be installed and failed with the following error:
https://gist.github.com/martenson/b77145f404f52f0ddc7b

Setting up TravisCI continuous integration for IUC tools

Suggested by @bgruening, it would be great to have the IUC tools tested automatically using TravisCI.

Note that my setup described here http://blastedbio.blogspot.jp/2013/09/using-travis-ci-for-testing-galaxy-tools.html as used on https://github.com/peterjc/galaxy_blast and https://github.com/peterjc/pico_galaxy does not use the Tool Shed packages - it "manually" installs the dependencies from source or using "apt-get install".

What would be ideal for the IUC tools (and my own tools in the long term) would be to have the TravisCI script setup a local Galaxy and a local Tool Shed, and use this to test our IUC packages as well.

python packages have no ssl support currently

I have some problems with our python packages, and probably all packages that are linking to the openssl package. Currently, the openssl package does not include any certificates which makes it useless for https transfers etc.

Building bx-python, which depends on python-2.7, will fail because it is not able to fetch some library via https.

We need to add somehow certificates to the openssl package to make this work. An other possibility would be to simply drop the openssl library dependency and rely on the system one. This was previously discussed here: #76

Any other ideas? docker-build with relocatable python binaries + certificates?
ping @natefoo

Make sure toolform_upgrade = False

In the config/galaxy.ini , the parameter for toolform_upgrade has to be set to be "False" for tool_factory_2 to work.

Use new tool form

toolform_upgrade = False

Meeting about Tool Development (December 2014)

I am going to throw out 2014-12-16 at 10:00 EDT / 15:00 GMT?

Lets make a list of topics to discuss here - my initial suggestions would be.

• Tool best practice guidelines (see https://wiki.galaxyproject.org/Tools/BestPractices).
• Versioning (of tool wrappers) (galaxyproject/tools-devteam#16)
• Tool Shed YAML manifest (see also galaxyproject/tools-devteam#9)
• Priorities for tools in 2015, how do we get stuff onto usegalaxy.org.
• Developer-convenient way to connect GH and TS repositories (added by Martin)
• Removing Test TS (added by Martin)
• Ideal Testing framework (added by Martin)
• GSoC?

package_fatotwobit_35x1 has bad download links

File: https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_fatotwobit_35x1/tool_dependencies.xml

refers to https://github.com/bgruening/download_store/raw/master/ucsc/...., which does not exist.

Is in the MTS at https://toolshed.g2.bx.psu.edu/view/iuc/package_fatotwobit_35x1

Was going to use for devteam's twobit data manager.

R package for limma

I noticed that I didn't finish nicely packaging this tool of mine (despite using it in published workflows):

• https://toolshed.g2.bx.psu.edu/view/peterjc/venn_list/
• https://testtoolshed.g2.bx.psu.edu/view/peterjc/venn_list/
• https://github.com/peterjc/pico_galaxy/tree/master/tools/venn_list

It needs the R package limma for plotting (and currently uses rpy too, which is another issue), which is not yet in the ToolShed as an IUC package. However, I spotted this in the Test Tool Shed (only):

• https://testtoolshed.g2.bx.psu.edu/view/tmcgowan/limma

Can we (the IUC) have an official limma package?

BioJS-MSA doesn't use galaxy prefix

If galaxy is running with a prefix like /galaxy/, BioJS-MSA is not aware of this.

Flip on Travis for repository

Okay, I screwed up here, and meant to open this as a PR, but then I committed to master and pushed before I checked my work. My bad, won't happen again. (Github really needs protected branches) That said, I push the .travis.yml file copied straight from tools-devteam, so...very little harm done.

Anyway, back on topic, since we're the IUC and love our standards SO much, we now have a .travis.yml config for linting all of the tools. 6e6439f Would it be possible for one of the galaxyproject admins (or someone with proper permissions) to turn on Travis for this repository?

IUC Packaging TODO List/Wish List

Galaxy Tools/Wrappers/Packages

In the interests of tracking things which we want to package or see included, here's a TODO list for the IUC. These can also make good projects for volunteers :)

Please feel free to add packages to this list and check them off whenever they get completed!

Biological

• RAMPART
• Circos
  • In Progress
• CRAM (under tools-devteam)
• Phinch
• A2m Alignment Format for HMMER
• CONTRAfold
• Some sort of Ontology Browser/Visualizer? I've been testing WebProtege but that's a full featured editor and may be a bit heavy weight for something as simple as .obo viz
• TMPred
• PEAR Done #45
• ProgressiveMauve (+ Mauve JS Viz) WIP
• Sealer/medusa https://github.com/bcgsc/abyss/tree/sealer-release / https://github.com/combogenomics/medusa

  Suites

• Seqtk availability: MTS
• GenomeTools GFF3 toolkit
  • In Progress
• HMMER
  • WIP
• QIIME
  • existing tools that need updates/cleanup.
  • dunno what this is
  • WIP
• PacBio Tools
  • Progress on BLASR, and somewhere on my hard drive I had a first stab at converting their workflows to galaxy JSON workflows. Just message me (@erasche) for them.

Imaging/Microscopy

• FFMpeg for microscopy video data.
• ImageMagick's convert for imaging data
  • progress with support for rotate/flip/format conversion
• CellProfiler

Text Analysis

• JsonPath

Data Sources

• CKAN

TPP Package issues

https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_tpp_4_8_0/tool_dependencies.xml#L5 actions need to be in an <action_group/>. Thanks @jmchilton

Move khmer wrappers here

See https://github.com/ged-lab/khmer/tree/master/scripts/galaxy

(This is the appropriate place, yes?)

EMBOSS tools needs an update

We need a new version of all EMBOSS tools. We should write a converter from EMBOSS ACD files to Galaxy xml files.

Please get in touch with us if you are interested in this project.

Numpy 1.7 dies due to missing libf77blas.so.3

When attempting to use MACS2 (which depends on numpy 1.7) I get the following error:

ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory

Atlas seems to be installed just fine, but it doesn't seem like that env.sh is sourced except during numpy installation. Not sure if that's the issue, but seems like it might be.

Google Summer of Code 2015

Should we try to get some coders to work on tool or workflow integration projects? Is that realistic and if so what is involved? If it is feasible lets collect some ideas here.

Do people have wishlists with interesting projects or would they be willing to mentor? I could outline some stuff for proteomics building on work by @bgruening and the Galaxy-P project.

http://www.google-melange.com/gsoc/homepage/google/gsoc2015

MEME

@bgruening jelle_ was asking about this on IRC.

You seem to have a meme package in the TTS, but while the development repo is listed as this one but there's no meme package here.

Additionally, are you planning on writing wrappers for it?

Needed:

• package #75
• tools

package_mummer_3_23 missing on GitHub

We have https://toolshed.g2.bx.psu.edu/view/iuc/package_mummer_3_23 and https://testtoolshed.g2.bx.psu.edu/view/iuc/package_mummer_3_23 on the Tool Sheds under the IUC account, but the package source is not yet here under https://github.com/galaxyproject/tools-iuc/tree/master/packages

I don't see it under https://github.com/bgruening/galaxytools/tree/master/packages with @bgruening's other packages. The tool shed commit notes the original was by @jj-umn

We could just commit the current package to git as-is from the ToolShed?

[The reason I was looking at this is I want to declare gnuplot as a dependency, see PR #7]

Tool Developers Meeting (February 2015)

At the end of the last Tool Developers meeting we decided to make it a bi-monthly thing. I have put together a Doodle poll with some time slots. It is still a while from now so probably best to fill out the poll closer to that date but I wanted to make sure it was on the radar.

http://doodle.com/q634w4ae2nhgaf84

Once we have decided on a time slot we can post of galaxy-dev (which we should have done last time).

Including MSA Formats in PR to Galaxy

In the past comments seemed to suggest it is easier to deal with datatypes in galaxy than separately. (E.g. galaxyproject/galaxy#148)

I need HMMER3 datatypes, so I'm going to open a PR shortly against Galaxy for including those. Would the IUC have any objections to having those included in the PR?

Gemini package requires a configured $HOME when run.

Home directory should not be assumed, but in testing, having the package set HOME to e.g. '.' for the working directory seemed to work OK.

Here is original error

Fatal error: Exit code 1 (Error)
Traceback (most recent call last):
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/gemini/gemini/install-data.py", line 169, in <module>
    install_annotation_files(args.anno_dir, args.dl_files, args.extra)
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/gemini/gemini/install-data.py", line 78, in install_annotation_files
    cur_config = read_gemini_config(allow_missing=True)
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/anaconda/lib/python2.7/site-packages/gemini/config.py", line 47, in read_gemini_config
    fname = _get_config_file(dirs, use_globals=use_globals)
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/anaconda/lib/python2.7/site-packages/gemini/config.py", line 35, in _get_config_file
    dnames = dirs + get_config_dirs(use_globals=use_globals)
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/anaconda/lib/python2.7/site-packages/gemini/config.py", line 30, in get_config_dirs
    dirs.append(os.path.join(os.environ["HOME"], ".gemini"))
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/anaconda/lib/python2.7/UserDict.py", line 23, in __getitem__
    raise KeyError(key)
KeyError: 'HOME'
Traceback (most recent call last):
  File "/galaxy/test/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/data_manager_gemini_database_downloader/a580422f6f1d/data_manager_gemini_database_downloader/data_manager/data_manager_gemini_download.py", line 34, in <module>
    main()
  File "/galaxy/test/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/data_manager_gemini_database_downloader/a580422f6f1d/data_manager_gemini_database_downloader/data_manager/data_manager_gemini_download.py", line 20, in main
    ret = subprocess.check_call( cmd, shell=True )
  File "/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/anaconda/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/galaxy/test/deps/gemini/0.10.0/iuc/package_gemini_0_10_0/42c72725f879/gemini/gemini/install-data.py --extra gerp_bp --extra cadd_score /galaxy/test/jobdir/000/349/349370/dataset_511195_files' returned non-zero exit status 1

libGD problems

Howdy Commissioners,

Figured I'd post an issue to solicit any debugging comments y'all might have. I'm building a Circos package (with small example tools included), and I'm running into some unpleasant troubles trying to install the perl GD package. They seem to be reproducible within Galaxy, but I'm not too familiar with how exactly packages are compiled (e.g. what environment is around, can I run some shell commands to replicate this failure, etc?).

Here's the log:

172.16.49.161 - - [19/Mar/2015:10:55:28 -0500] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://X:12000/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11;
 Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36"
Appending installation info to /home/users/cpt/cpt/esr/Projects/galaxy/galaxy-git/tool_deps/circos/0.67.5/iuc/package_circos_0_67_5/7cf705691b33/lib/perl5/x86_64-linux/perllocal.pod
Included Features:          GD_TIFF GD_VPX GD_XPM GD_JPEG GD_FONTCONFIG GD_FREETYPE GD_PNG GD_GIF GD_GIFANIM GD_OPENPOLYGON
GD library used from:       features:   GD_TIFF GD_VPX GD_XPM GD_JPEG GD_FONTCONFIG GD_FREETYPE GD_PNG GD_GIF GD_GIFANIM GD_OPENPOLYGON
Checking if your kit is complete...
Looks good

Good so far, the uhoh with pkg-config

Package gdlib was not found in the pkg-config search path.
Perhaps you should add the directory containing `gdlib.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gdlib' found
Package gdlib was not found in the pkg-config search path.
Perhaps you should add the directory containing `gdlib.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gdlib' found
Package gdlib was not found in the pkg-config search path.
Perhaps you should add the directory containing `gdlib.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gdlib' found
Package gdlib was not found in the pkg-config search path.
Perhaps you should add the directory containing `gdlib.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gdlib' found
Package gdlib was not found in the pkg-config search path.
Perhaps you should add the directory containing `gdlib.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gdlib' found
Package gdlib was not found in the pkg-config search path.
Perhaps you should add the directory containing `gdlib.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gdlib' found
Configuring for libgd version @.
Checking for stray libgd header files...
install_environment.STDOUT DEBUG 2015-03-19 10:55:29,405 Writing Makefile for GD
Writing Makefile for GD
Writing MYMETA.yml and MYMETA.json
** WARNING: found gd.h header file in /usr/include/gd.h, but it is expected at cflags:     /gd.h. This may cause compile errors! **

scary! But isn't the actual problem (so far)

/home/users/cpt/cpt/esr/Projects/galaxy/galaxy-git/tool_deps/perl/5.18.1/iuc/package_perl_5_18/f6efd799dc70/bin/perl lib/GD/Image.pm.PLS lib/GD/Image.pm
** WARNING: found gd.h header file in /usr/include/gd.h, but it is expected at cflags:     /gd.h. This may cause compile errors! **
** Possible problems found **
Extracting Image.pm (with variable substitutions)
Unrecognized argument in LIBS ignored: 'GD_TIFF'
Unrecognized argument in LIBS ignored: 'GD_VPX'
Unrecognized argument in LIBS ignored: 'GD_XPM'
Unrecognized argument in LIBS ignored: 'GD_JPEG'
Unrecognized argument in LIBS ignored: 'GD_FONTCONFIG'
Unrecognized argument in LIBS ignored: 'GD_FREETYPE'
Unrecognized argument in LIBS ignored: 'GD_PNG'
Unrecognized argument in LIBS ignored: 'GD_GIF'
Unrecognized argument in LIBS ignored: 'GD_GIFANIM'
Unrecognized argument in LIBS ignored: 'GD_OPENPOLYGON'
Unrecognized argument in LIBS ignored: 'libdir:'
install_environment.STDOUT DEBUG 2015-03-19 10:55:29,763 cp lib/GD/Image.pm blib/lib/GD/Image.pm
install_environment.STDOUT DEBUG 2015-03-19 10:55:29,763 cp lib/GD/Group.pm blib/lib/GD/Group.pm
install_environment.STDOUT DEBUG 2015-03-19 10:55:29,769 /home/users/cpt/cpt/esr/Projects/galaxy/galaxy-git/tool_deps/perl/5.18.1/iuc/package_perl_5_18/f6efd799dc70/bin/perl "-Iblib/arch" "-Iblib/lib" bdf_
scripts/bdf2gdfont.PLS bdf_scripts/bdf2gdfont.pl
cp lib/GD/Polyline.pm blib/lib/GD/Polyline.pm
cp lib/GD/Simple.pm blib/lib/GD/Simple.pm
cp lib/GD/Image.pm blib/lib/GD/Image.pm
cp lib/GD/Polygon.pm blib/lib/GD/Polygon.pm
cp lib/GD.pm blib/lib/GD.pm
cp lib/GD/Group.pm blib/lib/GD/Group.pm
AutoSplitting blib/lib/GD.pm (blib/lib/auto/GD)
/home/users/cpt/cpt/esr/Projects/galaxy/galaxy-git/tool_deps/perl/5.18.1/iuc/package_perl_5_18/f6efd799dc70/bin/perl "-Iblib/arch" "-Iblib/lib" bdf_scripts/bdf2gdfont.PLS bdf_scripts/bdf2gdfont.pl
Can't create bdf2gdfont.pl: Permission denied at bdf_scripts/bdf2gdfont.PLS line 11.
make: *** [bdf_scripts/bdf2gdfont.pl] Error 13

And there's the killer error. Permission denied. I don't know what the cwd is so I can't reproduce it, bdf2gdfont.PLS isn't found anywhere in my ~/, I've chown'd the galaxy folder, and even my entire home folder just in case, so I'm completely confounded what's causing this. ANY suggestions would be extremely helpful.

The installation goes on to fail in a pretty unexeciting way:

tool_shed.galaxy_install.tool_dependencies.recipe.step_handler DEBUG 2015-03-19 10:55:30,127 Attempting to download via url: http://circos.ca/distribution/circos-0.67-5.tgz
172.16.49.161 - - [19/Mar/2015:10:55:32 -0500] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://X:12000/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11;
 Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36"
tool_shed.galaxy_install.tool_dependencies.recipe.step_handler DEBUG 2015-03-19 10:55:36,377 Successfully downloaded from url: http://circos.ca/distribution/circos-0.67-5.tgz
galaxy.tools.toolbox.base DEBUG 2015-03-19 10:55:36,617 Appending to tool panel section: Graph/Display Data

package_samtools_1_2 should include tabix and bgzip

According to http://www.htslib.org/download/ , the bgzip, htsfile, and tabix utilities are now provided by HTSlib, but they can also be installed from a samtools or bcftools source package.

These are the command needed:

cd htslib-*
make bgzip tabix
cp -f bgzip tabix $INSTALL_DIR/bin

Then package_tabix_0_2_6 can be deprecated and substituted by package_samtools_1_2 where needed.

Gemini Tools lack test cases

Most tests for Gemini are failing with

RunToolException: Error creating a job for these tool inputs - {u'type': u'error', u'data': {u'infile': u'History does not include a dataset of the required format / build', u'annotate_source': u'History does not include a dataset of the required format / build'}}

reference test report

which turns out to be because there aren't any test cases. Anyone want to write some? :) @bgruening @blankenberg

package_matplotlib_1_2 cannot find numpy dependency

I tried to install package_matplotlib_1_2 on test.galaxyproject.org but did not succeed in spite of the package having numpy 1.7 in set_environment_for_install. Scipy seems to install correctly and it depends on the same numpy version.

install_environment.STDOUT DEBUG 2015-02-25 16:30:19,819 ============================================================================
install_environment.STDOUT DEBUG 2015-02-25 16:30:19,821                         * matplotlib.
basedirlist is: ['/usr/local', '/usr']
BUILDING MATPLOTLIB
            matplotlib: 1.2.1
                python: 2.7.5 (default, Oct 17 2013, 10:00:15)  [GCC 4.4.7
                        20120313 (Red Hat 4.4.7-3)]
              platform: linux2
REQUIRED DEPENDENCIES
                 numpy: no
============================================================================
                        * You must install numpy 1.4 or later to build
                        * matplotlib.

package_samtools_0_1_19 installation fails

On test.galaxyproject.org

phase.o: In function `loadpos':
/galaxy-repl/test/scratch/tmp-toolshed-mtdQrCRU1/samtools-0.1.19/phase.c:475: undefined reference to `gzopen64'
bedcov.o: In function `main_bedcov':
/galaxy-repl/test/scratch/tmp-toolshed-mtdQrCRU1/samtools-0.1.19/bedcov.c:69: undefined reference to `gzopen64'
libbam.a(bam_import.o): In function `__bam_get_lines':
/galaxy-repl/test/scratch/tmp-toolshed-mtdQrCRU1/samtools-0.1.19/bam_import.c:76: undefined reference to `gzopen64'
libbam.a(bam_import.o): In function `sam_open':
/galaxy-repl/test/scratch/tmp-toolshed-mtdQrCRU1/samtools-0.1.19/bam_import.c:472: undefined reference to `gzopen64'
libbam.a(bam_import.o): In function `sam_header_read2':
/galaxy-repl/test/scratch/tmp-toolshed-mtdQrCRU1/samtools-0.1.19/bam_import.c:126: undefined reference to `gzopen64'
libbam.a(bedidx.o):/galaxy-repl/test/scratch/tmp-toolshed-mtdQrCRU1/samtools-0.1.19/bedidx.c:107: more undefined references to `gzopen64' follow
collect2: ld returned 1 exit status
make[1]: *** [samtools] Error 1
make: *** [all-recur] Error 1

Prebuilt R packages link to X11

Reported by @unode on the IRC channel.

bedtools mapBed.xml tool is invalid

After latest push to the MTS, https://toolshed.g2.bx.psu.edu/view/iuc/bedtools/457b09031d57 reports that mapBed.xml is invalid:

This file requires an entry for "'input'" in the tool_data_table_conf.xml file. Upload a file named tool_data_table_conf.xml.sample to the repository that includes the required entry to correct this error.

which does not make much sense for me.

MACS2 not in main tool shed

On IRC:

jelle_: Any reason the macs2 suite maintained by iuc is only available in the test toolshed?

I'm preparing to push this now, so file your complaints if you have them!

MACS2 version update

May want to consider updating MACS2 to a newer binary version.

update gemini version for compatibilty with output from bcbio

Hi,

I am trying to load a gemini database generated by the bcbio pipeline (developed by @chapmanb). I uploaded the generated database into a data library, but when I go to the gemini tools the uploaded db is not shown in the options.
Included in bcbio is the latest version of GEMINI (0.15.1), but the galaxy tools only supports v0.10.0.
Any ideas on how to modify de database/tool/galaxy so I can import my db? Or can you update the included version in the tool?

Thanks!
Matthias

Possible Perl installation issue

Installing /home/galaxy/galaxy/tool_dependencies/perl/5.18.1/iuc/package_perl_5_18/f6efd799dc70/local-lib/man/man3/POD2::PT_BR::local::lib.3                                                                 
Appending installation info to /home/galaxy/galaxy/tool_dependencies/perl/5.18.1/iuc/package_perl_5_18/f6efd799dc70/local-lib/lib/perl5/x86_64-linux/perllocal.pod                                           
Can't locate local/lib.pm in @INC (you may need to install the local::lib module) (@INC contains: /home/galaxy/galaxy/tool_dependencies/perl/5.18.1/iuc/package_perl_5_18/f6efd799dc70/local-lib/perl5 /home#
BEGIN failed--compilation aborted.                                                                                                                                                                           
Can't write to cpanm home '/.cpanm': You should fix it with chown/chmod first.                                                                                                                               

During testing installation from TTS.

Travis Linted Problems

• /home/travis/build/galaxyproject/tools-iuc/tools/rgrnastar_240d/tool_dependencies.xml:10: element readme: Schemas validity error : Element 'readme': This element is not expected.
• /home/travis/build/galaxyproject/tools-iuc/tools/tool_factory_2/tool_dependencies.xml:9: element readme: Schemas validity error : Element 'readme': This element is not expected.
• /home/travis/build/galaxyproject/tools-iuc/packages/package_ballgown_1_0_3/tool_dependencies.xml:59: element readme: Schemas validity error : Element 'readme': Element content is not allowed, because the content type is a simple type definition.
• /home/travis/build/galaxyproject/tools-iuc/packages/package_graphclust_0_7_6/tool_dependencies.xml:22: element action: Schemas validity error : Element 'action': This element is not expected.
• /home/travis/build/galaxyproject/tools-iuc/packages/package_graphclust_0_7_6/tool_dependencies.xml:34: element action: Schemas validity error : Element 'action': This element is not expected.
• /home/travis/build/galaxyproject/tools-iuc/packages/package_graphclust_0_7_6/tool_dependencies.xml:46: element action: Schemas validity error : Element 'action': This element is not expected.
• /home/travis/build/galaxyproject/tools-iuc/packages/package_graphclust_0_7_6/tool_dependencies.xml fails to validate

JBrowse tool should support WebApollo IE for visualization

Just an note-to-self issue.

This is the step I was missing, how do JBrowse and WebApollo work together in Galaxy/Workflows/etc. With a JBrowse step, data is set up, tracks.conf/trackList.json is populated...a WebApollo IE could easily take advantage of that pre-processed data.

Automate pushing to TTS/MTS

Requires support for creating toolsheds in

• galaxyproject/bioblend#124
• galaxyproject/bioblend#128
• galaxyproject/planemo#101
• #74
• galaxyproject/galaxy#29
• galaxyproject/galaxy#30
• galaxyproject/galaxy#31
• add categories to #73

Todo

• bioblend + planemo support for updating repos with galaxyprojext/galaxy#30 (low priority)

This is just an issue to track progress on both all of those.

Move some download_store packages to BioArchive

Now that @nitesh1989 has built http://bioarchive.github.io/, perhaps it's time to move package_r_ggplot2's R packages to using bioarchive.

For those not familiar with bioaRchive, it's a dump of EVERY version of every package from Bioconductor

Gatk2 Missing phiX.fasta

Test Results

IOError: [Errno 2] No such file or directory: '/var/lib/jenkins/workspace/IUC-Github/tools/gatk2/test-data/phiX.fasta'

relevant for variant_combine and variant_eval

Make sure toolform_upgrade = False

In the config/galaxy.ini , the parameter for toolform_upgrade has to be set to be "False" for this to work.

Use new tool form

toolform_upgrade = False

Bedtools genomeCoverageBed cannot find 'zero_regions'

For version 2.22.0 of genomeCoverageBed, I see the Cheetah error NotFound: cannot find 'zero_regions'. This is because it lacks the $report prefix on the conditional variables.

Additionally the validator for unspecified_build seems odd, I had to hack that out to get it to function.

Automatically pushing of repos to MTS

According to @martenson, the MTS is now running 15.05, so I'd like to open discussion as to pushing to the MTS.

• galaxyproject/planemo#71
• galaxyproject/planemo#225

Points for discussion

• What bugs do we need to solve before we can turn that on?
  • Is this creating extraneous installable revisions?
  • Is this polluting the TS?
  • Do I need to implement galaxyproject/planemo#71 to automatically determine proper push order, or is simply pushing everything in the git repo a couple times "good enough"?
• Is everyone OK with automating the push process?

cc @bgruening @peterjc @nsoranzo

package_bedtools_2_19 missing on GitHub

File tools/bedtools/tool_dependencies.xml currently reads:

<?xml version="1.0"?>
<tool_dependency>
    <package name="bedtools" version="2.19.1">
        <repository name="package_bedtools_2_19" owner="iuc" />
    </package>
</tool_dependency>

However, this package in not currently in this repository.

MACS2: Incorrect BED format

The XLS to BED conversion in "callpeak" is incorrect as it places the name column at the very end instead of in column 4.
See https://github.com/galaxyproject/tools-iuc/blob/master/tools/macs2/macs2_callpeak.xml#L83

GraphicsMagick

• 1_3 is actually 1_3_18 https://toolshed.g2.bx.psu.edu/repos/iuc/package_graphicsmagick_1_3/file/4ce7b22bf603/tool_dependencies.xml (6ffe61d)
• 1_3_20 is not in this repository despite being owned by IUC: https://toolshed.g2.bx.psu.edu/repos/iuc/package_graphicsmagick_1_3_20/file/d87c457bbbc2/tool_dependencies.xml (736249f)