bxlab/bx-python

Tools for manipulating biological data, particularly multiple sequence alignments

bx-python's Issues

ImportError maf

from bx.align import maf
raises an ImportError
undefined symbol: PyString_FromStringAndSize

On line 10 of seekbzip2.py
from ._seekbzip2 import SeekBzip2

(in ipython3)

TypeError: write() argument must be str, not bytes when running maf_build_index.py with python3.6

I have latest release of bx-python (0.8.4) installed and came across the following error when index a small maf file using python 3.6:

Traceback (most recent call last):
  File "/home/mt1022/anaconda3/bin/maf_build_index.py", line 88, in <module>
    if __name__ == "__main__": main()
  File "/home/mt1022/anaconda3/bin/maf_build_index.py", line 85, in main
    indexes.write( out )
  File "/home/mt1022/anaconda3/lib/python3.6/site-packages/bx/interval_index_file.py", line 324, in write
    write_packed( f, ">3I", MAGIC, VERSION, len( self.indexes ) )
  File "/home/mt1022/anaconda3/lib/python3.6/site-packages/bx/interval_index_file.py", line 463, in write_packed
    f.write( pack( pattern, *vals ) )
TypeError: write() argument must be str, not bytes

The same file can be indexed successfully with python2.7. Could you help investigate what is causing the error?

pip install

bx-python does not install completely with pip install bx-python, as some shared libraries are not compiled:

bbi_file.so, bigbed_file.so, bigwig_file.so, bpt_file.so, cirtree_file.so

This causes problems with other packages, e.g. this issue with CrossMap.

One needs to download from PyPI and install with pip install bx-python-0.7.3.tar.gz

about bx.intervals.operations.join module

Hello.
I am trying to join the genomic intervals of two datasets with bx.intervals.operations.join module.
Two datasets (f1, f2) have chromosome, start, end, and information columns.
My code is like below.
f1 = open( "HIVD1.txt", 'r' )
f2 = open( "HIVD2.txt", 'r' )
f1 = set(f1)
f2 = set(f2)
result = bx.intervals.operations.join.join( f1, f2, mincols=1, leftfill=True, rightfill=False )
result
It just returns <generator object join at 0x1069d2cd0>.
How can I get and see the result?

Got wrong positions on chomped query maf subset

Hi,

Thank you for developing this nice package.

However, I have encountered issues when subset maf with maf_extract_ranges_indexed.py, the query position of chomped maf seems incorrect:

For example, I have a REF.maf like:

##maf version=1
a score=1643
s Ref.3                            219977717 1643 + 235667834 CCTGTAAAATATATAATCTTGAGCAAACTAGTTAGTCCAATTATTTGTGTTGGGCATTTCAACCACCAAAATCATTTAGGAAAAAGTTTGACCCTATTTCCCTTTCAGGTATCAATATAATATAAAATTTGTTAATGTTTATATGTTTGTCATGAAATAAATATATGAATGAATAAGATATATTGTGTCTATAAATTTTTATTTTCATGAATGATGAAGGTGTGTCCTTCATGACCTTCGTCTGAAGATCGTTATATCCGAAGGAGATAATGCTTCGAAGAACAAAGGTCCTTAATACTTAACGATTGTGTTGTCTTGTTCTTGATTCACAACATTTGAGAACAAGTGACCAACAGCTCTTTTGAGCCTAACTCCTCTAACTTTTTTGGCATTTCTCGAGGATTCCTTACGACTTAGACAAACATAGTTAGCTTGTAAAACAATTGACTAAGCCTAGGAAACGTACCTTTTTTCACATAGTTTTTATAGGATTTGCAATATGCTCAAAACTAAGTCTAAAAGTCAATTTTCTTGCACAAAAGAGTCAGTAAATCAAGGTGTAGGAAAGATAGTATAAGGTGTGTGC-AACTTATTCAAACACTTAAATCTCATGTTTTATAGTTTTACCCAAAGTTGCACTTTTGGCCTTTATTTCCATTCTTTTGAACTTGTTGCCTTTAATTTGCTTTTGCTAGAAGCCTAGAACGTGTGCTCAAGCTGGCATCAAAGTAATTAGTCTGATGTGCTATCACTATGGGTGACAATAAGCTCTAAATTTTACGCTATAAGATTTAAGAATCAGATCAGATTAGAATCAGACCATATTTTTATTCATTTTTGAACTAAAATTTATTTAGGGCACTACCATTTTGTGAAGAAACATTTGGATCGTGATCCATTACCACTCCTAACTGTCACTTGAATGTGAATCATTACCATTGTCATTTGCGTTACAATTCGTAAATCAAACAACTTGTTTGAACCAAATAACTTTCTACGTCATCCTCTACCGTGCACAACTTACATCAACTTTTTCACAAGCCACGACTTAAGGGGTGTTTGAATGCAATAGAGCTAATAGTTAGCTGTTAAAATTAGCTGAAGACATCTAAACAATATAGCTAATAGTTCAGCTATTAGCTATTTTTAGCAAATTAGCTAATAGTTAGCTAGCTATTTGTTAGCTAGCTAATTTCACCGGCAA-TTTTTTGTCAACTAACTATTAGCTCTAATGCATTGAAACACCATCTTTATTAAGGCCCTGTTTCAATCCCACTAGATAAACTTTAGCAACTTTTTTAGCTACTTTTAGCCATTTAGTGATCTAAACAAAAGAGCTAATGATGGTGTGATAAAATTTAGCACATATTATCTCATAAACTGCTCAAGAGATGCTAAAAGTAGCTAAACGTGGGCCAGGAACCCATTTTTCATCCATACCCTCCAACATCATTCCCTCT--CTCCGCATTAAGGGCATATGTGTCTTTTTATTCCAATGTACTAAACTTTATTAGAGTAGTCCAAATACCATAAGGAGCTAAACTTTAGCACTTCAATTTATATGGCTGAAGTTTAGCAGGAAGCTAAAATTTATCCTGTGAGATTGAAACAGGGCCTAAATATATAGCCTCCCTTATCCGTAC
s query                                 2314 1632 -      3946 CCTGTAGAATATATAATCTTGAGCAAACTAGTTAGTCCAATTATTTGTGTTGGGCATTTCAACCACCAAAATCATTTAGGAAAAGGTTTGACCCTATTTCCCTTTCAGATATCAATATAATATAAACTTTGTTAATGTTTATATGTTTGTCATGAAATAAATATATGAATGAATAAGATATATTGTGTGTATAAATTTTTATTTTCATGAATGATGAAGGTGTGTCCTTCATGACCTTCGTCTGAAGATCGTTATATCCGAAGGAGATAATGCTTCGAAGGACGAAGGTCCTTAATACTTAACGATTGTGTTGTCTTGTTCTTGATTCACAACATTTGAGAACAAGTGACCAACAGCTCTTTTGAGCCTAACTCCTCTAACTTTTTTGGCATTTCTCGAGGGTTCCTTACGACTTAGACAAACATAGTTAGCTTGTAAAACAATTGACTAAGCCTAGGAAACATACC-TTTTTCACATAGTTTTTATAGGATTTGCAATATGCTCAAAACTAAGTCCAAAAGTCAATTTTCTTGCACAAAAGAGTTAGTAAATCAAGGTGCAGGAAAGATAGTATAAGATGTGTGCTAACTTATTCAAACACTTAAATCTCATGTTTTATAGTTTTACCCAAAGTTGCACTTTTGGCCTTCATTTCCATTCTTTTGAACTTGTTGCCTTTAATTTGCTTTTGCTAGAAGCCTAGAACGTGTGCTCAAGCTGACATCAAAGTAATTAGTCTGATGTGCTATCACTATGGGTGACAATAAGCTCTAAATTTTACAATATAAGATTTAATGATCAGATCAGATTAGAATCGGACCATATTTTTATTCATTTTTGAACTAAAATTTATTTAGGGCACTACCATTTTGTGAAGAAACATTTGGATCGTGATCCATTACCACTCCTAGCTGTCACTTGAGTGTGAATCATTATCATTGTCATTTGCGTTACAATTCGTAAATCAAACAACTTGTTTGAACCAAATAATTTTCTACGTCATCCTCTACCGTGCACAACTTACATCAACTTTTTCACAAGCCACGACTTAAGGGGTGTTTGAATGCAATAGAGCTAATAGTTAGCTGTTAAAATTAGCTGAAGACATCTAAACAATATAGCTAATAGTTCAGCTATTAGCTATTTTTAGCAAATTAGCTAATA-------------GTTAGCTAGCTAATTCCACCGACAATTTTTTTGTCAACTAACTATTAGCTCTAATGCATTTAAACACCATCTTTATTAAGGCTCTGTTTCAATCCCACTAGATAAACTTTAGCAACTTTTTTAGCTACTTTTAGCCATTTAGTGATCTAAACAGGAGAGCTAATGGTGGTGTGATAAAATTTAGCACATATTATCTCATAAGCTGCTCAAGAGGTGCTAAAAGTAGCTAAACGTGGACCAGGAAACCCCATTTTCCATCCATA-CCTCCAACATCATTCCCTCTCCCTCCGCATTAAGGGCATATGTGTCTTTTTATTCCAATGTGCTAAACTTTATTAGAGTAGTCCAAACACCACAAGGAGCTAAACTTTAGCACTTCAATTTATATGGCTAAAGTTTAGCAAGAAGCTAAAATTTATCCTGTGGGATTGAAACAGGGCCTAAATATATAGCCTCCCTTATCCGTAC

And I was trying to subset the maf based on a region with command:

# >cat test.bed
# Ref.3   219978958       219979958

maf_extract_ranges_indexed.py -c REF.maf < test.bed > test.maf

The output test.maf was like:

##maf version=1
a score=1643
s Ref.3 219978958 402 + 235667834 ACACCATCTTTATTAAGGCCCTGTTTCAATCCCACTAGATAAACTTTAGCAACTTTTTTAGCTACTTTTAGCCATTTAGTGATCTAAACAAAAGAGCTAATGATGGTGTGATAAAATTTAGCACATATTATCTCATAAACTGCTCAAGAGATGCTAAAAGTAGCTAAACGTGGGCCAGGAACCCATTTTTCATCCATACCCTCCAACATCATTCCCTCT--CTCCGCATTAAGGGCATATGTGTCTTTTTATTCCAATGTACTAAACTTTATTAGAGTAGTCCAAATACCATAAGGAGCTAAACTTTAGCACTTCAATTTATATGGCTGAAGTTTAGCAGGAAGCTAAAATTTATCCTGTGAGATTGAAACAGGGCCTAAATATATAGCCTCCCTTATCCGTAC
s query      3543 403 -      3946 ACCATCTTTATTAAGGCTCTGTTTCAATCCCACTAGATAAACTTTAGCAACTTTTTTAGCTACTTTTAGCCATTTAGTGATCTAAACAGGAGAGCTAATGGTGGTGTGATAAAATTTAGCACATATTATCTCATAAGCTGCTCAAGAGGTGCTAAAAGTAGCTAAACGTGGACCAGGAAACCCCATTTTCCATCCATA-CCTCCAACATCATTCCCTCTCCCTCCGCATTAAGGGCATATGTGTCTTTTTATTCCAATGTGCTAAACTTTATTAGAGTAGTCCAAACACCACAAGGAGCTAAACTTTAGCACTTCAATTTATATGGCTAAAGTTTAGCAAGAAGCTAAAATTTATCCTGTGGGATTGAAACAGGGCCTAAATATATAGCCTCCCTTATCCGTAC

However， the exact position of the aligned query sequence was nothing like the position reported in the test.maf:

# the query whole sequence was stored in query.fa, locating the aligned block in the test.maf with seqkit locate function:
# > seqkit locate --gtf -p "ACCATCTTTATTAAGGCTCTGTTTCAATCCCACTAGATAAACTTTAGCAACTTTTTTAGCTACTTTTAGCCATTTAGTGATCTAAACAGGAGAGCTAATGGTGGTGTGATAAAATTTAGCACATATTATCTCATAAGCTGCTCAAGAGGTGCTAAAAGTAGCTAAACGTGGACCAGGAAACCCCATTTTCCATCCATACCTCCAACATCATTCCCTCTCCCTCCGCATTAAGGGCATATGTGTCTTTTTATTCCAATGTGCTAAACTTTATTAGAGTAGTCCAAACACCACAAGGAGCTAAACTTTAGCACTTCAATTTATATGGCTAAAGTTTAGCAAGAAGCTAAAATTTATCCTGTGGGATTGAAACAGGGCCTAAATATATAGCCTCCCTTATCCGTAC" query.fa -o test_location.gtf
# > cat test_location.gtf
query   SeqKit  location        2315    2717    0       -       .       gene_id "ACCATCTTTATTAAGGCTCTGTTTCAATCCCACTAGATAAACTTTAGCAACTTTTTTAGCTACTTTTAGCCATTTAGTGATCTAAACAGGAGAGCTAATGGTGGTGTGATAAAATTTAGCACATATTATCTCATAAGCTGCTCAAGAGGTGCTAAAAGTAGCTAAACGTGGACCAGGAAACCCCATTTTCCATCCATACCTCCAACATCATTCCCTCTCCCTCCGCATTAAGGGCATATGTGTCTTTTTATTCCAATGTGCTAAACTTTATTAGAGTAGTCCAAACACCACAAGGAGCTAAACTTTAGCACTTCAATTTATATGGCTAAAGTTTAGCAAGAAGCTAAAATTTATCCTGTGGGATTGAAACAGGGCCTAAATATATAGCCTCCCTTATCCGTAC";

It seems to me that maf_extract_ranges_indexed.py takes the left part of the chomped maf block as the right part when the query strand is "-".

Did I misunderstand something ? How could I get the correct postions when chomping maf ?

Please find attached the aforementioned files to reproduce the result:
chomp_test.tar.gz

Any suggestion would be grateful.

Best wishes,

Songtao Gui

python3.5 - TwoBitFile chromosome name is byte string

bx-python/lib/bx/seq/twobit.py

Line 76 in 325f495

name = self.read_p_string()

It seems for python3 ,this function "read_p_string" returns a byte string which is better to decode to str #

ImportError: undefined symbol: PyUnicodeUCS2_FromStringAndSize

I am trying to >>> import bx.intervals in python 2.7.5 which I have installed on a computing clusters. Only >>> import bx runs without any trouble, but >>> import bx.intervals gives me the following error:
ImportError: /usr/bin/venv/python2.7/site-packages/bx/intervals/intersection.so: undefined symbol: PyUnicodeUCS2_FromStringAndSize

This Stack thread says I have to recompile python: https://stackoverflow.com/questions/2584198/undefined-symbol-pyunicodeucs2-decode-whilst-trying-to-install-psycopg2/2584554#2584554. I tried that, but without any luck!

multiple sequence alignment

Hi,
Thanks for developing such a nice project. I am interested in multiple sequence alignment and I found this tool is good at this task. But I didn't know how to use the tool. Can you provide some simple use case?

Best,
Neng

Add link to ReadTheDocs in Github README

Please add a link to https://bx-python.readthedocs.io/en/latest/ in the README, I only found it because I recognised the file structure in the doc folder. You can add a shield as follows:

[![Read the Docs](https://img.shields.io/readthedocs/bx-python.svg)](https://bx-python.readthedocs.io/)

Thanks!

Incompatible types str and byte under python3

The following line causes an error when running on Python3. Setting the encoding on the 0 byte string seems to fix the problem.

bx-python/lib/bx/bbi/bpt_file.pyx

Line 74 in f0d2c3e

key += ( '\0' * ( self.key_size - len(key) ) )

Suggested change:

- key += ( '\0' * ( self.key_size - len(key) ) )
+ key += ( '\0'.encode('utf-8') * ( self.key_size - len(key) ) )

install problems without bitset.c file

Hi, I use the command " python setup.py install" to install bx-python on linux server, however, I got "gcc: lib/bx/bitset.c: No such file or directory" error when the scripts running. So, what should I do? My system environment is : python 2.7(anaconda2), gcc version 4.4.7. Thanks a lot.

Please remove autogenerated files from the source tree

Hello,

The files created by cython and anything else that is generated in an automated fashion please do not redistribute with source tree. This way one spots more easily when something fails and it allows the now freshly generated Debian package to work with the release of yours directly. Otherwise those files would first need to be removed as the upstream files are not allowed to be changed during the build if it is not a formal patch.

Here the files coming to our attention:

    lib/bx/_seqmapping.c
    lib/bx/align/_core.c
    lib/bx/align/_epo.c
    lib/bx/align/sitemask/_cpg.c
    lib/bx/arrays/array_tree.c
    lib/bx/arrays/bed.c
    lib/bx/arrays/wiggle.c
    lib/bx/bbi/bbi_file.c
    lib/bx/bbi/bigbed_file.c
    lib/bx/bbi/bigwig_file.c
    lib/bx/bbi/bpt_file.c
    lib/bx/bbi/cirtree_file.c
    lib/bx/bitset.c
    lib/bx/intervals/cluster.c
    lib/bx/intervals/intersection.c
    lib/bx/intseq/ngramcount.c
    lib/bx/misc/_seekbzip2.c
    lib/bx/misc/bgzf.c
    lib/bx/motif/_pwm.c
    lib/bx/pwm/_position_weight_matrix.c
    lib/bx/seq/_nib.c
    lib/bx/seq/_twobit.c
    lib/bx/bbi/bbi_file.h

Many thanks and regards,

Steffen

ZeroDivisionError thrown by bx.intervals.intersection.IntervalNode.__cinit__

I create a lot of IntervalNode objects and eventually this error pops up

File "intersection.pyx", line 395, in bx.intervals.intersection.IntervalTree.insert (lib/bx/intervals/intersection.c:4902)
  File "intersection.pyx", line 127, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1657)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 127, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1657)
  File "intersection.pyx", line 118, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1567)
  File "intersection.pyx", line 127, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1657)
  File "intersection.pyx", line 127, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1657)
  File "intersection.pyx", line 120, in bx.intervals.intersection.IntervalNode.insert (lib/bx/intervals/intersection.c:1600)
  File "intersection.pyx", line 91, in bx.intervals.intersection.IntervalNode.__cinit__ (lib/bx/intervals/intersection.c:1330)
ZeroDivisionError: float division

The issue seems to be that at
self.priority = ceil(nlog * log(-1.0/(1.0 * rand()/RAND_MAX - 1)))
I assume rand() can produce 0 or RAND_MAX, at which point (1.0 * rand()/RAND_MAX - 1) equals 0, hence ZeroDivisionError is thrown

Install instructions?

Can you add some info on installing?

No source tarballs for the release 0.8.8 on the PyPI website

Comparing an integer to builtin function in interval_index_file.py

In interval_index_file.py, BIN_OFFSETS_MAX[ 0 ] = max. Is this meant to be set as "MAX" rather than the python builtin function? Also the indent level in offsets_for_max_size seems to be wrong.

get on bx.align.maf.MAFIndexedAccess results in wrong alignment blocks

Hi,
either I'm doing something wrong or I've stumbled over a peculiar bug in the MAF indexing code.
My setup:
Linux/Python 3.7.1/Anaconda
bx-python=0.8.8 installed from -c bioconda

I use a multi-species, multi sequence maf file indexed via:
maf_build_index.py test.maf test.idx

I want to study SNPs locations on the MSA. I have created a small example repo that contains a test set and an ipynb to demonstrate the bug:
https://github.com/dandaman/bxmafissue/blob/master/BugTracer.ipynb

As you can see in the notebook, get (or probably the index) is somehow mixing up positions. In my full data set some positions are wrong some are not?!

Am I doing something wrong or is this a bug?

This old report https://www.biostars.org/p/122420/ seems to have encountered the same issue.

All the best,
Daniel

undefined symbol: PyUnicodeUCS2_FromStringAndSize

Hello, I encountered some errors while running this package.

ImportError: /home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/site-packages/bx/bbi/bigwig_file.so: undefined symbol: PyUnicodeUCS2_FromStringAndSize

I searched some solutions to deal with it. I guess the bx-python package requires the pyunicode version to be UCS2, but my python(2.7.9) is ucs4.

`>>> import sys

print sys.maxunicode
1114111`

1114111 represents UCS4 and 65535 represents UCS2.
So I try to change the PyUnicode of my python(2.7.9).

wget https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tgz tar -zxvf Python-2.7.9.tgz cd Python-2.7.9 ./configure --enable-unicode=ucs2 make make install

I know the default unicode is ucs2 in the configure step, but I still use it.
In the previous python version, I use conda to create a python(executable file) from a py2.7 environment, so I got the unicode ucs4.
Finally, it failed. Then I tried another solution:
$pip install bx-python==0.7.2 DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality. Collecting bx-python==0.7.2 Using cached bx-python-0.7.2.tar.gz (939 kB) ERROR: Command errored out with exit status 1: command: /home/zhangjj/software/miniconda3/envs/SVE/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-HY8Kf3/bx-python/setup.py'"'"'; __file__='"'"'/tmp/pip-install-HY8Kf3/bx-python/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-wqOwLI cwd: /tmp/pip-install-HY8Kf3/bx-python/ Complete output (24 lines): Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.35.tar.gz Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-HY8Kf3/bx-python/setup.py", line 9, in <module> use_setuptools() File "distribute_setup.py", line 152, in use_setuptools return _do_download(version, download_base, to_dir, download_delay) File "distribute_setup.py", line 131, in _do_download to_dir, download_delay) File "distribute_setup.py", line 201, in download_setuptools src = urlopen(url) File "/home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/urllib2.py", line 437, in open response = meth(req, response) File "/home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/urllib2.py", line 550, in http_response 'http', request, response, code, msg, hdrs) File "/home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/urllib2.py", line 475, in error return self._call_chain(*args) File "/home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/home/zhangjj/software/miniconda3/envs/SVE/lib/python2.7/urllib2.py", line 558, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 403: SSL is required ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Failed to load seekbzip2 module when dealing with bz2 files

Hi,

I have installed bx-python through pip install using python 2.7, the whole install step seems fine, and when dealing with unzipped maf files, everything works good.

However, when I tried to dealing with bz2 compressed maf files, I have got this error msg:

# my cmd:
# > maf_extract_ranges_indexed.py ref.maf.bz2 <test_region.bed
Traceback (most recent call last):
  File "/usr/local/bin/maf_extract_ranges_indexed.py", line 123, in <module>
    main()
  File "/usr/local/bin/maf_extract_ranges_indexed.py", line 67, in main
    index = bx.align.maf.MultiIndexed(maf_files, keep_open=True, parse_e_rows=True, use_cache=use_cache)
  File "/usr/local/lib/python2.7/dist-packages/bx/interval_index_file.py", line 175, in __init__
    for fname in filenames]
  File "/usr/local/lib/python2.7/dist-packages/bx/interval_index_file.py", line 178, in new_indexed_access
    return self.indexed_access_class(data_filename, index_filename, keep_open, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bx/interval_index_file.py", line 205, in __init__
    raise Exception("Trying to open .bz2 file but no seekbzip2 module found")
Exception: Trying to open .bz2 file but no seekbzip2 module found

I have already tried this action on another machine and it do work fine #69 .

So how could I solve this error ?

Any suggestion would be grateful.

Best,

Songtao Gui

Indexing tools unable to handle large MAF files

One example is the hg38 projected chr1 multiz100way from UCSC: http://hgdownload.soe.ucsc.edu/goldenPath/hg38/multiz100way/maf/chr1.maf.gz (5.9G gz'd, 66G gunzip'd)

maf_build_index.py chr1.maf
Traceback (most recent call last):
  File "/path_to/bin/maf_build_index.py", line 83, in <module>
    if __name__ == "__main__": main()
  File "/path_to/bin/maf_build_index.py", line 80, in main
    indexes.write( out )
  File "/path_to/lib/python2.7/site-packages/bx/interval_index_file.py", line 332, in write
    write_packed( f, ">I", base )
  File "/path_to/lib/python2.7/site-packages/bx/interval_index_file.py", line 463, in write_packed
    f.write( pack( pattern, *vals ) )
struct.error: 'I' format requires 0 <= number <= 4294967295

One possibility is to up the version number and store unsigned integers as unsigned long long >Q, which would max out at 18446744073709551615 vs 4294967295. Would double the packed size though.

Another potential workaround could be to break the MAF up into multiple files, but I haven't tested this.

xref: https://biostar.usegalaxy.org/p/18196/

Add support to release linux aarch64 wheels

Problem

On aarch64, pip install bx-python builds the wheels from source code and then installs it. It requires the user to have a development environment installed on their system. Also, it takes more time to build the wheels than downloading and extracting the wheels from PyPI.

Resolution

On aarch64, pip install bx-python should download the wheels from PyPI.

@nsoranzo, Please let me know your interest in releasing aarch64 wheels. I can help with this.

documentation?

are there any docs for the tool?

I was trying to use the axt submodule but with no success: what is the species_to_length argument?

thank you!

time.clock has been removed in Python 3.8

Ref : https://bugs.python.org/issue36895

python lib/bx/intervals/operations/quicksect.py
Traceback (most recent call last):
  File "lib/bx/intervals/operations/quicksect.py", line 169, in <module>
    main()
  File "lib/bx/intervals/operations/quicksect.py", line 141, in main
    starttime = time.clock()
AttributeError: module 'time' has no attribute 'clock'

Python 3.5 not supported for conda installs

For a linux-64 python 3.5.1 environment, the conda install instructions fail:

[zamparol@machine]$ python --version Python 3.5.1 :: Anaconda 2.4.1 (64-bit)

[zamparol@machine]$ conda --version conda 4.1.11

[zamparol@machine]$ conda install -c bioconda bx-python
Fetching package metadata .........
Solving package specifications: ....

The following specifications were found to be in conflict:

• bx-python
• python 3.5*
  Use "conda info " to see the dependencies for each package.

This is strange, since setup.py requires Python version 3.3 greater? A quick scan of the bioconda repo shows I'm not the first to encounter this error. Have you successfully built & deployed on python3 using conda?

Please fix clang warnings

Getting these with clang-40:

src/samtools/bgzf.c:247:17: warning: implicit conversion from 'const int' to 'bgzf_byte_t' (aka 'signed char') changes value from 139 to -117 [-Wconstant-conversion]
    buffer[1] = GZIP_ID2;
              ~ ^~~~~~~~
src/samtools/bgzf.c:255:17: warning: implicit conversion from 'const int' to 'bgzf_byte_t' (aka 'signed char') changes value from 255 to -1 [-Wconstant-conversion]
    buffer[9] = OS_UNKNOWN;
              ~ ^~~~~~~~~~
src/pwm_utils.c:11:27: warning: implicitly declaring library function 'strlen' with type 'unsigned long (const char *)' [-Wimplicit-function-declaration]
        int string_size = strlen(argv[1]);
                          ^
lib/bx/bbi/bbi_file.c:8257:32: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare]
  __pyx_t_11 = ((__pyx_v_start < 0) != 0);
                 ~~~~~~~~~~~~~ ^ ~
src/bunzip/micro-bunzip.c:629:12: warning: assigning to 'unsigned char *' from 'char *' converts between pointers to integer types with different sign [-Wpointer-sign]
                bd->inbuf=inbuf;
                         ^~~~~~

0.8.11 Doesn't install some built shared libraries

I am getting this error from the FreeBSD ports framework:

====> Checking for pkg-plist issues (check-plist)
===> Parsing plist
===> Checking for items in STAGEDIR missing from pkg-plist
===> Checking for items in pkg-plist which are not in STAGEDIR
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/_seqmapping.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/align/_core.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/align/_epo.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/align/sitemask/_cpg.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/arrays/array_tree.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/arrays/bed.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/arrays/wiggle.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/bbi/bbi_file.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/bbi/bigbed_file.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/bbi/bigwig_file.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/bbi/bpt_file.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/bbi/cirtree_file.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/bitset.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/intervals/cluster.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/intervals/intersection.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/intseq/ngramcount.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/misc/_seekbzip2.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/misc/bgzf.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/motif/_pwm.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/pwm/_position_weight_matrix.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/seq/_nib.so
Error: Missing: %%PYTHON_SITELIBDIR%%/7/bx-python-0.8.11/lib/bx/seq/_twobit.so
===> Error: Plist issues found.

The above listed shared libraries were built, written into plist, but were not installed.

Python 3.7
FreeBSD 12.2

StringIO & Python 3.5

StringIO is no longer supported in Python 3.5, instead io should be used:
http://stackoverflow.com/questions/11914472/stringio-in-python3

This creates an import error for any bx.bbi modules. A straightforward solution is provided in the link above, and would also aid with issue #9

Readme install needs liblzo2-dev

The readme file should mention required packages.

On Ubuntu, before running pip install,
sudo apt-get install liblzo2-dev

Error building bx-python for python-2.7.13

Using either pip or manually building bx-python results in the same error. This install is being attempted on a fresh install of python-2.7.13 on a CentOS 7.3 system.

/usr/local/bin/python2.7 setup.py build
running build
running build_py
running build_ext
Traceback (most recent call last):
File "setup.py", line 233, in
main()
File "setup.py", line 50, in main
cmdclass=command_classes )
File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/distutils/command/build.py", line 127, in run
self.run_command(cmd_name)
File "/usr/local/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/site-packages/Cython-0.25.2-py2.7-linux-x86_64.egg/Cython/Distutils/old_build_ext.py", line 185, in run
_build_ext.build_ext.run(self)
File "/usr/local/lib/python2.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/local/lib/python2.7/site-packages/Cython-0.25.2-py2.7-linux-x86_64.egg/Cython/Distutils/old_build_ext.py", line 192, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/usr/local/lib/python2.7/site-packages/Cython-0.25.2-py2.7-linux-x86_64.egg/Cython/Distutils/old_build_ext.py", line 203, in cython_sources
from Cython.Compiler.Main
File "/usr/local/lib/python2.7/site-packages/Cython-0.25.2-py2.7-linux-x86_64.egg/Cython/Compiler/Main.py", line 30, in
from .Symtab import ModuleScope
File "/usr/local/lib/python2.7/site-packages/Cython-0.25.2-py2.7-linux-x86_64.egg/Cython/Compiler/Symtab.py", line 18, in
from . import PyrexTypes
File "/usr/local/lib/python2.7/site-packages/Cython-0.25.2-py2.7-linux-x86_64.egg/Cython/Compiler/PyrexTypes.py", line 17, in
from .Code import UtilityCode, LazyUtilityCode, TempitaUtilityCode
File "Cython/Compiler/Code.py", line 838, in init Cython.Compiler.Code (/usr/local/src/python/Cython-0.25.2/Cython/Compiler/Code.c:81208)
File "/usr/local/lib/python2.7/re.py", line 194, in compile
return _compile(pattern, flags)
File "/usr/local/lib/python2.7/re.py", line 251, in _compile
raise error, v # invalid expression
sre_constants.error: unexpected end of regular expression

Minimal Python version

Hi, I've noticed that setup.py forbids installing bx-python on Python <2.4. However, from the code it seems the minimal version is at least 2.5, because of

lib/bx/align/epo.py
4:from __future__ import with_statement

which was added in 2.5.0.

Could you please clarify the minimum Python version requirements?

binned_bitsets_from_list errors when chromosome size is larger than set MAX (512M)

I was running RSeQC inner_distance.py with a plant species which has large chromosomes (many are > 512Mb). I encountered the following error:

inner_distance.py -i in2252_13Aligned.sortedByCoord.out.bam -o in2252_13.rseqc -r ../genes.bed
Get exon regions from ../genes.bed ...
Traceback (most recent call last):
  File "/opt/conda/envs/nf-core-rnaseq-1.4.2/bin/inner_distance.py", line 95, in <module>
    main()
  File "/opt/conda/envs/nf-core-rnaseq-1.4.2/bin/inner_distance.py", line 87, in main
    obj.mRNA_inner_distance(outfile=options.output_prefix,low_bound=options.lower_bound_size,up_bound=options.upper_bound_size,step=options.step_size,refbed=options.ref_gene,sample_size=options.sampleSize, q_cut = options.map_qual)
  File "/opt/conda/envs/nf-core-rnaseq-1.4.2/lib/python3.6/site-packages/qcmodule/SAM.py", line 3582, in mRNA_inner_distance
    exon_bitsets = binned_bitsets_from_list(ref_exons)
  File "/opt/conda/envs/nf-core-rnaseq-1.4.2/lib/python3.6/site-packages/bx/bitset_builders.py", line 143, in binned_bitsets_from_list
    last_bitset.set_range( start, end - start )
  File "lib/bx/bitset.pyx", line 216, in bx.bitset.BinnedBitSet.set_range
  File "lib/bx/bitset.pyx", line 184, in bx.bitset.bb_check_range_count
  File "lib/bx/bitset.pyx", line 180, in bx.bitset.bb_check_index
IndexError: 536882486 is larger than the size of this BitSet (536870912).

After examining biset.py and bitset_builders.py, I think the error was caused by the following hard-coded limit:

bx-python/lib/bx/bitset.pyx

Line 195 in 1731099

MAX=512*1024*1024

I don't know the reason behind this setting. I am wondering is it possible to increase that to enable dealing with genomes with large chromosomes?

Wheel for Python 3.8

Hello,

Could it be possible to have a wheel for python 3.8?

Thanks!

Importing ABC directly from collections was deprecated and will be removed in Python 3.10. Use collections.abc

lib/bx/seq/twobit.py
4:from collections import Mapping

lib/bx/misc/cdb.py
3:from collections import Mapping

scripts/aggregate_scores_in_intervals.py
19:from collections import Mapping

Exception: 2147483648 is larger than the maximum possible size (2147483647)

Hello,

Debian is not currently able to build bx-python on the armel, armhf, i386, mipsel architectures (amongst others) due to the following error:

======================================================================
ERROR: bx.interval_index_file_tests.test_interval_index_file
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.7_bx-python/build/bx/interval_index_file_tests.py", line 22, in test_interval_index_file
    ix.add( name, start, end, i, max=interval_index_file.MAX )
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.7_bx-python/build/bx/interval_index_file.py", line 276, in add
    self.indexes[name] = Index( max=max )
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.7_bx-python/build/bx/interval_index_file.py", line 346, in __init__
    self.new( min, max )
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.7_bx-python/build/bx/interval_index_file.py", line 364, in new
    self.offsets = offsets_for_max_size( max )
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.7_bx-python/build/bx/interval_index_file.py", line 136, in offsets_for_max_size
    raise Exception( "%d is larger than the maximum possible size (%d)" % ( max_size, BIN_OFFSETS_MAX[0] ) )
Exception: 2147483648 is larger than the maximum possible size (2147483647)

Example build log: https://buildd.debian.org/status/fetch.php?pkg=python-bx&arch=armel&ver=0.8.4-2&stamp=1564652009&raw=0

Should we skip this test for now?

Runtime errors using BigWig files.

I installed the latest bx-python version (0.8.0) on both python 3.3 and 2.7 and both got errors, but different errors, when opening (3.3) and retrieving data from (2.7) BigWig files.
Any ideas are greatly appreciated!

Python3.3:

Traceback (most recent call last):
  File "/home/unix/cgdeboer/bin/aggregateGenomeData.py", line 91, in <module>
    inFile1 = **BigWigFile(open(args.inFile))**
  File "bigwig_file.pyx", line 127, in bx.bbi.bigwig_file.BigWigFile.__init__ (lib/bx/bbi/bigwig_file.c:3437)
  File "bbi_file.pyx", line 119, in bx.bbi.bbi_file.BBIFile.__init__ (lib/bx/bbi/bbi_file.c:4404)
  File "bbi_file.pyx", line 129, in bx.bbi.bbi_file.BBIFile.open (lib/bx/bbi/bbi_file.c:4581)
  File "/home/unix/cgdeboer/.local/lib/python3.3/site-packages/bx_python-0.8.0-py3.3-linux-x86_64.egg/bx/misc/binary_file.py", line 37, in __init__
    bytes = file.read( 4 )
  File "/home/unix/cgdeboer/Python-3.3.3/lib/python3.3/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 1: ordinal not in range(128)

Python 2.7:

Traceback (most recent call last):
  File "/home/unix/cgdeboer/bin/aggregateGenomeData.py", line 125, in <module>
    values = **inFile1.get_as_array( locus[GENOMEDATA.CHR], stF, enF )**
  File "bigwig_file.pyx", line 153, in bx.bbi.bigwig_file.BigWigFile.get_as_array (lib/bx/bbi/bigwig_file.c:3795)
  File "bigwig_file.pyx", line 159, in bx.bbi.bigwig_file.BigWigFile.get_as_array (lib/bx/bbi/bigwig_file.c:3578)
  File "bbi_file.pyx", line 248, in bx.bbi.bbi_file.BBIFile._get_chrom_id_and_size (lib/bx/bbi/bbi_file.c:5656)
  File "bpt_file.pyx", line 76, in bx.bbi.bpt_file.BPTFile.find (lib/bx/bbi/bpt_file.c:1388)
  File "bpt_file.pyx", line 55, in bx.bbi.bpt_file.BPTFile.r_find (lib/bx/bbi/bpt_file.c:1154)
AttributeError: 'BinaryFileReader' object has no attribute 'read_bits64'

Delete intervals in interval tree

Is it possible to delete intervals

http://bx-python.readthedocs.io/en/latest/lib/bx.intervals.intersection.html#bx.intervals.intersection.IntervalTree

Support for using alternate index name for maf_extract_ranges_indexed.py

Would it be possible to add a -i option to use an index other than .index for the maf_extract_ranges_indexed.py script? I have read but not write access to the folder containing my maf. Relatedly, if the scripts could work with symbolic links, that would be helpful.

Thank you!

Missing data from BigWig files

I have a program that uses bx-python to extract data from a bigwig file for specific BED loci. For some reason certain chromosomes are sometimes skipped for having no data (BigWigFile.get_as_array( chrom, startPos, endPos ) returns None). Looking at the contents of the bigwig file manually or on IGV indicates that there is in fact data for the "missing" chromosomes:
This is an example peak that is being skipped:
chr6 157355 157505 chr6:157355-157505
Same region on IGV:

Looking at this region on the command line:

$ bigWigToBedGraph BWs/B_30.q30.bw  temp.out -chrom=chr6 -start=157355 -end=157505
$ cat temp.out
chr6	157361	157362	2
chr6	157363	157364	2
chr6	157372	157373	1
chr6	157388	157389	1
chr6	157392	157393	1
chr6	157403	157404	1
chr6	157413	157414	1
chr6	157415	157416	1
chr6	157419	157420	1
chr6	157424	157425	1
chr6	157432	157435	1
chr6	157439	157440	1
chr6	157443	157444	1
chr6	157477	157478	1
chr6	157491	157492	1
chr6	157493	157494	2
chr6	157504	157505	1

The following code is enough to reproduce the issue:

import bx
print(bx.__file__)
from bx.bbi.bigwig_file import BigWigFile
inFile1 = BigWigFile(open("BWs/B_30.q30.bw"))
values = inFile1.get_as_array( "chr6", 157355, 157505 )
print(values)

and produces this output:

$HOME/.local/lib/python2.7/site-packages/bx_python-0.8.0-py2.7-linux-x86_64.egg/bx/__init__.pyc
None

Note that None appears to only be returned when a chromosome is not found; when the chromsome is found and there is no data for the region, it returns a list of nans.

I can provide the BW if that would be helpful. I need somewhere to put it (150MB).

asking for support of arbitary length integer

HI,
I would like the bx.intervals.intersection to support arbitary length integer(to be specific, would like to support 2 ** 48), please consider if possible.

Publish Wheels

Can we publish binary wheels to PyPI for this package?

We have wheels on https://wheels.galaxyproject.org/packages/ - would it be as easy as twine uploading them?

bx_python-0.8.2-cp27-cp27m-manylinux1_i686.whl
bx_python-0.8.2-cp27-cp27m-manylinux1_x86_64.whl
bx_python-0.8.2-cp27-cp27mu-manylinux1_i686.whl
bx_python-0.8.2-cp27-cp27mu-manylinux1_x86_64.whl
bx_python-0.8.2-cp34-cp34m-manylinux1_i686.whl
bx_python-0.8.2-cp34-cp34m-manylinux1_x86_64.whl
bx_python-0.8.2-cp35-cp35m-manylinux1_i686.whl
bx_python-0.8.2-cp35-cp35m-manylinux1_x86_64.whl
bx_python-0.8.2-cp36-cp36m-manylinux1_i686.whl
bx_python-0.8.2-cp36-cp36m-manylinux1_x86_64.whl
bx_python-0.8.2-cp37-cp37m-manylinux1_i686.whl
bx_python-0.8.2-cp37-cp37m-manylinux1_x86_64.whl
bx_python-0.8.2-cp27-cp27m-macosx_10_6_intel.whl
bx_python-0.8.2-cp34-cp34m-macosx_10_6_intel.whl
bx_python-0.8.2-cp35-cp35m-macosx_10_6_intel.whl
bx_python-0.8.2-cp36-cp36m-macosx_10_6_intel.whl
bx_python-0.8.2-cp37-cp37m-macosx_10_6_intel.whl

error can't found bx.intervals module

I installed bx-python use " pip install bx-python" command in my linux. usually my default path is /uesr/python2.7/lib/python2.7/site-packages, but some file of this module is in /uesr/python2.7/bin. When I import the module bx.intervals in python's command line with the command "import bx.intervals", it worked well ,but when I worte the command into a *.py file , I got an error "ModuleNotFoundError: No module named 'bx'".Can you help me fix the error?

OverflowError: value too large to convert to int

I am still having this issue:

Issue from: https://bitbucket.org/james_taylor/bx-python/issues/51/max-value-of-interval

from bx.intervals.intersection import IntervalTree
b = IntervalTree()
b.add(300000000,300000100,"a")
b.find(300000010,300000011)
>>>['a']
b.add(2119513096, 2234682974,"b")
>>>Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "intersection.pyx", line 388, in bx.intervals.intersection.IntervalTree.insert (lib/bx/intervals/intersection.c:4823)
OverflowError: value too large to convert to int

StopIteration transformed to RuntimeError on python >= 3.7

This is due to https://www.python.org/dev/peps/pep-0479/

Traceback (most recent call last):
  File "/Users/mvandenb/miniconda3/envs/[email protected]/lib/python3.7/site-packages/bx/intervals/io.py", line 234, in iterwrapper
    self.current_line = next(self.input_wrapper)
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/mvandenb/src/galaxy/lib/galaxy/datatypes/converters/interval_to_bedstrict_converter.py", line 155, in <module>
    __main__()
  File "/Users/mvandenb/src/galaxy/lib/galaxy/datatypes/converters/interval_to_bedstrict_converter.py", line 132, in __main__
    for count, region in enumerate(bx.intervals.io.NiceReaderWrapper(fh, chrom_col=chromCol, start_col=startCol, end_col=endCol, strand_col=strandCol, fix_strand=True, return_header=False, return_comments=False)):
  File "/Users/mvandenb/miniconda3/envs/[email protected]/lib/python3.7/site-packages/bx/intervals/io.py", line 221, in __next__
    nextitem = super(NiceReaderWrapper, self).__next__()
  File "/Users/mvandenb/miniconda3/envs/[email protected]/lib/python3.7/site-packages/bx/tabular/io.py", line 105, in __next__
    line = next(self.input_iter)
RuntimeError: generator raised StopIteration

How to dealing with maf files compressed with bzip2 or lzop ?

Hi,

I have noticed that bx-python supports maf files compressed with bzip2 or lzop. But when practicing, I have encountered some errors:

For example, I would like to build index from compressed maf file with maf_build_index.py:
When dealing with bzip2, the script suggested that a bz2t should be created with bzip-table

bx-python/scripts/maf_build_index.py

Lines 32 to 33 in 1731099

	doc_optparse.exit("To index bz2 compressed files first "
	"create a bz2t file with bzip-table.")

However, I could not find tool with the exact name "bzip-table", the most similar one was seek-bzip-table cmd in seek-bzip

And after generating the bz2t file with seek-bzip-table, the build index step failed with error of:

# > seek-bzip-table test.maf.bz2 >test.maf.bz2t
# > maf_build_index.py test.maf.bz2
Traceback (most recent call last):
  File "/public/home/stgui/.linuxbrew/bin/maf_build_index.py", line 85, in <module>
    main()
  File "/public/home/stgui/.linuxbrew/bin/maf_build_index.py", line 63, in main
    maf_in = TextIOWrapper(maf_in, encoding="ascii")
AttributeError: 'SeekableBzip2File' object has no attribute 'readable'

And I could not figure out how to solve this. So I just move to test the lzo compressed MAF, but I was also stucked at generating lzop table with lzop_build_offset_table.py :

# > bzip2 -dc test.maf.bz2 | lzop > test.maf.lzo
# > lzop_build_offset_table.py <test.maf.lzo >test.maf.lzot
Traceback (most recent call last):
  File "/public/home/stgui/.linuxbrew/bin/lzop_build_offset_table.py", line 98, in <module>
    main()
  File "/public/home/stgui/.linuxbrew/bin/lzop_build_offset_table.py", line 47, in main
    assert magic == MAGIC, "Not LZOP file"
AssertionError: Not LZOP file

It seems the header of my lzo file did not match with the MAGIC string of lzop_build_offset_table.py, but the test.maf.lzo file could be normally manipulated with lzop.

So how could I dealing with compressed MAF files ? My real goal is to use maf_extract_ranges_indexed.py to extract subsets from compressed maf files.

Best wishes,

Songtao Gui

__version__ flag

Hi,
It seems that the version flag is no longer available in the latest bx-python version ?
Thanks

Could you please add 0.8.0 to pypi and the releases page?

I'd like to update the bx-python bioconda recipe to support py3 since bx-python now supports it.

No module named 'UserDict' when import bx.intervals.io

Getting ImportError: No module named 'UserDict' when running

from bx.intervals import io

I am using Python 3.5.2 and it looks like 2-to-3 migration
issues. Does a workaround exist?

typo

Hi

in README file, optomized should be optimized.

release tag maybee ?

is a release tag something that you may consider.

we have to deal with some reproducibilty documentation.
grabbing the archive is more easy than clone and checkout specific tag.

thanks for your consideration.

Is there documentation for bx-python outside the codebase?

the wiki page redirects to the repo main page which is weird...