tanghaibao/jcvi

Python library to facilitate genome assembly, annotation, and comparative genomics

jcvi's Issues

Failed: savefig failed. Reset usetex to False.

Hi,

It failed when saving figure. Infos are below:

16:26:24 [base] savefig failed. Reset usetex to False.
LaTeX was not able to process the following string:
'lp'
Here is the full report generated by LaTeX:

This is pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009/Debian)
entering extended mode
(./1d97300a39adbdb822e515e35e631e34.tex
LaTeX2e <2009/09/24>
Babel <v3.8l> and hyphenation patterns for english, usenglishmax, dumylang, noh
yphenation, loaded.
(/usr/share/texmf-texlive/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texmf-texlive/tex/latex/base/size10.clo))

! LaTeX Error: File `type1cm.sty' not found.

Type X to quit or to proceed,
or enter new name. (Default extension: sty)

Enter file name:
! Emergency stop.
<read *>

l.3 \renewcommand
{\rmdefault}{pnc}^^M
No pages of output.
Transcript written on 1d97300a39adbdb822e515e35e631e34.log.

/usr/local/lib/python2.7/dist-packages/matplotlib/font_manager.py:1288: UserWarning: findfont: Font family [u'sans-serif'] not found. Falling back to Bitstream Vera Sans
(prop.get_family(), self.defaultFamily[fontext]))
/usr/local/lib/python2.7/dist-packages/matplotlib/font_manager.py:1288: UserWarning: findfont: Font family [u'Helvetica'] not found. Falling back to Bitstream Vera Sans
(prop.get_family(), self.defaultFamily[fontext]))

Any suggestion about this problem?

Sincerely,
Tiany

Citation?

Hello all,

This is not a real issue, but I have used the JCVI library for several analyses in my study and I was wondering what way you would prefer to be cited?

Thanks,
Erik

translating scaffold position to genomic position

Hi,

Would it be possible to direct me to the part of the script where the scaffold positions are translated to the genomic positions?

I have a set of genic coordinates of the scaffolds and I would like to translate them to the genomic position but as I have used allmaps for generating the final assembly, it is important that I use the same criteria as allmaps for translating new positions.

Thanks in advance!

Customize the PDF output?

I am really happy with ALLMAPS, but we have 7 genetic maps, and the graphs are a little hard to read (see attached). Is there an easy way I can increase the page size in the PDF, so that I can get a higher resolution and more readable PDF that I could include in our paper?

chr8.pdf

Script for drawing gene model synteny?

Lots of very useful tools in this repository, thanks for sharing! I'm wondering if buried in here somewhere is a graphics script that would draw a synteny map for gene models instead of entire scaffolds/chromosomes as done in allmaps? I'm looking for something on more of the 20-100Kb scale to show some local rearrangements/indels between two isolates of the same species. Perhaps something like you pass two bed files of gene models as well as an anchors file from jcvi.compara.catalog ortholog - and the result looks similar to the karyotype result except with individual gene models and orthologs identified between them. I'm happy to try to help come up with a solution, but my matplotlib skills are not good....

ValueError: too many values to unpack

Hey,

I am trying to run Allmaps on a GeneticMap with 5127 markers and allmaps stops shortly after loading the bed file. Any ideas why this is happening? My bed file looks like this:

SCF_9 140192 140193 GeneticMap-2:4.594 SCF_9:140193
SCF_9 156511 156512 GeneticMap-2:7.781 SCF_9:156512
SCF_9 215281 215282 GeneticMap-2:22.010 SCF_9:215282

13:43:09 [base] Load file out.bed
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/ebio/abt6_projects9/guppy_reassembly/data/geneticmap/5_allmaps/jcvi/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1721, in
main()
File "/ebio/abt6_projects9/guppy_reassembly/data/geneticmap/5_allmaps/jcvi/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 820, in main
p.dispatch(globals())
File "/ebio/abt6_projects9/guppy_reassembly/data/geneticmap/5_allmaps/jcvi/local/lib/python2.7/site-packages/jcvi/apps/base.py", line 81, in dispatch
globalsaction
File "/ebio/abt6_projects9/guppy_reassembly/data/geneticmap/5_allmaps/jcvi/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1248, in path
remove_outliers=(not opts.noremoveoutliers))
File "/ebio/abt6_projects9/guppy_reassembly/data/geneticmap/5_allmaps/jcvi/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 501, in init
self.append(Marker(b))
File "/ebio/abt6_projects9/guppy_reassembly/data/geneticmap/5_allmaps/jcvi/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 476, in init
self.mapname, self.lg = b.accn.split("-")
ValueError: too many values to unpack

plot estimate gap size

Dear Haibao,

I was wondering whether it is possible to generate the gap size estimate plot like the one you have shown here [https://github.com/tanghaibao/jcvi/wiki/ALLMAPS:-How-to-estimate-gap-sizes] using ALLMAPS commands?

Another question is that whether it is possible to change the ylim in the plot option?

Thank you,
Best,
Homa

replacing Image with PIL.Image for grabseeds

Dear jcvi developers,

I run into troubles when trying to run grabseeds with pyhon2.7.

It appears that Image module cant be found. When replacing Image with PIL.Image, all works fine.
Additionally, i got a warning that skimage.filter was renamed to skimage.filters.

Do you know if the changes are valid or if i compromise other functions with this patch?

Following changes allowed me to successfully run grabseeds on the test picture:

diff --git a/apps/tesseract.py b/apps/tesseract.py
index 6939fab..7d82e06 100644
--- a/apps/tesseract.py
+++ b/apps/tesseract.py
@@ -46,7 +46,7 @@ http://wiki.github.com/hoffstaetter/python-tesseract
 # CHANGE THIS IF TESSERACT IS NOT IN YOUR PATH, OR IS NAMED DIFFERENTLY
 tesseract_cmd = 'tesseract'

-import Image
+import PIL.Image
 import StringIO
 import subprocess
 import sys
diff --git a/graphics/grabseeds.py b/graphics/grabseeds.py
index ced6548..bfdd73b 100644
--- a/graphics/grabseeds.py
+++ b/graphics/grabseeds.py
@@ -15,12 +15,12 @@ from math import sin, cos, pi
 import numpy as np
 from jcvi.graphics.base import plt, savefig, normalize_axes, Rectangle, latex

-from Image import open as iopen
+from PIL.Image import open as iopen
 from wand.image import Image
 from scipy.ndimage import binary_fill_holes, distance_transform_edt
 from scipy.optimize import fmin_bfgs as fmin
 from skimage.color import gray2rgb, rgb2gray
-from skimage.filter import canny, roberts, sobel
+from skimage.filters import canny, roberts, sobel
 from skimage.feature import peak_local_max
 from skimage.measure import regionprops, label
 from skimage.morphology import disk, closing, watershed

Best regards,
Michel

warning: spaces in the .csv files will prevent ALLMAPS from completing

I have been trying to figure out why ALLMAPS did not finish on one of my maps, then I realized that my .cvs file have spaces in them (after the commas). In this case, ALLMAPS will take " 1" (or " 2", or " 3") as the linkage group name, instead of "1", and it will fail at the end (after running the TSPs for all linkage groups). Just FYI.

jcvi.graphics.karyotype layout error

Dear Haibao,

Hello,

When I use karyotype.py, It couldn't read color attribute in layout file.

Here is the error.

17:04:06 [base] Load file layout
17:04:06 [base] Load file ar.bed
17:04:08 [base] Load file pineapple.bed
Traceback (most recent call last):
File "/home/wyim/bin/python2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/home/wyim/bin/python2.7/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/graphics/karyotype.py", line 316, in
main()
File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/graphics/karyotype.py", line 304, in main
Karyotype(fig, root, seqidsfile, layoutfile)
File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/graphics/karyotype.py", line 255, in init
layout = Layout(layoutfile, generank=generank)
File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/graphics/karyotype.py", line 79, in init
self.assign_colors()
File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/graphics/base.py", line 98, in assign_colors
self.assign_array("color", colorset)
File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/graphics/base.py", line 92, in assign_array
if not getattr(x, attrib):
AttributeError: 'LayoutLine' object has no attribute 'color'

Here is my command,

python -m jcvi.graphics.karyotype seqid layout

layout file contains

.8, .18, .81, 0, #ff7f00, Amborella, center, ar.bed
.7, .18, .81, 0, #377eb8, Ananas, center, pineapple.bed
e, 0, 1, pineapple.amborella.4x1.simple

seqid contains

LG01
LG02
AmTr_v1.0_scaffold00017
AmTr_v1.0_scaffold00019
AmTr_v1.0_scaffold00021

demo file for synteny.py

Hi,
Could you give some demo files for using synteny.py?

KeyError: ('SCF_15', 6340, 3871002) during agp compress

I am trying to compress to AGP files. They look like this:

SCF_1 1 14498021 1 W CTG_001 1 14498021 +
SCF_1 14498022 14525698 2 N 27677 scaffold yes map
SCF_1 14525699 16119896 3 W CTG_001 14498022 16092219 +
SCF_15 1 135269 1 W CTG_015 1 135269 +
SCF_15 135270 328631 2 N 193362 scaffold yes map
SCF_15 328632 582743 3 W CTG_058 1 254112 +
SCF_15 582744 582843 4 N 100 scaffold yes map
SCF_15 582844 3871002 5 W CTG_059 1 3288159 +

chr_2 1 3871002 1 W SCF_15 1 3871002 -
chr_2 3871003 3871102 2 N 100 scaffold yes map
chr_2 3871103 19990998 3 W SCF_1 1 16119896 -

The following command failed

python -m jcvi.formats.agp compress test_1.agp test_2.agp

With

Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/ebio/abt6_projects9/abt6_software/bin/jcvi/lib/python2.7/site-packages/jcvi-0.7.3-py2.7.egg/jcvi/formats/agp.py", line 1824, in
main()
File "/ebio/abt6_projects9/abt6_software/bin/jcvi/lib/python2.7/site-packages/jcvi-0.7.3-py2.7.egg/jcvi/formats/agp.py", line 741, in main
p.dispatch(globals())
File "/ebio/abt6_projects9/abt6_software/bin/jcvi/local/lib/python2.7/site-packages/jcvi-0.7.3-py2.7.egg/jcvi/apps/base.py", line 87, in dispatch
globalsaction
File "/ebio/abt6_projects9/abt6_software/bin/jcvi/lib/python2.7/site-packages/jcvi-0.7.3-py2.7.egg/jcvi/formats/agp.py", line 791, in compress
store[(a.component_id, a.component_beg, a.component_end)]
KeyError: ('SCF_15', 1, 3871002)

Any idea would goes wrong here?

Thanks a lot,
F

allmaps path failing

I have a pipeline that uses AllMaps on different assemblies. I have 4 assemblies and just as many maps. AllMaps finishes without complaint on 3 of 4 assemblies. On the final assembly I get the error below.

17:16:41 [allmaps] Working on a2_dxa-LGDxP7|0666-19 ...
Traceback (most recent call last):
  File "/loginhome/cwischmeyer/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/loginhome/cwischmeyer/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/mnt/nfs/demeter/loginhome/cwischmeyer/cwischmeyer-home-ubuntu/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1684, in <module>
    main()
  File "/mnt/nfs/demeter/loginhome/cwischmeyer/cwischmeyer-home-ubuntu/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 820, in main
    p.dispatch(globals())
  File "/loginhome/cwischmeyer/lib/python2.7/site-packages/jcvi/apps/base.py", line 81, in dispatch
    globals[action](sys.argv[2:])
  File "/mnt/nfs/demeter/loginhome/cwischmeyer/cwischmeyer-home-ubuntu/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1297, in path
    ngen=ngen, npop=npop, cpus=cpus, seed=seed)
  File "/mnt/nfs/demeter/loginhome/cwischmeyer/cwischmeyer-home-ubuntu/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 152, in __init__
    signs = self.assign_orientation()
  File "/mnt/nfs/demeter/loginhome/cwischmeyer/cwischmeyer-home-ubuntu/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 383, in assign_orientation
    pivot_oo = [pivot_oo.get(x, 0) for x in scaffolds]
UnboundLocalError: local variable 'pivot_oo' referenced before assignment

plotall error

Hi,
I have used ALLMAPS many times, but this is the first time I use 11 genetic maps.
The commands python -m jcvi.assembly.allmaps merge ... and python -m jcvi.assembly.allmaps path ... conclude fine, but when I run

stelo@H4:~/ALLMAPS_in_progress$ python -m jcvi.assembly.allmaps plotall map.bed --figsize=12x8 --dpi=600 --format=eps
16:59:38 [base] Load file `map.agp`
16:59:38 [base] Load file `map.lifted.bed`
16:59:42 [allmaps] Map contains 169097 markers in 121 linkage groups.
16:59:46 [base] Load file `weights.txt`
16:59:46 [base] Imported 11 records from `weights.txt`.
16:59:46 [allmaps] Map weights: [('CI', 1), ('WC', 1), ('27I', 1), ('LL', 1), ('TI', 1), ('SV', 1), ('NF', 1), ('5I', 1), ('CB', 1), ('Y5', 1), ('46B', 1)]
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1684, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 820, in main
    p.dispatch(globals())
  File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 81, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1680, in plotall
    plot(xargs + [seqid])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1544, in plot
    colors = dict((mapname, set2[i]) for i, mapname in enumerate(mapnames))
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1544, in <genexpr>
    colors = dict((mapname, set2[i]) for i, mapname in enumerate(mapnames))
IndexError: list index out of range

Is there a limit on the number of genetic maps for plotall?

Stefano

Why those scaffolds which can not be Oriented are random anchored everywhere?

Hi,
In allmaps lifted files, many scaffolds which can not be Oriented are random anchored everywhere. I would like to known why allmaps do in this way?
Thank you.

landscape heatmap colors

Dear Haibao,

I tried to draw landscape figure but the components heatmap colors are only generated gray.

Is there any option or do I need to add more options?

Won

python -m jcvi.graphics.landscape heatmap --heatmaps=test,test,Helitron Mcr_Scaffold_003.fas McrScaffold003 --diverge=RdYlBu --stacks=test,test,Helitron,aa

Did anyone have this error?

Traceback (most recent call last):
File "/software/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/software/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1725, in
main()
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 824, in main
p.dispatch(globals())
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/apps/base.py", line 82, in dispatch
globalsaction
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1379, in path
"--figsize={0}".format(opts.figsize)])
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1721, in plotall
plot(xargs + [seqid])
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/assembly/allmaps.py", line 1599, in plot
g = GeneticMap(ax1, x, y1, y2, markers, tip=tip, flip=flip)
File "/software/UHTS/jcvi/0.7.1/lib/python2.7/site-packages/jcvi/graphics/chromosome.py", line 181, in init
ratio = r / max_chr_len
ZeroDivisionError: float division by zero

Absolute Pathing Required for Merge

Hey All,

If you run merge with relative pathing (i.e. ../../foobar.csv) when you get the weights, the first column is empty, and the bed file identifies the group as a -1. This is fixed by providing absolute paths to input files.

Could do a os.path.abspath on all file loads.

Thank you,

Connor

Disk quota error

Hello,
I am having an issue with ALLMAPS in which I get an error:

python -m jcvi.assembly.allmaps plotall Map.bed --figsize=12x8 --dpi=600 --format=eps
/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/bigdata/bioinfo/pkgadmin/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/jcvi-0.5.5-py2.7.egg/jcvi/assembly/allmaps.py", line 1442, in <module>
    main()
  File "/bigdata/bioinfo/pkgadmin/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/jcvi-0.5.5-py2.7.egg/jcvi/assembly/allmaps.py", line 759, in main
    p.dispatch(globals())
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/jcvi-0.5.5-py2.7.egg/jcvi/apps/base.py", line 77, in dispatch
    globals[action](sys.argv[2:])
  File "/bigdata/bioinfo/pkgadmin/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/jcvi-0.5.5-py2.7.egg/jcvi/assembly/allmaps.py", line 1427, in plotall
    opts, args, iopts = p.set_image_options(xargs, figsize="10x6")
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/jcvi-0.5.5-py2.7.egg/jcvi/apps/base.py", line 316, in set_image_options
    from jcvi.graphics.base import ImageOptions, setup_theme
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/jcvi-0.5.5-py2.7.egg/jcvi/graphics/base.py", line 14, in <module>
    import matplotlib.pyplot as plt
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/colorbar.py", line 34, in <module>
    import matplotlib.collections as collections
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/collections.py", line 27, in <module>
    import matplotlib.backend_bases as backend_bases
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/backend_bases.py", line 62, in <module>
    import matplotlib.textpath as textpath
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/textpath.py", line 15, in <module>
    import matplotlib.font_manager as font_manager
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/font_manager.py", line 1421, in <module>
    _rebuild()
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/font_manager.py", line 1408, in _rebuild
    pickle_dump(fontManager, _fmcache)
  File "/opt/linux/centos/7.x/x86_64/pkgs/python/2.7.5/lib/python2.7/site-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/font_manager.py", line 957, in pickle_dump
    pickle.dump(data, fh)
IOError: [Errno 122] Disk quota exceeded

I am working in a cluster environment, have plenty of storage space, and gave the script 100gb of RAM to work with. Searching the net for "IOError: [Errno 122] Disk quota exceeded" message makes it seem like this might be a general issue in python and not necessarily an issue with jcvi and ALLMAPS, but any insight into this would be appreciated. Interestingly, after running allmaps path, the same error was given as well, but the resulting files (scaffolded fasta, AGP, .chain, etc...) were present in my directory, so I'm not sure how detrimental it was.

I am waiting on our SysAdmin to update from 0.5.5 to the most recent version (0.6.6 ?), so perhaps that could fixe the issue?

Mike

jcvi.assembly.allmaps: problem importing weights file

Hello Haibao,

I am trying to run allmaps using three different maps, but when importing the weights.txt file, I get the following error:

<a lot of logs, all is well>
10:55:41 [base] Load file `weights.txt`
10:55:41 [base] Imported 3 records from `weights.txt`.
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1442, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 759, in main
    p.dispatch(globals())
  File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 77, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1109, in path
    plotall([inputbed, "--links={0}".format(opts.links)])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1438, in plotall
    plot(xargs + [seqid])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1275, in plot
    weights = Weights(weightsfile, mapnames)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 562, in __init__
    pivot_weight, o, pivot = self.get_pivot(mapnames)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 583, in get_pivot
    for m, w in self.items() if m in mapnames)
ValueError: max() arg is an empty sequence

I am running python 2.7 on Ubunutu 14.04 with the last versions of the required packages.

my weights.txt file:

joinmap 1
mstmap 1
rqtl 1

I was hoping you could help me out.

Best,
Erik

Problem estimating gap sizes

Hello there,
I am trying to estimating gap sizes, but I get the following errors:

/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/scipy/interpolate/fitpack2.py:188: UserWarning:
The maximal number of iterations maxit (set to 20 by the program)
allowed for finding a smoothing spline with fp=s has been reached: s
too small.
There is an approximation returned but the corresponding weighted sum
of squared residuals does not satisfy the condition abs(fp-s)/s < tol.
warnings.warn(message)
Traceback (most recent call last):
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/jcvi-0.5.7-py2.7.egg/jcvi/assembly/allmaps.py", line 1574, in
main()
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/jcvi-0.5.7-py2.7.egg/jcvi/assembly/allmaps.py", line 788, in main
p.dispatch(globals())
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/jcvi-0.5.7-py2.7.egg/jcvi/apps/base.py", line 78, in dispatch
globalsaction
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/jcvi-0.5.7-py2.7.egg/jcvi/assembly/allmaps.py", line 908, in estimategaps
verbose=verbose)
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/jcvi-0.5.7-py2.7.egg/jcvi/assembly/allmaps.py", line 736, in compute_all_gaps
minsize, maxsize, verbose=verbose)
File "/share/nas2/genome/biosoft/Python/2.7.8/lib/python2.7/site-packages/jcvi-0.5.7-py2.7.egg/jcvi/assembly/allmaps.py", line 709, in compute_one_gap
converted_dist = int(round(cm_dist / ratio))
ValueError: cannot convert float NaN to integer

I was hoping you could help me out.
Best
Hebbe

Error in gaps

Dear Haibao,

I am trying to use the split option in ALLMAPS but in computing the gaps in the assembly, I encounter the following errors. To make sure the problem is not of my data, I tried it with the test data of ALLMAPS package but I still receive the following errors:

Command: python -m jcvi.formats.fasta gaps scaffolds.fasta

Errors:
16:13:57 [base] Load file scaffolds.fasta
Process Process-5:
Traceback (most recent call last):
File "/Users/homa/anaconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/Users/homa/anaconda2/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(_self._args, *_self._kwargs)
File "/Users/homa/anaconda2/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/apps/grid.py", line 157, in write
isize = queue_in.qsize()
File "/Users/homa/anaconda2/lib/python2.7/multiprocessing/queues.py", line 143, in qsize
return self._maxsize - self._sem._semlock._get_value()
NotImplementedError
16:14:03 [base] sort -k1,1 -k2,2n -k4,4 scaffolds.gaps.bed -o scaffolds.gaps.bed
16:14:03 [base] Load file scaffolds.gaps.bed
16:14:03 [fasta] Write gap (>=100bp) locations to scaffolds.gaps.bed.

Thanks for your help,
Best,
Homa

Provide command/action `version`

It’d be great if you could implement the action/command version, so that python -m jcvi.formats.fasta version prints the version of the module.

Why does whisker disappear in the karyotype figure of 3 species?

Hi HaiBao,
I got the karyotype figure successfully following https://github.com/tanghaibao/jcvi/wiki/MCscan-(Python-version).
Firstly, I got the karyotype figure between A and B.

Secondly, I got the karyotype figure between B and C.

Then I cat B.A.anchors B.C.anchors > B.2Spp.anchors; cat A.bed C.bed > 2Spp.bed; python -m jcvi.compara.synteny screen --minspan=10 --simple B.2Sp.anchors B.2Spp.anchors.simple and modified the context of seqids and layout:
seqids:
ChrA1,ChrA2,ChrA3,...
scaffoldB1,scaffoldB2,scaffoldB3,...
chC1,chC2,chC3,...
layout:

y, xstart, xend, rotation, color, label, va, bed

.8, .1, .8, 0, blue, A, top, 2Spp.bed
.6, .1, .8, 0, red, B, bottom, B.bed
.4, .1, .8, 0, green, C, top, 2Spp.bed

edges

e, 0, 1, B.2Spp.anchors.simple

but the whisker disappeared in figure:

How can I add the whisker?

Sincerely,
Tiany

Hyphens in file naming cause errors during path runtime

Hey All,

If you have files with hyphens in them, they will not be read correctly when converted into the bed format as the python does a split on "-" when loading. You will receive a value error from the python stack. Perhaps this should be checked before loading.

Thank you,

Connor

Buffer overflow with long filenames

Hi all,

Today I ran across a buffer overflow bug while using the ks app:

*** buffer overflow detected ***: /usr/local/bin/yn00 terminated
======= Backtrace: =========
*snip*

When I moved my folder to the home directory (resulting in a shorter absolute filename), everything worked fine. This is probably something in yn00 that is not taking in filenames properly.

By testing, it seems that filenames longer than 95 characters are not accepted by yn00:
95 letters long:

yn00: /usr/local/bin/yn00 /home/vandene/AbcdefghijklmnopqrstuvwxyzAbcdefghijklmnopqrstuvwxyzAbc/syn_analysis/yn-input.ctl >/dev/null
Th2v09886;Th2v13841,4.0198,0.4937,-1.0000,0.4932

96 letter long:

yn00: /usr/local/bin/yn00 /home/vandene/AbcdefghijklmnopqrstuvwxyzAbcdefghijklmnopqrstuvwxyzAbcd/syn_analysis/yn-input.ctl >/dev/null
*** buffer overflow detected ***: /usr/local/bin/yn00 terminated

Perhaps a check to see if filename is longer than that and give a warning?

Chromosome drawing & Ancient genome recontruction

Dear Haibao,

Hello,

1. Is there anyway to use .simple file for "jcvi.graphics.chromosome" ?
   Instead of ID_mapping, it will help to draw genome conservation.

2. For ancient genome reconstruction,
   which is better to use "jcvi.compara.pad" & "jcvi.compara.reconstruction"
   I want to use our genome for reconstruction and I want to check it with JCVI package.
   If you are interested in please let me know by email.

Thank you.

Use OR-Tools

For MIP programming:
https://developers.google.com/optimization/mip/integer_opt

problem with synteny analysis

Hi

I'm trying to do a synteny comparison between physcomitrella and selaginella. I download the data and convert it as in the tutorial. When running python -m jcvi.compara.catalog ortholog i get the following error.

$ python -m jcvi.compara.catalog ortholog patens selaginella
11:47:18 [driver] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
11:47:18 [driver] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
11:47:18 [base] lastdb selaginella selaginella.cds
11:47:30 [base] lastal -u 0 -P 1 -i3G -f BlastTab selaginella patens.cds >patens.selaginella.last
11:54:04 [synteny] Assuming --qbed=patens.bed --sbed=selaginella.bed
11:54:04 [base] Load file patens.bed
11:54:05 [base] Load file selaginella.bed
11:54:05 [blastfilter] Load BLAST file patens.selaginella.last (total 423596 lines)
11:54:05 [base] Load file patens.selaginella.last
11:54:08 [blastfilter] Pp3c8_25090V3 not in patens.bed
11:54:08 [blastfilter] Pp3c8_25090V3 not in patens.bed
11:54:08 [blastfilter] Pp3c24_20910V3 not in patens.bed
11:54:08 [blastfilter] Pp3c24_20910V3 not in patens.bed
...
11:54:08 [blastfilter] too many warnings.. suppressed
11:54:08 [blastfilter] running the cscore filter (cscore>=0.70) ..
11:54:08 [blastfilter] after filter (0->0) ..
11:54:08 [blastfilter] running the local dups filter (tandem_Nmax=10) ..
11:54:08 [blastfilter] after filter (0->0) ..
11:54:09 [synteny] Assuming --qbed=patens.bed --sbed=selaginella.bed
11:54:09 [base] Load file patens.bed
11:54:10 [base] Load file selaginella.bed
11:54:10 [base] Load file patens.selaginella.last.filtered
11:54:10 [synteny] A total of 0 BLAST imported from patens.selaginella.last.filtered.
11:54:10 [synteny] Chaining distance = 20
11:54:10 [base] Load file patens.selaginella.anchors
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/dist-packages/jcvi/compara/catalog.py", line 811, in
main()
File "/usr/local/lib/python2.7/dist-packages/jcvi/compara/catalog.py", line 73, in main
p.dispatch(globals())
File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 87, in dispatch
globalsaction
File "/usr/local/lib/python2.7/dist-packages/jcvi/compara/catalog.py", line 646, in ortholog
"--liftover={0}".format(last)])
File "/usr/local/lib/python2.7/dist-packages/jcvi/compara/synteny.py", line 1464, in scan
summary([anchor_file])
File "/usr/local/lib/python2.7/dist-packages/jcvi/compara/synteny.py", line 1042, in summary
nranchors = [_score(c) for c in clusters] # non-redundant anchors
File "/usr/local/lib/python2.7/dist-packages/jcvi/compara/synteny.py", line 260, in _score
x, y = zip(*cluster)[:2]
ValueError: need more than 0 values to unpack

I suspect it's a mistake with naming. it reads the ID's as for example Pp3c8_25090V3 whereas in the bed file it looks like this

$ head patens.bed
Chr01 8930 13135 Pp3c1_20V3.1 0 +
Chr01 8934 13028 Pp3c1_20V3.2 0 +
Chr01 16795 21548 Pp3c1_40V3.2 0 +
Chr01 18887 19968 Pp3c1_40V3.1 0 +
Chr01 20197 21468 Pp3c1_50V3.1 0 +

Any ideas?

1x2.anchors file

Dear Haibao,

Is there any simple way to generate 1x2.anchors ?

Thank you for your help.

Regards,

Won

BioNano data revisited.

Hi,

This tool seems to be capable of great feats if one has the right data (someone recently showed me some numbers). I do not have genetic map information, but I do have optical map information from BioNano. I might soon have more than one type of optical map (e.g. from different enzymes). I saw in #8, which was open and closed in March, someone already asked about bionano data and the response was that it could be possible to use with AllMaps, but was not officially supported. I was wondering if there has been any more exploration of using BioNano data with AllMaps by the developers or any users who might be reading this, and if so, how did it go?

best,

John

CSV input file

Hi Haibao,
I'm using three linkage maps to improve a draft genome assembly. It's not clear for me if I can use the CSV files containing all the linkage groups (one file for each map with all the 24 LGs) or whether I have to split LGs (producing a csv file for each LG) and analyze them separately.
I have also noticed that sometimes ALLMAPS clear all the temporary files in the working folder at the end of the analysis and sometimes not. Do you know why?
Thanks a lot
Max

problem in running liftOver on grid

Dear Authors,
I have an issue when I run the software. The problem is when the allmaps.py script calls liftOver from the grid. I tried to run liftOver on the grid calling the only the software from the bash shell and it works.

the error that I get is:

16:30:11 [chain] File written to mapMerged.chain.
16:30:11 [base] liftOver -minMatch=1 mapMerged.bed mapMerged.chain mapMerged.lifted.bed unmapped
/bin/bash: liftOver: command not found
16:30:11 [base] sort -k1,1 -k2,2n -k3,3n -k4,4 mapMerged.lifted.bed -o mapMerged.lifted.bed
sort: open failed: mapMerged.lifted.bed: No such file or directory
16:30:11 [base] Load file mapMerged.bed
16:30:11 [allmaps] Map contains 5584 markers in 14 linkage groups.
16:30:11 [base] Load file CUCSA_B10_PCCGS_PacBio_gDNA_contig_1_0.mod.fasta.sizes
16:30:11 [base] Load file mapMerged.chr.agp
16:30:11 [base] Load file mapMerged.agp
16:30:12 [base] Load file mapMerged.lifted.bed
Traceback (most recent call last):
File "/tools/general/app/anaconda-python-2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/tools/general/app/anaconda-python-2.7/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/assembly/allmaps.py", line 1725, in
main()
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/assembly/allmaps.py", line 824, in main
p.dispatch(globals())
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/apps/base.py", line 82, in dispatch
globalsaction
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/assembly/allmaps.py", line 1379, in path
"--figsize={0}".format(opts.figsize)])
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/assembly/allmaps.py", line 1721, in plotall
plot(xargs + [seqid])
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/assembly/allmaps.py", line 1550, in plot
cc = Map(bedfile, function=function)
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/assembly/allmaps.py", line 503, in init
bed = Bed(filename)
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/formats/bed.py", line 110, in init
for line in must_open(filename):
File "/home/ghefd/.local/lib/python2.7/site-packages/jcvi-0.6.9-py2.7.egg/jcvi/formats/base.py", line 391, in must_open
fp = open(filename, mode)
IOError: [Errno 2] No such file or directory: 'mapMerged.lifted.bed'

I think that the software is not found in the bash environment.

Any ideas?

thanks
Luigi

Synteny.py demo file

Hi Haibao

can you give some demo files for using synteny.py ?

allmaps fails in the final phase

I am trying to anchor a draft assembly composed of 795 contigs to a genetic map with 36,828 markers in 11 linkage groups.

ALLMAPS works fine solving the optimization problem for each chromosome, then it fails at the end with the following error message.

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1684, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 820, in main
    p.dispatch(globals())
  File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 81, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1331, in path
    build([inputbed, fastafile])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/assembly/allmaps.py", line 1451, in build
    fromagp([combined_agp, scaffolds, combined_fasta])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/formats/chain.py", line 153, in fromagp
    tSize = componentsizes[tName]
KeyError: 'tig00000085'

(the full log is attached)
log.txt

I want to note that tig00000085 exists in the original scaffold file quiver62fast.fasta, but it seems that allmaps overwrites the quiver62fast.fasta in this line:

11:44:56 [base] cat quiver62fast.chr.fasta quiver62fast.unplaced.fasta >quiver62fast.fasta

and in that new file quiver62fast.fasta tig00000085 does not exist anymore . Would that explain the KeyError? Why is allmaps overwriting the input files? Any advice on how to get rid of the error?

Force ALLmaps to use the user defined orientation to build pseudo chrmosomes

For my assembly, I have the orientation of scaffolds determined by HiC data and genetic map. I can use genetic map only to orient scaffold using ALLmaps. But how could I add the orientation provided by HiC data? I want ALLmaps to use the genetic map and also the specified orientation of scaffolds (from HiC data) to build the final pseudo chromosomes. Thank you!

allmaps fails in agp step

I am trying to anchor a draft assembly composed of 661 contigs to close species but it failed

Bvchr1_un_sca011 is in our comparative species bed file.

`

Traceback (most recent call last):
  File "/home/wyim/bin/python2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/wyim/bin/python2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/assembly/allmaps.py", line 1721, in <module>
    main()
  File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/assembly/allmaps.py", line 820, in main
    p.dispatch(globals())
  File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/apps/base.py", line 82, in dispatch
    globals[action](sys.argv[2:])
  File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/assembly/allmaps.py", line 1362, in path
    gaptype="map")
  File "/data/wyim/won/bin/jcvi/jcvi_env/lib/python2.7/site-packages/jcvi-0.6.6-py2.7.egg/jcvi/formats/agp.py", line 660, in order_to_agp
    size = sizes[ctg]
KeyError: 'Bvchr1_un_sca011'

`

I have no clue why it failed.

Any advice on how to get rid of the error?

question: cross-species synteny

Do you have a recommendation on which tool to use to generate a cross-species synteny map? You mention in your wiki to use BLAST bidirectional hits between the two species. Thanks.

concorde no more in jksrc

Hi

the tool concorde does not look to be available any more in the Jim Kent UCSC source (last version 343).

So recompile it - as mention in https://github.com/tanghaibao/jcvi/wiki/ALLMAPS%3A-How-to-install - could be problematic.

python -m jcvi.assembly.syntenypath bed not working

Hi,

I tried to create synteny map using allmaps but am sure what went wrong. Please have a look if you can notice an obvious problem please.

python -m jcvi.assembly.syntenypath bed Sp34.CEL_WS248.1x1.anchors
11:46:12 [base] Load file Sp34.CEL_WS248.1x1.anchors
11:46:12 [synteny] Assuming --qbed=Sp34.bed --sbed=CEL_WS248.bed
11:46:12 [base] Load file Sp34.bed
11:46:13 [base] Load file CEL_WS248.bed
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/dist-packages/jcvi-0.5.5-py2.7.egg/jcvi/assembly/syntenypath.py", line 422, in
main()
File "/usr/local/lib/python2.7/dist-packages/jcvi-0.5.5-py2.7.egg/jcvi/assembly/syntenypath.py", line 33, in main
p.dispatch(globals())
File "/usr/local/lib/python2.7/dist-packages/jcvi-0.5.5-py2.7.egg/jcvi/apps/base.py", line 77, in dispatch
globalsaction
File "/usr/local/lib/python2.7/dist-packages/jcvi-0.5.5-py2.7.egg/jcvi/assembly/syntenypath.py", line 84, in bed
"{0}:{1}".format(get_number(sseqid), sstart)))
File "/usr/local/lib/python2.7/dist-packages/jcvi-0.5.5-py2.7.egg/jcvi/formats/base.py", line 480, in get_number
return cast(d)
ValueError: invalid literal for int() with base 10: ''

problems in drawing dotplot

I downloaded the fasta and bed file of Athaliana (Arabidopsis thaliana) and Alyrata (Arabidopsis lyrata) from Phytozome as test data. And then I followed the steps from the wiki page of MCscan (Python version). The .anchors file was generated successfully, but an error occurred when I wanted to use command python -m jcvi.graphics.dotplot Ath.Aly.anchors to draw a synteny dotplot.

Here's the error info, and the .anchors file has been attached: Ath.Aly.zip

14:49:19 [synteny] Assuming --qbed=Ath.bed --sbed=Aly.bed
14:49:19 [base] Load file `Ath.bed`
14:49:20 [base] Load file `Aly.bed`
14:49:20 [dotplot] Showing a random subset of 10000 data points (total 22326) for clarity.
14:49:21 [dotplot] xsize=35386 ysize=32670
Traceback (most recent call last):
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/jcvi/graphics/dotplot.py", line 342, in <module>
    minfont=opts.minfont, palette=palette)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/jcvi/graphics/dotplot.py", line 83, in dotplot_main
    minfont=minfont, palette=palette, chrlw=chrlw, title=title)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/jcvi/graphics/dotplot.py", line 180, in dotplot
    th = TextHandler(fig)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/jcvi/graphics/base.py", line 52, in __init__
    self.build_height_array(fig)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/jcvi/graphics/base.py", line 66, in build_height_array
    w, h = TextHandler.get_text_width_height(fig, size=i)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/jcvi/graphics/base.py", line 58, in get_text_width_height
    tp = mpl.textpath.TextPath((0,0), txt, size=size, usetex=usetex)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/matplotlib/textpath.py", line 434, in __init__
    self._vertices, self._codes = self.text_get_vertices_codes(prop, s, usetex=usetex)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/matplotlib/textpath.py", line 515, in text_get_vertices_codes
    verts, codes = text_to_path.get_text_path(prop, s, usetex=True)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/matplotlib/textpath.py", line 139, in get_text_path
    glyph_info, glyph_map, rects = self.get_glyphs_tex(prop, s)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/matplotlib/textpath.py", line 290, in get_glyphs_tex
    self.tex_font_map = dviread.PsfontsMap(dviread.find_tex_file('pdftex.map'))
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/site-packages/matplotlib/dviread.py", line 853, in find_tex_file
    stderr=subprocess.PIPE)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/subprocess.py", line 709, in __init__
    errread, errwrite)
  File "/PUBLIC/software/public/System/Python-2.7.6/lib/python2.7/subprocess.py", line 1326, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Thanks for your attention.
Xiaofei Zeng

BioNano data

Dear Haibao,

Is there anyway to use Bionano data?

The tutorial shows only opgen data and I think it's a little bit difficult to use.

But I just want to check it.

Won

[Grabseed] Background color

Hello Haibao,

I have a suggestion regarding Grabseed background.

Grabseed shows color of seed and sometimes it grab the background as a seed.

If user can define the background color as an input option, Grabseed can exclude the similar color based on their color name.

For example backgroud is blue (from user option) then Grabseed can exclude the similar color such as blue, royalblue, darkblue, lightblue and etcs from results.
https://en.wikipedia.org/wiki/Web_colors

I should move to Bay area to drink with you for weekend :)

Cheers,

Error: ZeroDivisionError: float division by zero

Aloha!

I've run Allmaps on variations of my data without error previously, but on the last run I received the error: "ZeroDivisionError: float division by zero" when it was in the process of writing the chr.pdfs. Do you happen to know how I can fix this?

Attached is the error output.

Thanks!
Sheina
Allmaps_21053.err.txt

Sex chromosome in ALL

Cross-species synteny 1x1.anchor file 3 column

Hi

Could you just shortly explain what the third column defines of the 1x1.anchor file?

Ca_02944 Medtr2g105830 2116
Ca_02943 Medtr2g105680 3858
Ca_02942 Medtr2g105640 1908

thx!

Grabseed with test data

Dear Haibao,

"grabseeds" simply wouldn't work on test image.

Will you please check this?

python -m jcvi.graphics.grabseeds seeds test.JPG
17:25:35 [Image] Error closing: 'NoneType' object has no attribute 'close'
17:25:35 [grabseeds] Image ./test.resize.jpg loaded (376px x 250px).
17:25:35 [Image] Error closing: 'NoneType' object has no attribute 'close'
17:25:35 [grabseeds] Image ./test.resize.jpg loaded (376px x 250px).
17:25:35 [grabseeds] Running canny edge detection ...
/usr/local/lib/python2.7/dist-packages/skimage/filters/init.py:28: skimage_deprecation: Call to deprecated function canny. Use skimage.feature.canny instead.
def canny(*args, **kwargs):
17:25:35 [grabseeds] Find objects with pixels between 47 (0.05%) and 47000 (50%)
17:25:35 [grabseeds] A total of 52 objects identified.
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/wyim/.local/lib/python2.7/site-packages/jcvi/graphics/grabseeds.py", line 591, in
main()
File "/home/wyim/.local/lib/python2.7/site-packages/jcvi/graphics/grabseeds.py", line 114, in main
p.dispatch(globals())
File "/home/wyim/.local/lib/python2.7/site-packages/jcvi/apps/base.py", line 84, in dispatch
globalsaction
File "/home/wyim/.local/lib/python2.7/site-packages/jcvi/graphics/grabseeds.py", line 533, in seeds
square = img[(y0 - d):(y0 + d), (x0 - d):(x0 + d)]
TypeError: slice indices must be integers or None or have an index method

Thank you .

Won

image reading

"seeds" simply wouldn't work on this image. I confirmed the problem to be because it's in grayscale, since it worked just fine after I changed the color mode using GIMP.

kyle@bioinfkyle ~/corn/scan_test/scan/quarter $ python -m jcvi.graphics.grabseeds seeds ./img009.jpg 
/usr/local/lib/python2.7/dist-packages/skimage/filter/__init__.py:6: skimage_deprecation: The `skimage.filter` module has been renamed to `skimage.filters`.  This placeholder module will be removed in v0.13.
  warn(skimage_deprecation('The `skimage.filter` module has been renamed '
17:11:41 [grabseeds] Image `./img009.jpg` resized from 2472px:3414px to 1000px:1381px
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/jcvi/graphics/grabseeds.py", line 586, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/jcvi/graphics/grabseeds.py", line 114, in main
    p.dispatch(globals())
  File "/usr/local/lib/python2.7/dist-packages/jcvi/apps/base.py", line 81, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python2.7/dist-packages/jcvi/graphics/grabseeds.py", line 438, in seeds
    oimg = load_image(resizefile)
  File "/usr/local/lib/python2.7/dist-packages/jcvi/graphics/grabseeds.py", line 399, in load_image
    h, w, c = img.shape
ValueError: need more than 2 values to unpack

Concatenating scaffolds

Hi,

I was wondering when concatenating scaffolds reported in the file (file.lifted.bed), what is the gap
length inserted between the two scaffolds?

In some cases in other genomes, for example they use a value of 5000 bp or 100 bp between scaffolds when they concatenate them to one, however, I couldn't figure out how Allmaps does this.

Thank you,
Homa

ZeroDivision error when plotting karytype

Dear JCVI developer,

i would like to plot a karyotype but get a nasty Zerodivision error back.

python -m jcvi.graphics.karyotype seqids layout
21:41:31 [base] Load file `layout`
21:41:31 [base] Load file `paxillaris.bed`
21:41:32 [base] Load file `tomato_p.bed`
Traceback (most recent call last):
  File "/home/moser/Tools/anaconda3/envs/python2/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/moser/Tools/anaconda3/envs/python2/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/moser/Tools/ALLMAPs/code/jcvi/graphics/karyotype.py", line 316, in <module>
    main()
  File "/home/moser/Tools/ALLMAPs/code/jcvi/graphics/karyotype.py", line 304, in main
    Karyotype(fig, root, seqidsfile, layoutfile)
  File "/home/moser/Tools/ALLMAPs/code/jcvi/graphics/karyotype.py", line 280, in __init__
    tr = Track(root, lo, gap=gap, height=height, lw=lw, draw=False)
  File "/home/moser/Tools/ALLMAPs/code/jcvi/graphics/karyotype.py", line 130, in __init__
    ratio = span / total
ZeroDivisionError: float division by zero

I suspect the layout file to cause the error. Could the rotation = 0 cause the problem?

i changed it to integers but the error persits.

Could you give me some help to resolve this?

Thank you very much,
Michel

layout:

# y, xstart, xend, rotation, color, label, va,  bed
 .6, .1, .8, 0, blue, axillaris, top, paxillaris.bed
 .4, .1, .8, 0, red, tomato, top, tomato_p.bed
# edges
e, 0, 1, paxillaris.tomato_p.anchors.simple

seqids:

SL2_40ch01,SL2_40ch02,SL2_40ch03,SL2_40ch04,SL2_40ch05,SL2_40ch06,SL2_40ch07,SL2_40ch08,SL2_40ch09,SL2_40ch10,SL2_40ch11,SL2_40ch12
chrchr1,chrchr2,chrchr3,chrchr4,chrchr5,chrchr6,chrchr7