mjsull / chromatiblock Goto Github PK
View Code? Open in Web Editor NEWColinear block visualisation tool
License: GNU General Public License v3.0
Colinear block visualisation tool
License: GNU General Public License v3.0
Not an issue, but can chromatiblock be parallelized (with the option to specify a number of threads)? :)
Hello,
As said in the title, the demo for H. pylori linked in your ReadMe is down; it returns an error 404 when following the link on your ReadMe (supposedly redirecting toward https://mjsull.github.com/chromatiblock/H_pylori.html ).
Is there any other demo available somewhere?
Thank you!
Hi,
I was wonder how to add gene annotations (the triangles) to core blocks? In your examples (and my own data), I can only get the annotations to appear on noncore blocks using the -gb and -gl flags.
Thanks!
Tony
Hi:
I'm looking at gene rearrangement in the neighbourhood flanking a single core block in related plasmid sequences - when the core block is oriented in the strand=+ direction, the sequence is correctly graphed in left flank -> core -> right flank order.
However, when the orientation of the core block is reversed (strand=-), the sequence is graphed as core -> left+right flank, with the non-core blocks from both flanks overlaid over each other on the right. Nothing is plotted to the left of the core block (refer to the last 2nd-4th row of the attached screenshot)
Chromatiblock was installed and executed on a fresh conda env (Python 3.8.6) as recommended by the Github.
I have been trying to run your tool on some E. coli sequences and I ran into a few errors. (I am using Version: 0.3.0
installed with conda
.)
The first is when I inserted a bunch of predicted plasmid contigs. They were quite different contigs and not everything could be aligned into a 'core genome'. However, the only message Chromatiblock provided was "IndexError: list index out of range" (see below). If you could provide a little message with a clue for the user what the actual problem could be and how to fix this, that would be much appreciated.
Traceback (most recent call last):
File "/miniconda3/envs/chromatiblock/bin/Chromatiblock.py", line 1410, in <module>
order_blocks_core(block_dict)
File "/miniconda3/envs/chromatiblock/bin/Chromatiblock.py", line 668, in order_blocks_core
if k.block == core_order[0][0]:
IndexError: list index out of range
The second error was with a set of whole genomes (scaffolds from a WGS experiment). Chromatiblock started collecting fasta sequences into the input.fasta
file, but stopped halfway. It just stopped, without a warning. I found the message "fasta entry duplicated" in a log file, but when I checked the input files, there were no duplicate fasta header lines. Besides, Chromatiblock seems to just count the sequences (naming them 0_0, 0_1, 0_2, ... 0_n), so I don't see how a duplicate would be a problem in the end. Will it not just become one of those numbers to Chromatiblock? Anyway, in such a case it would be helpful to know what the duplicate is, and where to find it (what file it is in). (E.g. print 'fasta' and 'zename' in line 444 of Chromatiblock.py
.) That would also help users prepare their files properly for Chromatiblock.
Hi, again not an issue but a recommendation for the interactive plot: Maybe the genome names could be listed on the left side of the MSA and colinear-blocks ? This way I don't always have to look up the order.
<script src="http://ariutta.github.io/svg-pan-zoom/dist/svg-pan-zoom.min.js"></script>\n'
Hello.
I ran the script like this:
chromatiblock -d ~/beastruns/chromatiblock/raw/ -w ~/beastruns/chromatiblock/tmp -o figure.svg
and it sad this to me:
error: parse error in /home/alexander/beastruns/chromatiblock/tmp/input.fasta on line 1: empty sequence
Traceback (most recent call last):
File "/Software/chromatiblock", line 1479, in <module>
block_dict = get_blocks(args.working_directory, name_dict, seq_dict, gene_list, cats)
File "/Software/chromatiblock", line 562, in get_blocks
with open(sibel_dir + '/blocks_coords.txt') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/alexander/beastruns/chromatiblock/tmp/blocks_coords.txt'
The ~/beastruns/chromatiblock/raw/ directory contains 85 separate fasta sequences.
I've checked the input.fasta file and sure enough, it was empty. Then i tried to run it like this:
chromatiblock -f ~/beastruns/chromatiblock/raw/* --force --keep -w ~/beastruns/chromatiblock/tmp -o figure.svg
and it failed due to wrong file concatenation.
error: parse error in /home/alexander/beastruns/chromatiblock/tmp/input.fasta on line 2: illegal character: >
Traceback (most recent call last):
File "/home/alexander/miniconda3/envs/chromatiblock/bin/chromatiblock", line 1479, in <module>
block_dict = get_blocks(args.working_directory, name_dict, seq_dict, gene_list, cats)
File "/home/alexander/miniconda3/envs/chromatiblock/bin/chromatiblock", line 562, in get_blocks
with open(sibel_dir + '/blocks_coords.txt') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/alexander/beastruns/chromatiblock/tmp/blocks_coords.txt'
The sequences were like:
>0_0
atatata>1_0
tatatata>2_0
etc.
I traced the error back to write_fasta_sibel function and changed line 423 to
out.write(line+'\n')
Now it works with -f ~/beastruns/chromatiblock/raw/*
, but -d ~/beastruns/chromatiblock/raw/
still writes an empty file.
Also, it says
No core blocks found. No regions >1000bp were found once in all genomes. Please use more closely related genomes.
though i set minimum block size to 100, is it ok?
I ran chromatiblock on a simple dataset and it worked great, however when I added in more genomes, order, and gene annotations as per the advanced example, I get the following error:
./chromatiblock -w ./working -o image.pdf -gf trnastart.txt -l orderlist.txt -c categories.txt -d ./
Simplification stage 1 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Simplification stage 2 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Simplification stage 3 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Simplification stage 4 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Finding synteny blocks and generating the output...
Traceback (most recent call last):
File "./chromatiblock", line 1555, in
draw_blocks(core_array, noncore_pos, core_size, args.out, args.genome_height, args.gap, legend_size, args.working_directory, args.ppi, args.categorise != None, args.svg_pan_zoom_location, args.hue_start, args.hue_end)
File "./chromatiblock", line 1283, in draw_blocks
svg.drawOutRect(figure_width / 100 + 50 + gap, curr_y, figure_width / 200, genome_height, color, out_color)
TypeError: unsupported operand type(s) for +: 'float' and 'NoneType'
Here is a link to download the full input and working directory from the run that failed.
Thanks!
% ./Chromatiblock.py --version
Chromatiblock 0.3.0
to stdout and return code 0
Looks more professional?
And matches repo name
and run as chromatiblock
rather than python Chromatiblock.py
you know, in case you rewrite it in Perl one day :-P
I've narrowed it down to a particular genome (B_erythrophleiGAS401.fna) from NCBI that is causing it to error. Here is a smaller dataset of two genomes that gives a similar error using the newest version of chromatiblock. The output I get is in the output file, also posted here:
./chromatiblock -w ./ -o image.pdf -d ./
output:
Simplification stage 1 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Simplification stage 2 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Simplification stage 3 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Simplification stage 4 of 4
Enumerating vertices of the graph, then performing bulge removal...
[..................................................]
Finding synteny blocks and generating the output...
Traceback (most recent call last):
File "/local/cluster/lib/python3.7/xml/etree/ElementTree.py", line 1630, in feed
self.parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 3, column 9
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./chromatiblock", line 1555, in
draw_blocks(core_array, noncore_pos, core_size, args.out, args.genome_height, args.gap, legend_size, args.working_directory, args.ppi, args.categorise != None, args.svg_pan_zoom_location, args.hue_start, args.hue_end)
File "./chromatiblock", line 1351, in draw_blocks
cairosvg.svg2pdf(url=os.path.join(working_dir, "temp.svg"), write_to=out_file, dpi=ppi)
File "/local/cluster/lib/python3.7/site-packages/cairosvg/init.py", line 80, in svg2pdf
output_height=output_height)
File "/local/cluster/lib/python3.7/site-packages/cairosvg/surface.py", line 144, in convert
**kwargs)
File "/local/cluster/lib/python3.7/site-packages/cairosvg/parser.py", line 407, in init
forbid_external=not unsafe)
File "/local/cluster/lib/python3.7/site-packages/defusedxml/common.py", line 131, in fromstring
parser.feed(text)
File "/local/cluster/lib/python3.7/xml/etree/ElementTree.py", line 1632, in feed
self._raiseerror(v)
File "/local/cluster/lib/python3.7/xml/etree/ElementTree.py", line 1531, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 3, column 9
Thanks!
Originally posted by @alexweisberg in #6 (comment)
Hello!
I´ve been using Chromatiblock, and have managed to produce apparently correct visualizations. I´ve tried adding a protein list containing various aminoacid sequences for annotation using -gb, however, some of these (which I am sure exist in the genome, given that I am taking the protein annotations from those genomes for the visualization) are not showing. I would then like to know if there is a maximum number of proteins to be displayed, or a minimum distance between them that exist for them to be shown.
Thanks in advance!
🔗 openjournals/joss-reviews#2451
I'll put in this issue my comments. So far I really enjoyed trying this tool that looks really useful.
The section "Demo" should be expanded (as it could be the very first attempt of using the software by a user) adding:
A section describing the produced output:
-o
switch can be described better, it's not immediately clear that can be used to select a different output format by changing the extension.--keep-intermediates
or similar is specified. In this way it would not be so necessary to describe the files)Minor issues:
<program> Copyright (C) <year> <name of author>
Summary: the program works as expected and is simple to run.
The program if invoked with -h
provides the expected documentation.
If invoked without parameters will print "No input files found use -f or -d", and I would add "Type --help
for usage".
Downloading the repository and trying to run the program without installing the dependencies results in:
/bin/sh: Sibelia: command not found
Traceback (most recent call last):
File "./chromatiblock", line 1548, in <module>
block_dict = get_blocks(args.working_directory, name_dict, seq_dict, gene_list, cats)
File "./chromatiblock", line 690, in get_blocks
with open(sibel_dir + '/blocks_coords.txt') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'simple_wd/blocks_coords.txt'
For beginner users, this might not be an easy to understand message, and I would suggest to consider adding a "dependency check".
Here I found some issues:
chromatiblock -d simple_demo -w simple_wd -o H_pylori.pdf -l simple_demo/order_list.txt
, output here). I couldn't open the file with Inkscape.Traceback (most recent call last):
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/xml/etree/ElementTree.py", line 1693, in feed
self.parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 3, column 9
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./chromatiblock", line 1555, in <module>
draw_blocks(core_array, noncore_pos, core_size, args.out, args.genome_height, args.gap, legend_size, args.working_directory, args.ppi, args.categorise != None, args.svg_pan_zoom_location, args.hue_start, args.hue_end)
File "./chromatiblock", line 1351, in draw_blocks
cairosvg.svg2pdf(url=os.path.join(working_dir, "temp.svg"), write_to=out_file, dpi=ppi)
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/site-packages/cairosvg/__init__.py", line 76, in svg2pdf
return surface.PDFSurface.convert(
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/site-packages/cairosvg/surface.py", line 142, in convert
tree = Tree(
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/site-packages/cairosvg/parser.py", line 405, in __init__
tree = ElementTree.fromstring(
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/site-packages/defusedxml/common.py", line 131, in fromstring
parser.feed(text)
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/xml/etree/ElementTree.py", line 1695, in feed
self._raiseerror(v)
File "/Users/telatina/miniconda3/envs/chromatiblock/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 3, column
The code lacks comments, and this makes it hard to contribute to it. This should be improved before publication.
Comment/Question: Both SVG and HTML are hardcoded in the script. While I understand that this makes more portable the script and removes the problem of missing external files, I wonder if using specialized libraries would make the program more future-proof.
Since there are no automatic tests, it would be useful to add a bash script that automatically:
Hi @mjsull,
Chromatiblock is a great tool that I think many people fill find useful. For the most part I had no issues going from multiple FASTAs to Chromatiblock output.
Unfortunately in the current version (from Conda), there is an issue with the SVG output (also affects PDF and PNG outputs) that will need to be fixed to complete this review. I don't foresee this being a huge issue to fix since at first glance it appears to be syntax related.
Below are my comments related to openjournals/joss-reviews#2451. While reading my comments, please keep in mind, many of them are minor and only suggestions for usability improvements.
Feel free to reach out for clarifications.
Thank you for such a great tool!
Robert
Currently the SVG creation has an issue which also breaks PDF and PNG creation.
Here's output from trying to open the produced SVG in Firefox. It opened in Inkscape but was a truncated version the HTML output.
XML Parsing Error: not well-formed
Location: file:///F:/review/test.svg
Line Number 3, Column 10:
<a href=test_files/0.html> <rect stroke="#5b0a0a" stroke-opacity="1.000000" stroke-alignment="inner"
---------^
PDF and PNG also produce similar error: xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 3, column 9
SVG and HTML outputs for your review
Italics for C. difficile
The genus (Clostridium) is given in Figure 1's description, but I think it should also be given at first mention in the main text.
An example of a global alignment of 28 complete C. difficile genomes is shown in Fig. 1A.
An example of a global alignment of 28 complete Clostridium difficile genomes is shown in Fig. 1A.
Typo in Installation Via Conda. (environemtn)
The link in Direct Download is broken. It looks like it also points to a previous release. An alternative could be to add a Github Release (latest by date) badge from shields.io could with this for future releases.
requirements section
a. Capitalize header (Requirements)
b. A subheader named Optional would help separate whats needed. At the moment, only Python >= 3.6.0 is all I see because the bold.
c. Typo in Sibelia description (autmoatic)
d. Some file formats are all caps (PDF, PNG) some are not (svg, html)
e. Include links to the optional tools
In Figure Description
a. Italics on Cdiff
b. "ge-nomes" typo
# OS
cat /proc/version
Linux version 4.19.0-9-amd64 ([email protected]) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.118-2+deb10u1 (2020-06-07)
# Conda
conda --version
conda 4.8.3
# Setup
conda create -y -n chromatiblock-review -c conda-forge -c bioconda chromatiblock
chromatiblock
. Otherwise, at minimum the required parameters should be described (-f
or -d
, -w
, and -o
).chromatiblock
No input files found use -f or -d
The usage should be improved with the use of positional arguments or subparsers. Currently there is no separation between the required arguments and truly optional arguments. I think based on the either or for -f
and -d
, subparsers would be easiest to implement.
Working directory is a required parameter (https://github.com/mjsull/chromatiblock/blob/master/chromatiblock#L1489), but its not in the usage epilog (https://github.com/mjsull/chromatiblock/blob/master/chromatiblock#L1428-L1432)
If no extension is given (e.g. -o test
) for the output file, it checks to see if test.svg exists (after its already rerun everything) and if so doesn't overwrite it (https://github.com/mjsull/chromatiblock/blob/master/chromatiblock#L1356-L1360). But this check is made pointless if chromatiblock is run again with -o test.svg
since the existing test.svg
will be overwritten (https://github.com/mjsull/chromatiblock/blob/master/chromatiblock#L1344-L1345) because the check doesn't happen.
SVG file that was generated did not work. Attached my outputs for your inspection. When opening the SVG with Chrome or Firefox I get:
XML Parsing Error: not well-formed
Location: file:///F:/review/test.svg
Line Number 3, Column 10:
<a href=test_files/0.html> <rect stroke="#5b0a0a" stroke-opacity="1.000000" stroke-alignment="inner"
---------^
Traceback (most recent call last):
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/xml/etree/ElementTree.py", line 1693, in feed
self.parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 3, column 9
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rpetit/miniconda3/envs/chromatiblock-review/bin/chromatiblock", line 1555, in <module>
draw_blocks(core_array, noncore_pos, core_size, args.out, args.genome_height, args.gap, legend_size, args.working_directory, args.ppi, args.categorise != None, args.svg_pan_zoom_location, args.hue_start, args.hue_end)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/bin/chromatiblock", line 1351, in draw_blocks
cairosvg.svg2pdf(url=os.path.join(working_dir, "temp.svg"), write_to=out_file, dpi=ppi)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/cairosvg/__init__.py", line 76, in svg2pdf
return surface.PDFSurface.convert(
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/cairosvg/surface.py", line 142, in convert
tree = Tree(
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/cairosvg/parser.py", line 405, in __init__
tree = ElementTree.fromstring(
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/defusedxml/common.py", line 131, in fromstring
parser.feed(text)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/xml/etree/ElementTree.py", line 1695, in feed
self._raiseerror(v)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 3, column 9
Traceback (most recent call last):
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/xml/etree/ElementTree.py", line 1693, in feed
self.parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 3, column 9
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rpetit/miniconda3/envs/chromatiblock-review/bin/chromatiblock", line 1555, in <module>
draw_blocks(core_array, noncore_pos, core_size, args.out, args.genome_height, args.gap, legend_size, args.working_directory, args.ppi, args.categorise != None, args.svg_pan_zoom_location, args.hue_start, args.hue_end)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/bin/chromatiblock", line 1353, in draw_blocks
cairosvg.svg2png(url=os.path.join(working_dir, "temp.svg"), write_to=out_file, dpi=ppi)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/cairosvg/__init__.py", line 66, in svg2png
return surface.PNGSurface.convert(
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/cairosvg/surface.py", line 142, in convert
tree = Tree(
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/cairosvg/parser.py", line 405, in __init__
tree = ElementTree.fromstring(
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/site-packages/defusedxml/common.py", line 131, in fromstring
parser.feed(text)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/xml/etree/ElementTree.py", line 1695, in feed
self._raiseerror(v)
File "/home/rpetit/miniconda3/envs/chromatiblock-review/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 3, column 9
(chromatiblock-review) rpetit@ogston:~/review$ chromatiblock --version
Version 0.4.2(chromatiblock-review) rpetit@ogston:~/review$
--input_directory
checks for common extensions (https://github.com/mjsull/chromatiblock/blob/master/chromatiblock#L1481). Heads up the Genbank files downloaded from NCBI Assembly, they will have 'gbff' extension. Might be useful to allow the user to specify the extension(s).
It would be useful to add the parameter in the missing -w
and -o
messages. E.g. "Please specify a working directory (-w)."
-o
/--out
I think would be better split into something like --output_format
(default: svg (could use choices) and --prefix
(default: chromatiblock). Because at the moment, a typo in in the extension will cause a SVG file to be created. Example: -o test.htm
would give test.htm.svg
. This could at least prevent the user from completing a chromatiblock run only to find out they goofed on the extension.
Currently existing outputs and working directories are overwritten. It might be useful to notify the user and exit unless they force existing outputs to be overwritten (e.g. --force
).
Is it necessary to keep the working directory? Alternative is to delete, and have the user request it kept.
With the the PDF and PNG, being a SVG conversion, I think would be useful to output HTML or Images (SVG,PNG,PDF). Currently if I ask for SVG, and change my mind to PDF, chromatiblock reruns everything to just convert the SVG to PDF. So if I want all three images I have to run Chromatiblock three times.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.