epaule/blobtools2

Redesigning BlobTools to support interactive data exploration

BlobTools2

A new implementation of BlobTools with support for interactive data exploration via the BlobToolKit viewer.

More information and tutorials are available at blobtoolkit.genomehubs.org/blobtools2/

BlobToolKit is described in our BlobToolKit paper:

BlobToolKit – Interactive quality assessment of genome assemblies Richard Challis, Edward Richards, Jeena Rajan, Guy Cochrane, Mark Blaxter G3: GENES, GENOMES, GENETICS April 1, 2020 vol. 10 no. 4 1361-1374; https://doi.org/10.1534/g3.119.400908

About

Similar to BlobTools v1, BlobTools2 is a command line tool designed to aid genome assembly QC and contaminant/cobiont detection and filtering. In addition to supporting interactive visualisation, a motivation for this reimplementation was to provide greater flexibility to include new types of information, such as BUSCO results and BLAST hit distributions.

BlobTools2 supports command-line filtering of datasets, assembly files and read files based on values or categories assigned to assembly contigs/scaffolds through the blobtools filter command. Interactive filters and selections made using the BlobToolKit Viewer can be reproduced on the command line and used to generate new, filtered datasets which retain all fields from the original dataset.

BlobTools2 is built around a file-based data structure, with data for each field contained in a separate JSON file within a directory (BlobDir) containing a single meta.json file with metadata for each field and the dataset as a whole. Additional fields can be added to an existing BlobDir using the blobtools add command, which parses an input to generate one or more additional JSON files and updates the dataset metadata. Fields are treated as generic datatypes, Variable (e.g. gc content, length and coverage), Category (e.g. taxonomic assignment based on BLAST hits) alongside Array and MultiArray datatypes to store information such as start, end, NCBI taxid and bitscore for a set of blast hits to a single sequence. Support for new analyses can be added to BlobTools2 by creating a new python module with an appropriate parse function.

To learn more about the development of the BlobTools approach, take a look at the papers by Laetsch DR and Blaxter ML, 2017 and Kumar et al., 2013.

Installing

The most up to date instructions can be found at blobtoolkit.genomehubs.org/install/.

Examples

Create a new dataset

• blobtools create

Adding more data

• Coverage
• BLAST/Diamond hits
• BUSCO results

Setting dataset metadata

• File-based metadata
• Adding taxonomy information
• Updating individual keys
• Adding external links

Filtering datasets

• Specifying filter parameters
• Filtering data files

Visualising datasets

• Interactive visualisation
• Command line visualisation

Contributing

If you find a problem or want to suggest a feature, please submit an issue.

If you want to contribute code, pull requests are welcome but please make sure your code passes the linting, testing and style checks before submitting a pull request. Additional development dependencies are listed in requirements.txt

Run linter/testing locally:

./run_tests.sh

Set up pre-commit hook to automate test running:

ln -s ../../pre-commit.sh .git/hooks/pre-commit