iychoi/biospectra

BioSpectra: Search, Classify and Inspect metagenomic samples

biospectra

BioSpectra: A Metagenomic Read Classification System

BUILD

System Requirements: Java 1.7.0, Ant

This code uses "Ant" build system for compilation. So you may need to install "Ant" build system before you start compilation. Please type "ant" in the root of this package (where "build.xml" exists).

Jar package will be generated under "/dist" path. All dependencies(lucene 4.0) will also be copied to "/dist/lib" path after the compilation.

EXECUTION

Index Construction

biospectra_indexing.sh <path_to_reference_FASTA_files> <path_of_index>

BioSpectra accepts input FASTA path as a directory. In this case, BioSpectra will recursively find all FASTA files and process.

Classification

biospectra_classify.sh <path_of_index> <path_to_query_FASTA_files> <path_to_output_dir>

BioSpectra accepts query FASTA path as a directory. In this case, BioSpectra will recursively find all FASTA files and process. Classification results (in form of json) will be stored as a file with the same filename as input with a extension ".result". Also it automatically generates a summary file that shows timetaken of classifications and total number of "classified", "vague" or "unknown" reads.

BENCHMARK DATASETS

Source package contains a script to download NCBI bacteria reference datasets at /benchmark/down_ncbi_bacteria_reference.sh. To work with this datasets, you will need at least 25GB of disk space including size of generated index form this datasets.

A simulated query datasets extracted from NCBI bacteria reference datasets with fixed error ratio is provided at /benchmark/bacteria_ncbi_simulated directory. For example, a /benchmark/bacteria_ncbi_simulated/sample_0.04.fa file contains reads (each in length about 100) randomly extracted from NCBI bacteria datasets with 4% of sequencing errors at random positions.

Another simulated query datasets extracted from FAMeS(Fidelity of Analysis of Metagenomic Samples) datasets are provided at /benchmark/simHC.20.500 directory. Each file in the directory contains 500 reads that are extracted by a single species (referring a filename).

biospectra's Issues

Handle command line arguments

Need to handle command line commands and usage prints.

Accuracy test

Use the same dataset as CLARK.

They used 3 datasets from the Kraken website (HiSeq, MiSeq and simBA-5) which are located here:
https://ccb.jhu.edu/software/kraken/dl/accuracy.tgz

and one additional dataset they made available here:
simHC.20.500
http://clark.cs.ucr.edu/Download/simHC.20.500.fa.gz

Post processing of high abundance reads or no matching reads.

High abundance reads can be found when the reads are shared across many organisms. In this case, we should do post-process to classify them by using 1st-pass results.

No matching reads are basically "unknown".

Enhance indexing performance

Ref.
http://www.onjava.com/pub/a/onjava/2003/03/05/lucene.html?page=1

Fasta File as a query

Take Fasta file as a query file. So that metagenomic sample file will be taken as an input of search system.

This will output:

• matching reads : total count, organism, confidence score
• unknown reads : total count

handle reads larger than 4KB in indexing

Currently, k-mer tokenizer has a bug that only handles first 4KB of reads. This need to be fixed as read can bigger than that.

Spaced k-mers for accuracy

Try spaced k-mers (simple masking) mentioned below paper.

http://bioinformatics.oxfordjournals.org/content/early/2015/08/10/bioinformatics.btv419.full

Optimistic query term drop for kgrams

Drop redundant terms in case described below:
Query: AAAABBBBCCCC (4mers)
to AAAA, AABB, BBBB, BBCC, CCCC
drop AAAB, ABBB, BCCC
when AAAA, AABB, BBBB, BBCC, CCCC returns the same results (meaning no search for dropping kgrams are redundant)

Replace FASTA parser

Current FASTA parser uses String class as a data structure of parsed read.

• parse "GI" in header
• handle very long sequence in a read (up to GB scale) using InputStream.

Document scoring

Need to research what scoring function default lucene uses.
Need to optimize it for better performance and confidence.