noamteyssier/gia

gia: Genomic Interval Arithmetic

Hello!

I'm Noam Teyssier, a Bioinformatics PhD Candidate in the Kampmann Lab and Goodarzi Lab at UCSF.

I work at the intersection of functional genomics, machine learning, dynamical modeling, and systems biology in the context of neurodegenerative diseases.

I highly value open source and try to share all the tools I've developed during my own research.

gia's Issues

Mix file formats when applicable

Need to be able to mix file formats (i.e. bed3, bed6, bed12) when provided as inputs to operations requiring 2 files.

Multiple Files

• closest
• intersect
• subtract

Single Files

• complement
• extend
• get_fasta
• merge
• sample
• sort

Can close #65 once done

Use consistent naming for format shorthand

add badges to readme

Arbitrary number of inputs to functions

Suggested in #60

Implement complement

restructure cli module and use argument structs to minimize function call signatures

performance claims

I meant to do some testing on my own, but I may never get there. I'm one of the authors of BEDOPS. It is not easy to imagine a 6x or so improvement in runtimes, as these are linear (or n log n for sorting) time algorithms in bedops/closest-features utilities.

There are a couple of things that stand out to me in the bioarxiv paper. Mainly, timed tests are at most 1 second for the slowest tool which indicates very, very small inputs (Figures 1 and 2). If the trend held with large inputs, that would be far more interesting and impressive. Right now, the differences might be attributable to things that do not generalize beyond 1 second, for example.

The memory overhead shown for bedops (Figure 5) makes me think that they used the "megarow" build of BEDOPS. That build is meant for very large sequencing results (nanopore and pacbio). It scales to those much larger data at the cost of some small memory overhead but also considerable time overhead. It would be worth measuring time/memory against that larger build but also against the more popular (and default) build for utilities in BEDOPS.

You can use the switch-BEDOPS-binary-type utility to switch between typical (default) and megarow builds of utilities in BEDOPS.

Retain BED6 format?

Hi Noam!

I'm running a lot of bedtools intersect commands that I would love to replace with gia, but I was relying on the information in the bed6 format being retained.

e.g. fileA.bed
chr1 29300 29400

e.g. fileB.bed
chr1 29301 29400 CTAACTTTCCTATCAT-1 41 +
chr1 29328 29427 CTAACTTTCCTATCAT-1 40 -

e.g. output I need with the cell barcode.
chr1 29301 29400 CTAACTTTCCTATCAT-1 41 +

In this case, would I need to use bedrs instead of gia & create an interval type with my additional field?
-- Amanda

Stranded Methods

This is an issue to track the development of implementing stranded methods

• Closest
• Complement
• Extend
• Get Fasta
• Intersect
• Merge
• Random
• Sample
• Sort
• Subtract

Use rayon for parallel sorting of interval sets

include get_fasta

Implement streamed complement

Increased File Type Support

• BED12 - #53
• VCF - #49

Use built in bed formats in bedrs

Incorporate Streamed Methods

Streamable Methods

• Closest
• Complement
• Extend
• Intersect
• Subtract

Named Streamable Methods

• Closest
• Complement
• Extend
• Intersect
• Subtract

Autodetermine file format

Right now the default is to read everything in as BED3 unless providing an alternative format to the -T flag.

Ideally this should autodetermine the format to be the number of columns in the input and fall-back to BED3 if it fails to do so.

There should also be two flags for left format and right format in case this is necessary.

try using a memory mapped fasta instead of opening a bunch of files

implement closest functionality

BED12 Support

build up and mdbook and start working on documentation

incorporate a subtract method

Use Translator in Read / Write methods

Implement flank command

Remove all Container trait and use IntervalContainers

build naive coverage subcommand

Try a naive implementation of the coverage subcommand

Support for intersect -wo flag

Write the original A and B entries plus the number of base
		pairs of overlap between the two features.
		- Overlaps restricted by -f and -r.
		  Only A features with overlap are reported.

implement sampling feature

Fix extend issue when using named and genome at the same time

Add support for compressed files

Hi @noamteyssier,

This seems like a great potential tool, but my bed files tend to be very large, so it would be awesome if you would considered allowing for compressed input (maybe through https://github.com/luizirber/niffler) and multithreaded compression of the output (maybe through https://github.com/sstadick/gzp).

Thanks for considering!
Mitchell

Complete support for BED6 File Format

Prepare cargo.toml for publish

implement with functionality on intersection

• with A
• with B

VCF support

Are there plans to also support VCFs?

GIA 0.2.0

Matching development of bedrs-0.2

• Convert all instances of Containers into static structs of IntervalContainer
• Convert all numeric instances of Bed3, Bed4, Bed6, Bed12 into bedrs structs
• Handle mixed file formats and combinatorics with dispatch methods

Prepare readme for release

Reproduce bedtools methods

• intersect
• window
• closest
• coverage
• map
• genomecov
• merge
• cluster
• complement
• shift
• subtract
• slop
• flank
• sort
• random
• shuffle
• sample
• spacing
• unionbedg

bed6 sort has incorrect index into retranslater

implement a random method

• number of intervals
• interval size
• number of chr

reorder commands alphabetically

add bed4 as an input format

set up get-fasta to use a stream instead of loading all intervals into memory

Implement shift subcommand

set up fractional intersections

Add a catch-all meta interval for formats that can't be automatically determined

For Bed-like files with a variable number of fields and no fixed structure

create windowed overlaps

incorporate streamed iterator to keep memory usage constant

bed12 support, -wo flag support

Thanks so much for providing this tool to the community! I am finding it much faster than the existing toolkits.
Would it be possible to provide support for the bed12 format? Additionally, would it be possible to extend functionality to additional bedtools intersect flags such as -wo?
For people working in the single-molecule sequencing space these additions would be massively helpful.

Thanks again!

Incorporate natural language sorting in sort

Not having lex sorting can lead to a bug in streaming methods

named genome file support

use faiquery as fai query backend