nickbabcock/bitter

Extract bits from a byte slice

bitter's Issues

Write support?

Interested in using this crate but it is a bit unergonomic to having to resort to a different crate to write the bit patterns and then switch to this to read them

Expose number of bits left in lookahead buffer

The number of bits to peek or consume is capped at the number of bits left in the lookahead buffer, but the number of bits left isn't exposed. The only workaround for the caller is to invoke refill_lookahead themselves (perhaps unnecessarily) and do manual bookkeeping which duplicates bitter's internal bookkeeping.

Looking at a solution probably like:

fn lookahead_bits(&self) -> u32 {
}

Unclear error when peeking zero bits

In working on some older code using your library, I accidentally called BigEndianReader::read_bits(0) and got this panic:

thread '...' panicked at 'attempt to shift right with overflow', .../bitter-0.6.0/src/lib.rs:793:9

It wasn't immediately obvious what caused this without reading the source and checking the docs. Perhaps *EndianReader::peek or read_bits could use another debug_assert! to make the error message clearer?

If you'd like, I can submit a PR.

Faster way to read unaligned bytes

Currently "unaligned" bytes are read with "read and refill", but I think we could do quite a bit better.

The idea is to take the buffer as is and bitshift the all bytes in one loop with SIMD.

The below produces nice auto-vectorized code that is >10x faster on bigger reads.

For example here we are "4 bits into a byte":

pub fn read_n_bytes(&mut self, n: usize, input: &[u8], out: &mut [u8]) {
    let left_mask = 0b00001111;
    let right_mask = 0b11110000;
    for i in 0..n {
        let left_part = input[i] & left_mask;
        let right_part = input[i + 1] & right_mask;
        out[i] = left_part | right_part;
    }
}

I gave it a shot myself but couldn't quite get the state correct after this operation.

Big endian support

Thank you for the crate! Supporting big-endian bitstreams would be a useful feature.

Release 1.0

I've not needed any new features or API changes in over a year (coming on two), if this continues for another 6 months, I will cut the current version (0.6.2) as (1.0).

There are nice to haves like the IO adapter, but since I don't personally need this feature I've not taken this feature to the finish line. I don't suspect it'd be a breaking change anyways.

Other tasks like exploring removing unsafe (#23) and transitioning away from basing everything on a large macro won't cause any API or behavior changes and can be done after 1.0

Shim 57 to 64 bit reads

Bitter's 56 bit read limitation can be annoying to workaround. Imagine a use case of a length prefixed value of up to 64 bits is read.

    let mut bits = LittleEndianReader::new(&[]);
    let len = bits.read_bits(6).unwrap() as u32;
    if len <= bitter::MAX_READ_BITS {
        bits.read_bits(len).unwrap();
    } else {
        let lo = bits.read_bits(bitter::MAX_READ_BITS).unwrap();
        let hi = bits.read_bits(len - bitter::MAX_READ_BITS).unwrap();
        let result = (hi << bitter::MAX_READ_BITS) + lo;
    }

Not only is it annoying, there could be faster ways to structure the reads (eg: consume the entire lookahead buffer, refill it, and continue reading). It makes sense for bitter to have this responsibility.

This is probably best suited for only the read_bits function, and peek and consume can keep their limitation (but probably update their assertions to give better error messages).

EDIT: Shim zero bit reads too 🤔