Allow passing in header0 to stream readers?

[mhvk]

Allow passing in header0 to stream readers?

[theXYZT]

I benchmarked 4 use cases where I use baseband.open as a context manager and do something inexpensive like get fh.shape:

import baseband


# Opening a single DADA file (no format given)
def dada_no_format():
    with baseband.open('sample.dada') as fh:
        z = fh.shape
    return z


# Opening a single DADA file (format given)
def dada_with_format():
    with baseband.open('sample.dada', format='dada') as fh:
        z = fh.shape
    return z


# Opening sequential GUPPI files (to test SequentialFileReader)
def guppi_multifile_no_format():
    fs = ['fake.0.raw', 'fake.1.raw', 'fake.2.raw', 'fake.3.raw']
    with baseband.open(fs) as fh:
        z = fh.shape
    return z


# Opening sequential GUPPI files (to test SequentialFileReader) with format given
def guppi_multifile_with_format():
    fs = ['fake.0.raw', 'fake.1.raw', 'fake.2.raw', 'fake.3.raw']
    with baseband.open(fs, format='guppi') as fh:
        z = fh.shape
    return z

The results for these are:

In [2]: %timeit dada_no_format()
11.4 ms ± 132 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [3]: %timeit dada_with_format()
3.28 ms ± 328 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [4]: %timeit guppi_multifile_no_format()
27.8 ms ± 2.78 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [5]: %timeit guppi_multifile_with_format()
7.63 ms ± 592 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Giving a format leads to a significant speed up. Beyond that, you would have to create a reader over 3000 times to make it worth me typing this sentence.

I'd say readers are quick enough for now until someone comes up for a compelling use case for why they need to open a new reader object every 10 milliseconds.

---

PS: Perhaps there's a particular file format that's slower to open than GUPPI/DADA? Also, all files I used are small test files - I don't think the size of the files should have any bearing on the efficiency of creating a reader instance (but I could be wrong).

[mhvk]

@theXYZT - OK, I think this is a bit of a non-issue, then (I think VDIF would be slower as it needs to read several headers, and Mark 4 as it needs to find the start, but unlikely to be so much slower that it actually matters).

p.s. It is expected that it is faster if you given format, since then it doesn't have to try all formats to see which one works. There is obvious room for improvement in how it selects formats that might work...

[mhvk]

Conclusion: not really important, so closing for now.