Using cog3pio to determine byte ranges in COG files?

[TomNicholas]

Using cog3pio to determine byte ranges in COG files?

[weiji14]

Oh hi Tom, I was just chatting to @norlandrhagen at the Pangeo weekly meeting 😆. I will say that yes, it should be possible to work out the byte ranges from just the GeoTIFF's header, and we can expose that via an API function somehow. I just need to figure out how GDAL does this, and re-implement it here (easier said than done).

Note that I'm already using object-store in cog3pio (#5), and passing a HTTP url to a GeoTIFF should already work. Reading from s3 (or azure, gcp, etc) will work too if I enable the feature flag here and recompile:

cog3pio/Cargo.toml

Line 17 in 33285da

object_store = { version = "0.9.0", features = ["http"] }

I'm aware that the Zarr v3 implementation is using object-store, and keeping an eye on progress at https://github.com/roeap/object-store-python. Would definitely be keen to standardize on object-store as the 'fsspec-for-Rust'.

[TomNicholas]

Awesome! Thanks @weiji14

Would definitely be keen to standardize on object-store as the 'fsspec-for-Rust'.

Yeah this would be great, and I like the way you've described the aim there.

[martindurant]

I would add, that there's no reason that kerchunk needs to use fsspec - it's really a package of ways to make reference files. Therefore, if you come up with a way to get COG offsets, it can easily live there together with the other reference makers. Does TIFFFile do the required work too?