Explore direct, non-networked reads for faster I/O

[filodb]

Explore direct, non-networked reads for faster I/O

[rstrickland]

I see the rationale, but would caution that building out Cassandra-like functionality (including multi-DC replication) is non-trivial. Personally it seems to make more sense to use a proven storage layer than to try to reinvent that. What I could see would be a pluggable storage layer, but then you risk reducing to the lowest common denominator. I'll also add that I probably wouldn't use FiloDB if it meant having to run additional infrastructure outside of our existing Cassandra-Spark installation. The drop-in library aspect makes it more appealing and easier to adopt IMHO.

[velvia]

@rstrickland the ColumnStore API is pluggable - in fact there are two implementations, the C* one and an in-memory one.

@milliondreams so a non-C* columnStore, and reading directly from disk, and off-heap are all separate issues. How about we change this ticket to be for direct (i.e. no-network) reads from disk, to enable faster read performance? That's closer goal than a non-C* columnstore. We could, for example, directly parse SSTables - though I think we have a ways to go before this becomes a performance bottleneck. This allows us to reuse C* infrastructure but still read quickly.

[velvia]

Let's close this for now.