mzabaluev/chunked-bytes

A non-contiguous byte output buffer for Rust

chunked-bytes's Issues

Clarify need for strictly enforced chunk size

In the current implementation (as of 0.1.0-beta.3), ChunkedBytes is only meant to be a non-contiguous buffer to avoid linear reallocations when writing out protocol messages of varying sizes.
However, some applications may want the produced chunks to never exceed the configured size.

Some examples are:

• Datagram protocols that need to adhere to an MTU.
• Any non-vectored output with a discoverable buffer size that either accepts up to the buffer worth of data or reports busy; File probably fits here.

Disadvantages of capping the chunk size:

• Potentially somewhat worse memory utilization due to discarding any excess given by BytesMut::reserve and the underlying allocator.
• Need for extra bookkeeping to maintain our own cursor over the staging buffer and avoid possible undersized tail chunks resulting from allocation excess; alternatively, tokio-rs/bytes#396 is required.
• Need to split Bytes slices passed to push_chunk if they are larger than the chunk size.

Perf measurements

Hi. Thanks a lot. This is exactly what I'm looking for and I have a perfect use case to test the performance. I'll report here soon.