sunrise-choir/flumedb-rs

flumedb-rs's Issues

Stable?

Hi. Is the release on crates.io https://crates.io/crates/flumedb stable?

Reading on uninitialized buffer may cause UB ( `read_entry()` )

Hello 🦀,
we (Rust group @sslab-gatech) found a memory-safety/soundness issue in this crate while scanning Rust code on crates.io for potential vulnerabilities.

Issue Description

flumedb-rs/src/go_offset_log.rs

Lines 212 to 221 in d6dbb60

	fn read_entry<F>(frame: &Frame, read_at: &mut F) -> Result<ReadResult, Error>
	where
	F: FnMut(&mut [u8], u64) -> io::Result<usize>,
	{
	// Entry is [payload size: u64, payload ]
	let mut buf = Vec::with_capacity(frame.data_size);
	unsafe { buf.set_len(frame.data_size) };
	let n = read_at(&mut buf, frame.data_start())?;

flumedb-rs/src/offset_log.rs

Lines 330 to 341 in d6dbb60

	fn read_entry<ByteType, F>(frame: &Frame, read_at: &mut F) -> Result<ReadResult, Error>
	where
	F: FnMut(&mut [u8], u64) -> io::Result<usize>,
	{
	// Entry is [payload size: u32, payload, payload_size: u32, next_offset: ByteType]
	let tail_size = size_of_frame_tail::<ByteType>();
	let to_read = frame.data_size + tail_size;
	let mut buf = Vec::with_capacity(to_read);
	unsafe { buf.set_len(to_read) };
	let n = read_at(&mut buf, frame.data_start())?;

Methods go_offset_log::read_entry() & offset_log::read_entry() create an uninitialized buffer and passes it to user-provided F . This is unsound, because it allows safe Rust code to exhibit an undefined behavior (read from uninitialized memory).

This part from the Read trait documentation explains the issue:

It is your responsibility to make sure that buf is initialized before calling read. Calling read with an uninitialized buf (of the kind one obtains via MaybeUninit<T>) is not safe, and can lead to undefined behavior.

Suggested Fix

It is safe to zero-initialize the newly allocated u8 buffer before read_at(), in order to prevent user-provided code from accessing old contents of the newly allocated heap memory.

Also, there are two nightly features for handling such cases.

• https://doc.rust-lang.org/std/io/struct.Initializer.html
• https://rust-lang.github.io/rfcs/2930-read-buf.html

Thank you for checking out this issue 👍

Store the entire JSON message instead of just the content.

This will use slightly more disk space but means we don't need to go and hit the offset log for the entire message if needed.

To save some space in the future, we could store the message using cbor and transcode to json if needed.