BrotliEncoderCompressStream/BrotliDecompressStream should use &mut [mem::MaybeUninit<u8>] for output

[dropbox]

BrotliEncoderCompressStream/BrotliDecompressStream should use &mut [mem::MaybeUninit] for output

from rust-brotli.

[danielrh]

Hmm how would you recommend factoring this in without making the interface difficult to use (like happened with socket2, where ordinary "safe" users would have to invoke the "unsafe" tags just to use the lib) and without forcing users to change their upstream types?

would we need a separate, parallel code path that would have the uninit type everywhere?

[DoumanAsh]

@danielrh Not really, essentially both are the same under hood (pointer + size)

So you could have 1 function that takes pointer + size

E.g. how I write similar functions: https://github.com/DoumanAsh/compu/blob/master/src/encoder/mod.rs#L178-L198

You do not necessary need to expose function that takes raw pointers, but it is most convenient wrapper to get things done.

Alternatively, especially because most of the code works with slices, you can also make MaybeUninit main path and use it:

#[inline(always)]
BrotliEncoderCompressStream(
    s: &mut BrotliEncoderStateStruct<Alloc>,
    op: BrotliEncoderOperation,
    available_in: &mut usize,
    next_in_array: &[u8],
    next_in_offset: &mut usize,
    available_out: &mut usize,
    next_out_array: &mut [u8],
    next_out_offset: &mut usize,
    total_out: &mut Option<usize>,
) -> i32 {
    //Way one
    let next_out_array = unsafe {
        core::mem::transmute(next_our_array)
    }
    //Way two
    let next_out_array = unsafe {
        core::slice::from_raw_parts_mut(next_out_array.as_ptr_mut() as *mut mem::MaybeUninit<u8>, next_out_array.len())
    };
    //We just call Uninit variant
    BrotliEncoderCompressStreamUninit(s, op, available_in, next_in_array, next_in_offset, available_out, next_out_array, next_out_offset, total_out)
}

fn BrotliEncoderCompressStreamUninit(
    s: &mut BrotliEncoderStateStruct<Alloc>,
    op: BrotliEncoderOperation,
    available_in: &mut usize,
    next_in_array: &[u8],
    next_in_offset: &mut usize,
    available_out: &mut usize,
    next_out_array: &mut [mem::MaybeUninit<u8>],
    next_out_offset: &mut usize,
    total_out: &mut Option<usize>,
) -> i32 {
   //todo
}

[danielrh]

One of the goals of this library is to have no unsafe functions. Or at least the default features would eschew unsafe. I guess we would need a feature or parallel lib for this interface.

[DoumanAsh]

This is unfortunate consequence of uninitialized memory and Rust being unfriendly to it
Still, if unsafe is prohibited then it cannot be helped

But do note that it is actually necessary if you want to maintain correct C interface
Current Rust API basically forces you to assume C's API input is initialized:
E.g. here
https://github.com/dropbox/rust-brotli/blob/master/src/ffi/compressor.rs#L301

Although in practice this UB is harmless and well defined as you only write into resulting slice