alloy-rs/rlp

Fast implementation of Ethereum RLP serialization

rlp's Issues

New and improved library and macro API

Library API

pub trait RlpEncodable: RlpLength {
    fn rlp_encode<T: BufMut>(&self, encoder: &mut Encoder<T>);

    fn rlp_encode_raw<T: BufMut>(&self, encoder: &mut Encoder<T>);

    fn rlp_len(&self) -> usize {
        let raw_len = self.rlp_len_raw();
        length_of_length(raw_len) + raw_len
    }

    fn rlp_len_raw(&self) -> usize; // Important: no default!!
}

// Might not be needed.
// pub trait DynRlpEncodable: RlpLength { /* ... */ }
// pub trait RlpLength { /* rlp_len, rlp_len_raw */ }
// impl<T> DynRlpEncodable for T where T: RlpEncodable { /* ... */ }

pub trait RlpDecodable<'de> {
     fn rlp_decode(decoder: &mut Decoder<'de>) -> Result<Self>;

     fn rlp_decode_raw(decoder: &mut Decoder<'de>) -> Result<Self> {
          Self::rlp_decode(decoder)
     }
}

pub struct Encoder<T: BufMut> {
    out: T,
}

// `struct Rlp` is replaced with `Decoder`.
pub struct Decoder<'de> {
    buf: &'de [u8],
}

impl<'de> Decoder<'de> {
    // Methods moved from `Header` ...

    // Methods for creating errors that contain source information like byte position
}

pub struct Error {
    bytepos: usize,
    kind: ErrorKind,
}

pub enum ErrorKind {
    // Previous `enum Error` ...
}

Other, not pictured:

• Add trait implementations for tuples, with the same behavior as structs with derived impls
  • This with the improved macro attributes should remove the need of dyn Encodable, enabling use of a generic Encoder.

Improvements:

• trait and method names prefixed with rlp to avoid conflicts on common types
• separate methods for "raw" encoding/decoding, meaning without header, to allow #[rlp(flatten)], see below

Derive macros

New attributes:

• (field) #[rlp(flatten)]: encode and decode using raw
• (container) #[rlp(tagged)]: impls for enums
  • tagged: defaults to the variant's discriminant value, or the variant can have an #[rlp(tag = <expr>)] attribute.
    • Example: P2PMessage
  • not specifying this fails, to leave room for future enum representations
• (container) #[rlp(transparent)]: replaces the separate *Wrapper macros
  • also account for #[rlp(skip)] in field number calculation

[Bug] Encodable::length returns incorrect values on 32-bit targets

Component

rlp

What version of Alloy are you on?

alloy-rlp v0.1.0 (higher version are also affected)

Operating System

None

Describe the bug

When deriving Encodable::length, the underlying function alloy_rlp::length_of_length uses the hard-coded value of 8 for the number of bytes in a usize. However, on a 32-bit target, this is incorrect. In certain cases, this results in invalid encodings on 32-bit targets.

Using the correct number of bytes by changing the affected line to the following fixes this problem:

1 + (usize::BITS as usize / 8) - payload_length.leading_zeros() as usize / 8

Of course, if it would be helpful in any way, I can also create a PR for this tiny fix.

Porting from ethers_core RlpStream

Hey, was looking to port some ethers_core RlpStream code to use Encode.

The struct uses some ethers_core primitives so I was looking for some guidance on how I could write a custom impl or modify the derive to work. A basic example of the struct im using.

pub struct ExecutionBlockHeader {
    // remaining items
    pub beneficiary: Address, // ethers Address which is more or less the same as Address from alloy_primitives
    // remaining items
}

The existing code uses an RlpStream from ethers_core. I am fairly close to getting the custom impl with something like

// encode individual elements
// get length of encoded buffer
// prepend Header

but the output is slightly off. I was trying to look through the derive macros but I'm fairly new to rust and thought I'd just ask as its something I could contribute to this project by writing a how to although I know you are recommending to just use the macros.

Was just asking if there are any extra steps that I might be missing which might be causing the output to differ.

It is feasible to change the ethers::Address to an alloy_primitive but this is a contribution to lighthouse and would require a lot more changes out of scope.

edit: Realised I can just do a new struct and impl From for NewStrcut and derive on the NewStruct

Add #[rlp(flatten)] by adding encode_fields and payload_length to Encodable

`RlpEncodableWrapper` for structs with more than one field

Currently, RlpEncodableWrapper/RlpDecodableWrapper can only be derived for single-field structs.
This contradicts the documentation, which explicitly mentions more than one field:

Derives Encodable for the type which encodes the fields as-is, without a header: <fields...>

There are several use cases where "headerless" encoding would be very helpful. So it should be considered if the documented behavior should be added. Otherwise, at least the documentation should be corrected to match the implementation.

An example of such a use case would be to implement something like #[serde(flatten)] by manually calling the headerless encoding of the structs to be flattened. This could, e.g., be useful for simplifying EIP-2718 transaction encoding.

`#[rlp(..)]` does not allow multiple attributes

If you want to derive both RlpDecodable and RlpEncodable for a struct that has a field that needs to be omitted, it requires both the #[rlp(default)] (for decoding) and #[rlp(skip) (for encoding) attributes.

However, if you combine these two values into a single field annotation, the procedural macro is no longer able to detect the attributes correctly and raises an error. For example:

struct Foo {
  pub bar: u64,
  #[rlp(default, skip)]
  pub cache: OnceLock<u64>,
}

raises the compiler error the trait bound std::sync::OnceLock: access_list::_::alloy_rlp::Decodable is not satisfied.

Instead writing:

struct Foo {
  pub bar: u64,
  #[rlp(default)]
  #[rlp(skip)]
  pub cache: OnceLock<u64>,
}

does compile successfully.