kuba2k2/datastruct

Combination of struct and dataclasses for easy parsing of binary formats

datastruct's Issues

Where to start

If you want to start using this project, there is very little documentation to this powerful tool.

Here are few projects where to look for "usages":

https://github.com/tuya-cloudcutter/cloudcutter-universal/blob/master/cloudcutter/modules/dhcp/structs.py
https://github.com/tuya-cloudcutter/bk7231tools/blob/main/bk7231tools/analysis/kvstorage.py
https://github.com/libretiny-eu/ltchiptool/blob/master/uf2tool/models/partition.py

How can I use substructures?

How can I use WString as substructure for CommandReadVariable? And is it possible to get length of WString inside CommandReadVariable?

def wstring(length: int, *, default: str = ...):
    return adapter(
        encode=lambda value, ctx: value.encode("utf-16le"),
        decode=lambda value, ctx: value.decode("utf-16le"),
    )(field(length, default=default))

@dataclass
class WString(DataStruct):
    variable_length: int = built("H", lambda ctx: len(ctx.variable_name))
    variable_name: str = wstring(lambda ctx: ctx.variable_length * 2)

@dataclass
class CommandReadVariable(DataStruct):
  tag_id: int = field('H', default= 1)
  message_length: int = built("H", lambda ctx: len(ctx.variable_name) * 2 + 3)
  message_type: MessageType = const(MessageType.CommandReadVariable)(field("B"))
  variable_name: WString

How to use adapters?

How to use adapters? For example, how to use ipv4_field adapter?

Alternative way of variable length bytes field.

Hi,

This is not issue, this is more of "documenting" of features for future users.

I needed something that is like "vartext" but with bytes. The example shown in README.md didn't work well.

So i implemented varbytes field.

def varbytes(
    length: Value[int],
    *,
    default: bytes = ...
):
    class Bytes(Adapter):
        def encode(self, value: bytes, ctx: Context) -> bytes:
            return value

        def decode(self, value: bytes, ctx: Context) -> bytes:
            return value

    return adapter(Bytes())(
        field(
            lambda ctx: (
                len(ctx.P.self) if ctx.G.packing else evaluate(ctx, length)
            ),
            default=default,
        )
    )

This is strongly inspired by vartext, but simplified as it only supports variable length. (Yes encode and decode could be lambdas, meh)

@dataclass
@datastruct(endianness=Endianness.LITTLE, padding_pattern=b"\x00")
class Packet1(DataStruct):
    list: int = field('B')
    num: int = field('H')
    count: int = field('H')
    data: bytes = varbytes(lambda ctx: ctx.G.root.length-5)

@dataclass
@datastruct(endianness=Endianness.LITTLE, padding_pattern=b"\x00")
class Packet(DataStruct):
    length: int = built("B", lambda ctx: ctx.body.sizeof())
    body: Any = switch(lambda ctx: ctx.typ)(
            PACKET1 = (Packet1, subfield()),
        )

When packing Packet.length is automatically calculated from length of data in Packet1 (or any other body packet)
When unpacking Packet1.data length is calculated from Packet.length - 5 (which is length of B+H+H))