Marshalling of optional empty byte is different from proto.Marshal

[planetscale]

Marshalling of optional empty byte is different from proto.Marshal

[vasanth-kandasamy]

@vmg / @jasonlong Could you please check the above issue.

[viswajithiii]

Can confirm that I am seeing this as well.

[mattrobenolt]

To confirm, is this from encoding a message like, OptionalByte{Value: nil} or OptionalByte{Value: []byte{}} ?

I would suspect these encode differently since optional fields are ignored from the proto encoding when the value doesn't exist.

I can check into this later, but that's what stands out to me without any other investigation.

[mattrobenolt]

Ah, ok, so I see the difference and can confirm slightly different behavior.

package main

import (
	"vtprotobuftest/test"
	"fmt"
	"google.golang.org/protobuf/proto"
)

func main() {
	a := test.OptionalByte{
		Value: nil,
	}
	b := test.OptionalByte{
		Value: []byte{},
	}
	fmt.Println("proto.Marshal:")
	fmt.Printf("%#v\n", must(proto.Marshal(&a)))
	fmt.Printf("%#v\n", must(proto.Marshal(&b)))

	fmt.Println()
	fmt.Println("vtproto:")
	fmt.Printf("%#v\n", must((&a).MarshalVT()))
	fmt.Printf("%#v\n", must((&b).MarshalVT()))
}

func must[T any](v T, err error) T {
	if err != nil {
		panic(err)
	}
	return v
}

Is my test script, just using the proto from the OP.

Output is:

proto.Marshal:
[]byte{}
[]byte{0xa, 0x0}

vtproto:
[]byte{}
[]byte{}

So it's the empty byte array case, not a nil byte array. Lemme see if I can dig in and see where we differ.

[mattrobenolt]

So this also seems to be explicitly for when optional is used, when it's just bytes value = 1;, it does marshal to no bytes. So I'm guessing the standard encoder need to explicitly say "this is missing" vs "there are no bytes".

[viswajithiii]

Yeah exactly. That's the purpose of optional.