standardize on a body+slots structure for serialized reference-bearing object graphs

[warner]

standardize on a body+slots structure for serialized reference-bearing object graphs

[warner]

more notes:

We have a couple of different places where we pass a serialized capability-bearing object graph in a pair of values. The first value is a string, the result of calling JSON.stringify on the graph with a special replacer argument that does two things: replace non-JSON objects like undefined/NaN/-0/Infinity/symbols/bigints with special "@qclass" markers so they can be deseriaized into the equivalent objects on the other end, and replace capability slots with @qclass markers that have type: 'slot'. These type: 'slot' markers have an index: property that points into the second value, which is an array of references. Each reference has a type: like export, import, promise, or device, and a type-specific index: value.

The general idea is that the string part doesn't change at all, from the time some Vat uses syscall.send to start the message on its journey, through the kernel into the run-queue, off the run-queue into some other vat (perhaps the comms vat, in which case the message is sent to some other machine entirely and goes through the receiving comms vat and back into the far kernel and run-queue), and finally delivered to the deserialization code on the receiving side.

The slots part, however, is translated at each boundary. When leaving a vat, any local objects must be converted into exports. When entering the kernel, any imports are mapped through the vat-specific C-List table into exports from other vats (or kernel promises). When the message leaves the run-queue and is sent by the kernel into a vat, any exports from other vats must be added to the receiving vat's c-list and translated into imports (exports from the receiving vat are "coming home" and simply have the vatID removed). Kernel promises sent into a vat must be added to the c-list too. The comms vat must do similar mapping as the slots travel from one machine to another.

We aren't very consistent with how we name these two pieces right now. syscall.send() takes two separate arguments named argsString and vatSlots, but vatManager.js internally creates a data structure with properties named argsString and slots. syscall.fulfillToData takes fulfillData and vatSlots, while reject takes rejectData and vatSlots. syscall.doCallNow (used to invoke devices, which can return data synchronously) takes argsString and argsSlots, and returns an object with properties named data and slots. And the serialize() function in marshal.js returns an object with properties named argsString and slots because it was first used for serializing the arguments to syscall.send.

I think we should consolidate all of these cases into something consistent. I propose a single object (used in both arguments of a function and as the return value of serialize() and doCallNow()), with two properties: body (the string) and slots (the array of references).

[warner]

I've made some progress on this in the 28-body-slots branch, but it's not complete.

[warner]

In the rust port, I've been using a type named CapData to represent this combination of a body (the string of bytes, currently JSON encoded) and a slots list of object/promise references. This appears inside a Message as the args field (along with method, the message name, and result, which is an optional promise reference). It also shows up in two of the possible Resolution types (resolve-to-data and reject). Messages then appear in both syscall.send and dispatch.deliver, as well as the run-queue, and the queues inside each Promise (when pipelining is not in use). The Resolution types appear in syscall.resolve, dispatch.notify, and the kernel's promise table.

There's a (weak?) argument that the method name should be bundled closely with the arguments, such that a Message had two fields (XYZ and result), and "XYZ" would consist of method and args. The argument is that the fundamental operation is invoking an object, and the particular flavor of invocation (i.e. what method to invoke) is just as non-fundamental as the arguments to that invocation. I.e. the method name is just as much of an argument as everything else. This is how e.g. "contract" invocations in Ethereum's EVM work: there's only one entry point, which gets an array of bytes as its single argument, but there's a convention about how to encode the method name into that array, along with the other values we traditionally call "arguments".

But I'm not currently inclined to break things up that way, so I'm going to update the code to use a { body, slots } structure for "CapData", and { method, args, result } for "Message".

[warner]

in the old repo. this was SwingSet issue 28