Some WASI imported functions need to write to memory

[myobie]

Some WASI imported functions need to write to memory

[tessi]

Thanks for opening this well prepared issue 💚 I agree in that we should tackle it.

We should pass memory (and maybe even more context) to callback functions. This is already done for DynamicFuncs for us in Rust-land (we just chose to ignore the vm::Ctx at the current implementation).

vm::Ctx offers us the memory function which we could convert into a memory object and pass down to elixir.

We need to rework how memory resources function, though. Currently it remembers the instance and, for each task, asks it for its exports, finds the WASM-land memory, asks for the specified view (datatype and offset). Instead we need to save the memory object directly in the resource.

It should not be too hard to change the existing memory code. I would suggest to tackle this in three steps:

1. finish the import feature without passing memory in
2. (could be in parallel) rework the existing memory resource to hold a WASM memory instead of WASM instance
3. pass the memory into elixir callbacks for imported functions

About 3: I guess we have to change the method signature of elixir-callbacks here. Maybe they should receive a tuple of two elements:

• context - a map with %{memory: Wasmex.Memory.t()} and maybe later, WASM tables or other context-related infos)
• and the normal params array we pass in now

@myobie what do you think?

[tessi]

I will add a TODO to #9 to change the signature of elixir callbacks to not only receive params but {context, params} (where context will be the empty map for now).

This is hoping that people have an easier time migrating to the bright future of us passing memory and other context into callbacks.

[tessi]

☝️ that being said, I think you could get around the GenServer process attempting to call itself by not using the Wasmex GenServer interface, but implement your own - this way you would not need to call the GenServer just to retrieve the memory through a process call (you'd already have the Wasmex.Instance in your GenServers state and could create the memory directly from it).

It is a workaround I don't like (because I want the higher-level GenServer interface to work), but it may get you a step further in your WASI experiement 🤞

[myobie]

Your notes and suggested steps make sense and I agree finishing the import feature as is is the top priority.

I did start down the road of a custom GenServer this morning but realized it was too far.

[tessi]

@myobie could you give the current master another try? I added support for memory access to elixir callbacks in #16

[myobie]

@tessi it works for my use case for random_get. An example that works for me is:

{:ok, instance} =
  Wasmex.start_link(%{
    bytes: bytes,
    imports: %{
      wasi_snapshot_preview1: %{
        random_get:
          {:fn, [:i32, :i32], [:i32],
           fn %{memory: memory}, address, size ->
             bytes = :crypto.strong_rand_bytes(size)

             bytes
             |> :binary.bin_to_list()
             |> Enum.with_index()
             |> Enum.each(fn {byte, index} ->
               IO.puts("setting #{byte} at #{address + index}")
               Wasmex.Memory.set(memory, address + index, byte)
             end)

             0
           end}
      }
    }
  })

I can't use the function to write a binary because I don't want it to end with a null byte. I'm not sure if it's worth having two functions: one for write binary and one for write null terminated string.

[tessi]

Good point with the string writing. After some reading and getting more proficient in Rust it became clear to me that we shouldn't rely on null-bytes in our string helpers. We should always use the memory index and the strings length to read or write memory (as it is done by Rust in the String type).

Sooner or later I'll change this API to not use null-bytes, but lengths. Feel free to (but no need to) contribute :)

[tessi]

In other news: You may have a look at #17 (and a WIP PR at #18) that adds WASI support. I plan to optionally pre-fill the imports object with WASI implementations written in Rust. They should ideally be overwritable from elixir if you want to implement some functions yourself.

Maybe this is helping you in your plans.

[myobie]

IMO this issue can be closed now that #16 has been merged.