wapc / wapc-go Goto Github PK
View Code? Open in Web Editor NEWGolang-based WebAssembly Host Runtime for waPC-compliant modules
License: Apache License 2.0
Golang-based WebAssembly Host Runtime for waPC-compliant modules
License: Apache License 2.0
Currently, the wazero engine takes over all the initialization for a module. It creates the runtime, compiles the module, creates the module config, etc. The issue is that I might want to customize something in that. For example, I want to provide my own stdout, stderr, stdin, and I want to set some env variables.
I propose that instead of creating the module within the library, you simply take a wazero runtime, as well as maybe a config struct containing all the required stuff for the exports, if needed, and export all the required functions, similar to wazero's own wasi_snapshot_preview1 package. This will allow users to do whatever they need to for their modules using wazero's API for that purpose instead of having to alter them to fit wapc's code.
If you want to keep the current interface, you can simply add new functions that do what I described and let users choose if they want a more out-of-the-box experience or if they want to customize things. This issue entirely prevents me from using this library, as much as I want to, because I need to customize some things in order to run the modules I need to run.
Also, using func(string)
for stdout is a horrible, horrible choice and will on its own discourage most Go developers from ever using this library. Stdout should always be an io.Writer
, as that is how Go represents data streams such as stdout.
Per the waPC specifications it looks like after the guest is invoked initially, the supported flow is the guest is allowed to make 0..N __host_call
before returning.
There's another flow, where communication is bi-directional, or where multiple __guest_call
s occur from a single invoke
:
__guest_call
This flow currently fails for the final __guest_call
in wasmer
(assumption is it also fails in wasmtime
), but works for wazero
. This is due to the use of invokeContext
. In wasmer
/ wasmtime
this is a pointer stored on the shared instance and the response copied to it (here), but the return value of invoke
is read from the original invokeContext
(here). With multiple __guest_call
in a single flow, the shared context is replaced. The output is copied to the invokeContext
of the previous __guest_call
and the final __guest_call
in the chain returns a zero value.
wazero
supports this flow since each __guest_call
via invoke
using wazero
gets its own context, not a shared context on the instance (here).
Whew, with that out of the way. Is the above bi-directional flow supported? If not, should it be supported and the implementation for wasmer
and wasmtime
changed to allow it?
Hi all!
I'm trying to run the example from https://github.com/wapc/wapc-go#example with a wasm module built in go using https://github.com/wapc/wapc-guest-tinygo#example
But I get
panic: Failed to instantiate the module:
link error: Import not found, namespace: wasi_snapshot_preview1, name: fd_write
Any hint will be super appreciated
Hi all, I've just started messing around with waPC in Go. I'm curious as to how people are using the host call function. Can it be seen as a kind of bi-directional/ call-response communication channel (like Go's chan
). Or is it more used for calling other wasm functions from the host. I'm interested to see all the use cases. Thanks
https://github.com/wapc/wapc-go/blob/master/module.go#L398
Sometimes we want to fetch the error inside the guest invocation, we cannot. GuestCall will only be error if the invoke command is not found or there is panic.
if err != nil {
return nil, errors.Wrap(err, "error invoking guest")
}
if context.guestErr != "" {
return nil, errors.WithStack(errors.New(context.guestErr))
}
I would like to use wazero's ModuleConfig
to provide a WASI FS and some other goodies for my esteemed guests.
https://pkg.go.dev/github.com/tetratelabs/wazero#ModuleConfig
Looking at the wapc interface this does not seem possible.
https://github.com/wapc/wapc-go/blob/v0.6.0/engines/wazero/wazero.go#L126-L127
Given that we already have Module.UnwrapRuntime
in the wazero engine (escaping the wapc.Module interface), do you think we could also get a Module.WithConfig
?
module.(*Module).WithConfig(func(config wazero.ModuleConfig) wazero.ModuleConfig {
return config.
WithFS(os.DirFS(`/tmp/candy/mountain`)).
WithRandSource(rand.New(rand.NewSource(2008))).
WithNanotime(func(context.Context) int64 { return clock.nanotime() }, 1)
})
Ability to mock time and rand for deterministic execution is imperative for my use case and probably a lot of others.
The current signatures (after documenting) highlight some ambiguity in terms of intent. For example, is it intentional to change all writers and loggers at runtime (if so, no one is implementing this goroutine safe)? Or is it just to allow changing these before each call to Instantiate?
// Module is a WebAssembly Module.
Module interface {
// SetLogger sets the waPC logger for `__console_log` function calls.
SetLogger(logger Logger)
// SetWriter sets the writer for WASI `fd_write` calls to stdout (file descriptor 1).
SetWriter(writer Logger)
// Instantiate creates a single instance of the module with its own memory.
Instantiate() (Instance, error)
// Close releases resources from this module, ignoring any errors.
// Note: This should be called before after calling Instance.Close on any instances of this module.
Close()
}
I'm guessing the latter, that this is just to ensure each instance can get a different logger or writer. If so, can we change the signature to make this clear? This will ensure the references used can be final and don't need to be tracked for runtime updates.
// Module is a WebAssembly Module.
Module interface {
// Instantiate creates a single instance of the module with its own memory.
// * consoleLogger is the waPC logger for `__console_log` function calls.
// * stdout is the writer for WASI `fd_write` calls to stdout (file descriptor 1).
Instantiate(consoleLogger Logger, stdout Logger) (Instance, error)
// Close releases resources from this module, ignoring any errors.
// Note: This should be called before after calling Instance.Close on any instances of this module.
Close()
}
While at it, io.Writer is probably the better choice for stdout, as it is the same as os.Stdout..
The problem stems from the fact that both wasm runtimes (wasmer and wasmtime) export some of the same symbols into the flat c namespace. For example, wasm_config_new is exported by both runtimes. So, when wapc links, which one does it get? I noticed this when trying to integrate wasmer's engine metering and being confounded by wasm_config_new not calling the wasmer export. I'm not sure how to work around this as there are numerous symbols to account for.
Hello! I came across this project after watching a talk from the Wasm Summit.
For context, I'm the lead developer of the Product Discovery team at 1Password.
I've been working on the Suborbital Development Platform for some time now, and one of its main goals is to create a viable server-side Wasm framework.
It would appear that I have inadvertently re-created some of the work you're doing here! The hivew
toolchain I've been building includes what I've been calling the "Runnable FFI API", and it appears to be doing a lot of similar things to waPC!
I'm curious - would there be interest in having a call some time to see how I could potentially stop re-inventing the wheel and perhaps migrate the Hive FFI API over to waPC? I would of course love to contribute to this project as well. I would hate to fragment any kind of standard in this young ecosystem.
Please let me know! My email is connor at suborbital dot dev
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.