Comments (6)
Thank you very much @dicej! I really appreciate the detailed explanation and you taking the time to create the complete test case.
Now it is working. After all, it was not an issue with the code but with my system. For some reason I had a big amount of leaked memory that was producing the system running out of memory when compiling the component. After rebooting it works.
I will also take a look to the wasmtime compile
command and the instantiate_pre
, since looks like something I will need for what I am doing.
Again, thank you very much and continue the good work! I am really excited about using componentize-py
and the component model to run Pipeless hooks. For some context, we are using right now Pyo3 there to run Python hooks from Rust every time there is a new frame.
from componentize-py.
Hi, @miguelaeh. Good question. The Val
type you linked to is part of Wasmtime's API for working with core Wasm modules, whereas componentize-py
and WIT use the Component Model API: https://docs.rs/wasmtime/latest/wasmtime/component/index.html.
The equivalent call
function and Val
type in that API are https://docs.rs/wasmtime/latest/wasmtime/component/struct.Func.html#method.call and https://docs.rs/wasmtime/latest/wasmtime/component/enum.Val.html, respectively. However, you probably don't want to use either of those. Instead, you'll want to use https://docs.rs/wasmtime/latest/wasmtime/component/macro.bindgen.html, which will generate host bindings for your .wit
file, and those bindings will generally be easier to use and more efficient than using the dynamic call
/Val
API. See e.g. https://github.com/bytecodealliance/componentize-py/blob/main/src/test/tests.rs for an example of using that macro. We should probably add a simple example to the examples
directory, too, since the tests are pretty complicated.
Hope that helps. I'm happy to answer any follow-up questions here, and https://bytecodealliance.zulipchat.com/#narrow/stream/217126-wasmtime is also a great place to ask questions.
from componentize-py.
Thank you very much @dicej for the clarifications!
I will try with the path you mentioned and let you know.
Feel free to close this issue since is actually not an issue.
from componentize-py.
Hi @dicej ,
Following the docs you shared it was really simple to create the Rust program. It is able to recognize the types generated by bindgen
. However, it gets stuck forever creating the component from the WASM file generated by componentize-py
and the OS kills the process.
fn main() {
let mut config = Config::new();
config.wasm_component_model(true);
config.debug_info(true);
let engine = Engine::new(&config).unwrap();
println!("Creating component");
let component = Component::from_file(&engine, "./process.wasm").unwrap(); <- This never ends (see below)
println!("Component created");
...
After a long time, it gets killed by the OS:
➜ pipeless-wit git:(master) ✗ cargo run
Compiling pipeless-wit v0.1.0 (/home/miguelaeh/projects/wasm-components-experiments/pipeless-wit)
Finished dev [unoptimized + debuginfo] target(s) in 8.27s
Running `target/debug/pipeless-wit`
Creating component
[1] 428659 killed cargo run
For some context, this is my Python implementation:
import hook
from hook.imports.types import Frame, Context
class Hook(hook.Hook):
def hook(self, frame: Frame, ctx: Context) -> str:
return "Hello, World!"
and the WIT:
package pipeless:hooks;
interface types {
record frame {
uuid: string,
original: list<list<list<u8>>>,
modified: list<list<list<u8>>>,
width: u32,
height: u32,
pts: u64,
dts: u64,
duration: u64,
fps: u8,
input-timestamp: float64,
inference-input: list<list<list<u8>>>,
inference-output: list<list<list<u8>>>,
pipeline-id: string,
}
record context {
to-change: string,
}
}
world hook {
use types.{frame, context};
export hook: func(f: frame, c: context) -> string;
}
It is my first time with the component model so I may be missing something trivial here.
from componentize-py.
Hi @miguelaeh,
I just pushed a complete test case which works for me here: https://github.com/dicej/pipeless.
Note that it is important to build the host with the --release
flag, since the default debug build is much slower. Cranelift (which is the compiler Wasmtime uses to compile Wasm to native code) has a lot of work to do with components generated by componentize-py
since they include the entire CPython interpreter and native library code. Even with --release
compiling the component takes over 5 seconds on my Mac M2 Pro, and the debug build takes a lot longer.
For reference, you can pre-compile a .wasm file to a native .cwasm file using wasmtime compile
if performance is an issue. You can also use instantiate_pre
instead of instantiate
if you find you need to make a lot of instantiations of the same component.
from componentize-py.
One more thing: I noticed you mentioned that the OS is killing your process before it finishes compiling the component. That might mean it's running out of memory, which --release
might help with. If that doesn't help, you might need to use wasmtime compile
to generate a .cwasm on a machine with more memory.
from componentize-py.
Related Issues (20)
- Expand CI test coverage
- Add interface version support HOT 1
- Remove init function and types from the final output
- Use interface package as part of module name in generated bindings HOT 1
- Use `wasi-virt`-based VFS to store Python code
- Download dependencies automatically
- Add support for type aliases
- Optionally produce component which imports libpython311.so, libc.so, etc. instead of bundling them
- time.sleep sleeps indefinitely HOT 3
- re-init random seed(s) on resume
- pyo3_runtime.PanicException: internal error: entered unreachable code HOT 3
- Running component without WASI HOT 2
- Run MyPy on all test code and bindings HOT 1
- Running example HOT 3
- Matrix-math example - ImportError... HOT 4
- List errors which might be raised in function docstrings
- Make the types generated for WIT resources context managers
- feature: add zlib to cpython build HOT 1
- Generated component has dependency on wasi:cli/[email protected] HOT 14
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from componentize-py.