mewz-project/mewz

A unikernel designed specifically for running Wasm applications and compatible with WASI

Mewz

Mewz is a unikernel designed specifically for running Wasm applications and compatible with WASI.

What's new with Mewz

There are now various Wasm runtimes, but they operate on general-purpose operating systems such as Linux or Windows.

Mewz is a specialized kernel designed for running Wasm. Mewz runs a single Wasm application within the kernel by linking it together during the build process with the Wasm application. (A kernel composed in this manner is commonly referred to as a unikernel.) In this way, Mewz provides the minimal required features and environment for executing Wasm.

Quick Start

There are some example programs. Please check out examples.

Option1: Docker

We prepare for a Docker image that has a environment for running Mewz.

curl -o helloworld.wat https://raw.githubusercontent.com/Mewz-project/Wasker/main/helloworld.wat
docker run -v .:/volume ghcr.io/mewz-project/mewz helloworld.wat

This image internally run Wasker, build Mewz, and run it on QEMU.

Option2: Dev Container

You can use Dev Container on GitHub Codespaces or your local VSCode.

To start Codespaces,

• Click Code -> Codespaces -> New codespace on this repository page.
• Wait for a while, then you can see VSCode on browser.
• Open terminal on VSCode

# On the Dev Container
git submodule update --init
curl -o helloworld.wat https://raw.githubusercontent.com/Mewz-project/Wasker/main/helloworld.wat
wasker helloworld.wat
zig build -Dapp-obj=wasm.o run

Option3: Build from source

Compile a Wasm file into a native object file, using Wasker. Follow the instruction here.

Then, build Mewz and run it on QEMU with the following commands.

zig build -Dapp-obj=<path to the object file generated by Wasker> run

To use file systems, specify the directory by -Ddir=<path to dir>. See examples/static_file_server.

Warning

This option makes an archive of the directory by tar and attach it to QEMU.

Note

QEMU's port 1234 is mapped to localhost:1234.

Run integration tests

zig bulid -Dtest=true run

Current Status

Feature	Status
WASI Preview 1	In Progress: Partial Implementation (Please refer to #1)
Socket	✅ (WasmEdge Compatible)
Component Model	Not yet
File System	On memory, read only
Network	✅

mewz's Issues

Replace `Waiter` with `hlt`

Now we use Waiter (https://github.com/mewz-project/mewz/blob/main/src/sync.zig#L36) to wait for interruption and data.

But we could replace it with simply hlt, because it stops until an interruption occurs.

To do so, we would avoid exclusion control and save CPU time of host.

Add options to specify the maximum number of TCP connections

The maximum number of TCP connections are now hard-coded.

We may need to enable to specify it by something like options.

To change the number, we need to tune these variables: #6

But we're not familiar with lwIP's parameters in lwipopts.h. We've arbitrarily determined them. The settings of Unikraft may be helpful as a reference.

UDP support

Mewz now only supports TCP sockets. We may need UDP sockets with DNS and so on.

Just like TCP sockets, we'll use lwIP's raw API.

https://www.nongnu.org/lwip/2_1_x/group__udp__raw.html

RingBuffer: handle cases when the read/write buffer is larger than the internal buffer

https://github.com/Mewz-project/Mewz/blob/main/src/util.zig#L34

If data.len > self.buffer.len, it causes panic.

Handle file descriptors as `usize`

File descriptors are handled as i32 in WASI functions arguments, like this.

pub export fn fd_close(fd: i32) callconv(.C) WasiError

We pass them around as i32. But indices must be usize in Zig, so we must cast them every time we index the file descriptor table.

    pub fn get(self: *Self, fd: i32) ?*Stream {
        const streams = self.streams.acquire();
        defer self.streams.release();
        const s = &streams.*[@as(usize, @intCast(fd))];
        if (s.* == null) {
            return null;
        }

        return @as(*Stream, @ptrCast(s));
    }

To avoid it, we want to cast fd to usize in WASI functions, and handle it as usize in stream module.

lwIP error handling

We're roughly handling the errors returned by lwIP functions, like this.

https://github.com/Mewz-project/Mewz/blob/main/src/tcpip.zig#L67

We need to respond appropriately depending on the type of error. The error types are defined here.

Support all of WASI preview1 functions

The support status for WASI functions is as follows:

• args_get
• args_sizes_get
• environ_get (but no environment variables support)
• environ_sizes_get
• clock_res_get
• clock_time_get
• fd_advise
• fd_allocate
• fd_close
• fd_datasync
• fd_fdstat_get
• fd_fdstat_set_flags
• fd_fdstat_set_rights
• fd_filestat_get
• fd_filestat_set_size
• fd_filestat_set_times
• fd_pread
• fd_prestat_get
• fd_prestat_dir_name
• fd_pwrite
• fd_read
• fd_readdir
• fd_renumber
• fd_seek
• fd_sync
• fd_tell
• fd_write
• path_create_directory
• path_filestat_get
• path_filestat_set_times
• path_link
• path_open
• path_readlink
• path_remove_directory
• path_rename
• path_symlink
• path_unlink_file
• poll_oneoff
• proc_exit
• proc_raise
• sched_yield
• random_get
• sock_accept
• sock_recv
• sock_send
• sock_shutdown

`ghcr.io/mewz-project/mewz` can't be pulled

$ docker pull ghcr.io/mewz-project/mewz
Using default tag: latest
latest: Pulling from mewz-project/mewz
af107e978371: Pulling fs layer 
4278792552db: Pulling fs layer 
9c01b7de78bf: Pulling fs layer 
3245434c345b: Waiting 
e438c664582f: Waiting 
1f89fd20aea9: Waiting 
3aff96a866aa: Waiting 
45bc183e102b: Waiting 
7a8408419b6b: Waiting 
7070c6497c75: Waiting 
error pulling image configuration: download failed after attempts=1: unknown blob

Some blobs such as sha256:96eb8313ddf564d2d66a262eaa10266ab06aa5933da279021aa3623ce5e740f3 (application/vnd.docker.container.image.v1+json) seems missing
https://explore.ggcr.dev/?image=ghcr.io%2Fmewz-project%2Fmewz%40sha256%3A8039c6107729564632b502735086f8d27357e4630e67d665068bb2663e2abab7

devcontainer fails with `zig build`

I am a zig newbie and would like to try to learn.

I'm having problems compiling mewz, I run zig build with the following error. Can you help me with this? Thanks for the help.

root ➜ /workspaces/mewz (main) $ zig build
++ git rev-parse --show-toplevel
++ git rev-parse --show-toplevel
+ REPO_ROOT=/workspaces/mewz
+ cd /workspaces/mewz
+ REPO_ROOT=/workspaces/mewz
+ cd /workspaces/mewz
+ ls build/newlib/libc.a
+ mkdir -p build/lwip
ls: cannot access 'build/newlib/libc.a': No such file or directory
+ mkdir -p build/newlib
+ cd build/lwip
+ cmake ../../lwip-wrapper
+ cd build/newlib
+ ../../submodules/newlib/newlib/configure --target=x86_64-elf --disable-multilib --prefix=/usr/local/newlib-x86_64-elf
./scripts/build-newlib.sh: line 12: ../../submodules/newlib/newlib/configure: No such file or directory
run ./scripts/build-newlib.sh: error: the following command exited with error code 127:
./scripts/build-newlib.sh 
CMake Warning (dev) in CMakeLists.txt:
  No project() command is present.  The top-level CMakeLists.txt file must
  contain a literal, direct call to the project() command.  Add a line of
  code such as

    project(ProjectName)

  near the top of the file, but after cmake_minimum_required().

  CMake is pretending there is a "project(Project)" command on the first
  line.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at CMakeLists.txt:15 (include):
  include could not find requested file:

    /workspaces/mewz/lwip-wrapper/../submodules/lwip/src/Filelists.cmake


CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.25)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!
See also "/workspaces/mewz/build/lwip/CMakeFiles/CMakeOutput.log".
run ./scripts/build-lwip.sh: error: the following command exited with error code 1:
./scripts/build-lwip.sh 
Build Summary: 1/6 steps succeeded; 2 failed (disable with --summary none)
install transitive failure
└─ install mewz.elf transitive failure
   └─ zig build-exe mewz.elf Debug x86_64-freestanding transitive failure
      ├─ run ./scripts/build-newlib.sh failure
      └─ run ./scripts/build-lwip.sh failure
error: the following build command failed with exit code 1:
/workspaces/mewz/zig-cache/o/7c36e40cbb2402eb80e289dd70bd7100/build /usr/bin/zig/zig /workspaces/mewz /workspaces/mewz/zig-cache /root/.cache/zig --seed 0xd3f16844

Add a test for reading a file

We need a test for reading a file.

• path_open to open a file
• fd_read to read the file content, and compare it with the expected content
• fd_close to close the file