queer/squish

An experimental, rootless, Alpine-based container runtime

squish

squish is a novel rootless-only container runtime. The name is never capitalised, so if it were to come at the start of a sentence, it would still be written squish.

Why?

squish was born out of frustration with existing container runtimes. They all have their drawbacks (rootful, lack of port rebinds, cache-unfriendliness, slow container starts, ...) that lead to usage thereof being a frustrating experience at best. squish attempts to address these shortcomings.

Preliminary testing shows that squish can get a viable Alpine-rootfs container up in ~5ms. This is an initial figure, and will change over time.

Things that haven't been implemented but are planned

squish also avoids the typical OCI-style container images. The goal of squish is that the only "image" you deploy is a binary, and a manifest with the list of SDKs it uses. At container runtime, the various SDKs are bind-mounted into the container dynamically. Both the rootfs and all SDK layers are mounted read-only.

Roadmap

Feature	Description	State
Alpine	Read-only Alpine rootfs	✔️
Networking	slirp4netns networking + port binds	✔️
Mounts	Bind-mount files and directories ro and rw	✔️
Rootless	Containers without root	✔️
Container networking	Inter-container networking	TODO
Cgroups	Resource limitations etc	TODO
Systemd cgroup driver	Set up cgroups via systemd	TODO
Layer downloads	Download layers via HTTP	TODO
Seccomp	Syscall filtering	TODO
Dynamic port rebinds	(Re)bind container ports at runtime	TODO
/proc & /sys driver	FUSE vfs driver to safely mock /proc & /sys	TODO

What won't be implemented?

• Persistence of containers between daemon reboots
• Getting a shell in a container

Local development

1. Set up your environment by running ./setup.sh
2. Run the daemon with env RUST_BACKTRACE=1 RUST_LOG=debug cargo run -p daemon
3. Create a container with cargo run -p cli -- create test/squishfiles/default.toml
4. You did it! Read the cli source to learn more commands

Testing

squish currently only has e2e tests. You can run them by running ./test/test-e2e.sh.

Where did the name come from?

The idea started out as making something like Flatpak for servers -- although squish has significantly diverged since then -- and so the original working name was "squishpak," which eventually shortened into "squish."

Misc

http-asm: https://github.com/poletaevvlad/http-asm

squish's Issues

bikeshed: Layers

Layers are an integral part of squish, obviously. This bikeshed is specifically around these things are gonna work and what they look like. @oxy is dealing with alpine-ostree still, and that'll shape design here.

feature(daemon,pid1): Consider replacing slirp4netns with pasta(1)

https://passt.top/passt/about/#pasta-pack-a-subtle-tap-abstraction

feature(daemon,pid1): seccomp

https://man7.org/linux/man-pages/man2/seccomp.2.html

bikeshed: Registry

Related to #2.

While banyan is the "layers" "server" (alpine-ostree memes), we still need a consistent centralisable registry for storing binaries + layer manifests. Layer manifests are a separate concept from what's currently called a squishfile; squishfiles are a mixture of both layer manifests and information about how to run a container, currently, and ideally this should be split up.

Current thoughts:

• Split squishfile.toml into squish-app.toml and squish-manifest.toml
• Make a registry server (basically an S3 bucket?) that stores binaries + the corresponding squish-manifest.toml
• "Image tags" are immutable, and are basically just the location the "image" is uploaded to ($USER_OR_ORG/$NAME/$VERSION or smth like that)
• When booting an "image," pull its squish-manifest.toml, parse the layers, and pull all of them
• Cache layers locally to prevent re-pulls
• Treat binaries + layer manifests + layers as immutable, ie disallow overwriting an existing image (exceptions: ex. yanking from a (public?) repo?)

Challenges:

• Where the fuck do I get enough money to run a public/"primary" squish repository lmao

feature(cli,daemon): Support runtime port rebinds

This is an annoyance with Docker and would be awesome to have in squish.

feature(pid1): cgroup support

This is a bit hairy as cgroups require root for initial configuration -- definitely something that has to be set up as a part of package installation. What's the best way to do this?

bikeshed: Terminology and UX around "pods"

Because squish is using slirp4netns to handle networking, it's not as simple as "just start two containers and have them magically network together." Instead, we have to do something more akin to how podman handles things. This leads to bikeshedding about what the h*ck this is gonna be called and how it'll be exposed to the end user (CLI, API, etc.).

feature(daemon): Use alpine mirrors for rootfs downloads

https://cz.alpinelinux.org/alpine/MIRRORS.txt

bikeshed: Squishfiles

Formats to consider:

• JSON (lol), probably as HJSON
• TOML
• Dockerfile-like?
• Dhall

Formats to not consider:

• YAML

feature(daemon): Ensure port binds are valid / unbound before starting container

This may require taking out a global lock on container starts so as to prevent a race condition.

feature(daemon): Pass squishfile via memfd instead of tmpfile

• Create memfd
• Write to it
• Seek to zero
• nix::fnctl::fnctl to turn off FD_CLOEXEC
• Pass fd number as arg to pid1

feature: Mount /tmp rw

feature(squishfile,pid1): Allow user-specified bind mounts to be rw

See #23.

This would allow for ex. user-specified directory mounts for storing database files.

feature: Host directory/file bind mounts

feature(cli): Support passing squishfile components as cli args

feature: Dynamically cache Alpine rootfs based on version in squishfile

feature: Starting and stopping containers + persistent containers

Creating a container should just make the on-disk information about it, storing as much as possible? in a JSON? file in the container's directory. This is information like

• Container ID
• Command to be run
• Bind mounts
• Port bindings

The daemon should then be able to use this information to bootstrap a container process.