aristanetworks/bst

A one-stop shop for process isolation

bst

bst (pronounced "bestie") is a one-stop shop for running programs in isolated Linux environments. It is, effectively, a combination of unshare, mount, setarch, chroot, and many others; taking care of all the low-level minutæ to get in an environment that is as isolated as possible.

The main purpose of bst is running CI/build processes in a somewhat deterministic fashion.

Usage

$ bst [options] <exe> <args...>

See man 1 bst for more detailed information about how to use this program, including examples.

Why bst?

While bst is a multi-purpose tool, its main purpose is to serve as a building block for larger container systems. In CI systems running lots of commands in rapid succession, the cost of spinning up Docker containers can be unacceptable. For instance, on an 8-core laptop, over 10 runs, it takes 1.15 seconds to run /bin/true on an Alpine Linux Docker image, while bst takes 0.07 seconds to setup and run the same program in an isolated environment.

$ perf stat -n -r 10 -- docker run --rm -it alpine true

 Performance counter stats for 'docker run --rm -it alpine true' (10 runs):

            1,1503 +- 0,0156 seconds time elapsed  ( +-  1,36% )

$ perf stat -n -r 10 -- bst -r alpine /bin/true

 Performance counter stats for 'bst -r alpine /bin/true' (10 runs):

           0,07352 +- 0,00470 seconds time elapsed  ( +-  6,40% )

bst is not and does not want to be a replacement for Docker, but is meant to be used by tooling wanting low-overhead isolated environments.

Another strong suit of bst is that, by design, it can be used unprivileged. bst uses well-defined semantics for user namespaces to give unprivileged users the rights to enter different environments in a safe and controlled manner.

Quickstart

Installing

There are two ways to install bst: downloading a prepackaged binary, or building from source:

Installing a binary package

Download the binary archive of the latest release.

Extract the archive into /, making sure to preserve xattrs. bst is installed into /usr/local.

$ sudo tar --xattrs --xattrs-include='*' -xf bst-x86_64.tar.xz -C /
$ export PATH=$PATH:/usr/local/bin
$ bst --version
v1.0.0-rc1

Building from source

bst uses Meson for its build system (requires python, ninja, sudo, and libcap). Additionaly, it uses scdoc to build its man pages.

From the source directory:

$ meson ./build
$ ninja -C ./build
$ sudo ninja -C ./build install

The last step installs bst into /usr/local.

Using bst

First, make sure that your current user has a slice of sub-UIDs and sub-GIDs allocated:

$ id
uid=1000(barney) gid=1000(barney) groups=1000(barney)

$ grep -H . /etc/sub{u,g}id
/etc/subuid:barney:1000000:65536
/etc/subgid:barney:1000000:65536

See man 5 subuid and man 5 subgid for what these values signify.

Once this is done, you should just be able to try it out:

$ bst
# id
uid=0(root), gid=0(root), groups=0(root)

bst's Issues

Add missing rlimits

Currently, only three limits are implemented.

Provide a more turnkey approach for mounting sysfs

Either of these may be correct in a given situation, or may fail

Host sysfs mount === user ns sysfs mount:

bst --mount /sys,/sys,none,rbind true

Other:

bst --mount none,/sys,sysfs,default true

Maybe bst should just fallback to a rbind mount when it EPERMs, and add a --no-fallback-sysfs-mount

Add --unshare counterparts to --share

Normally, all namespaces are unshared by default, but the lack of --unshare options makes it hard to only unshare a select few while keeping the rest. This is mostly made visible when trying to re-enter a persisted spacetime whose init died -- in which case, you need to unshare a new PID namespace.

With this, we could call bst --share-all=<path> --unshare-pid. Right now, the alternative is to either unpersist the pid ns file, or manually pass all --share flags individually.

Extend the documentation with more examples

I am trying to build a system like HackerRank, i.e., the code submitted by the user should be executed on the server. From what I understand, bst can be used for this.

If I run bst, it opens my shell as mentioned in the documentation. If I delete files the changes are persisted outside my isolated environment. Can somebody please explain what I am missing?

I tried changing the root with --root, but this is what I get:

samuel@Titan ~/p/bst (main) [1]> bst --root ~/Desktop/test-root ls
bst-init: execvpe ls: No such file or directory

PS: I have no knowledge about namespaces.

unit test broken?

--- command ---
05:44:32 /home/anisinha/workspace/bst/test/cram.sh /home/anisinha/workspace/bst/test
--- stdout ---
diff --git a/home/anisinha/workspace/bst/test/net.t b/home/anisinha/workspace/bst/test/net.t.err
index ed61ab5..5338901 100755
--- a/home/anisinha/workspace/bst/test/net.t
+++ b/home/anisinha/workspace/bst/test/net.t.err
@@ -19,8 +19,8 @@ MACVLANs
 IPVLANs
 
 	$ bst --nic parent,dummy,address=fe:ed:de:ad:be:ef bst --nic ipvlan,type=ipvlan,link=parent -- ip link show ipvlan
-	2: ipvlan@if2: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
-	    link/ether fe:ed:de:ad:be:ef brd ff:ff:ff:ff:ff:ff link-netnsid 0
+	2: ipvlan: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
+	    link/ether fe:ed:de:ad:be:ef brd ff:ff:ff:ff:ff:ff
 
 Adding addresses
 
@@ -29,7 +29,7 @@ Adding addresses
 	    link/ether fe:ed:de:ad:be:ef brd ff:ff:ff:ff:ff:ff
 	    inet 172.20.0.1/32 brd 172.20.0.1 scope global dummy
 	       valid_lft forever preferred_lft forever
-	    inet6 fe80::fced:deff:fead:beef/64 scope link tentative 
+	    inet6 fe80::fced:deff:fead:beef/64 scope link 
 	       valid_lft forever preferred_lft forever
 
 	$ bst --nic dummy,type=dummy,address=fe:ed:de:ad:be:ef --ip 172.20.0.1/16,dev=dummy -- ip addr show dummy
@@ -37,7 +37,7 @@ Adding addresses
 	    link/ether fe:ed:de:ad:be:ef brd ff:ff:ff:ff:ff:ff
 	    inet 172.20.0.1/16 brd 172.20.255.255 scope global dummy
 	       valid_lft forever preferred_lft forever
-	    inet6 fe80::fced:deff:fead:beef/64 scope link tentative 
+	    inet6 fe80::fced:deff:fead:beef/64 scope link 
 	       valid_lft forever preferred_lft forever
 
 Adding routes
@@ -59,4 +59,4 @@ Adding routes
 	default via 172.20.0.1 dev lo 
 
 	$ bst --route dev=lo -- ip route show
-	default dev lo scope link 
+	default dev lo 
diff --git a/home/anisinha/workspace/bst/test/bst.t b/home/anisinha/workspace/bst/test/bst.t.err
index 230ba9d..d0e01cd 100755
--- a/home/anisinha/workspace/bst/test/bst.t
+++ b/home/anisinha/workspace/bst/test/bst.t.err
@@ -121,6 +121,8 @@ Testing hostname semantics
 Testing persistence
 
 	$ mkdir -p foo bar; trap 'bst-unpersist foo && rmdir foo bar' EXIT; bst --persist=foo sh -c 'mount -t tmpfs none bar && echo hello > bar/greeting' && [ ! -f bar/greeting ] && bst --share mnt,user=foo sh -c '[ "$(cat '"$PWD"'/bar/greeting)" = "hello" ]'
+	bst: bind-mount /proc/1173705/ns/mnt to foo/mnt: Invalid argument (is the destination on a private mount?)
+	[1]
 
 Testing --limit core / general tests
 	$ bst --limit core=0 test/print_limits core
-------

Change --limit-<limit>=n:m options to --limit <limit>[,<limit>...]=n:m

This makes the option more consistent with --share, --unshare, and --persist.