Giter Site home page Giter Site logo

containers / bootc Goto Github PK

View Code? Open in Web Editor NEW
529.0 22.0 69.0 1.47 MB

Boot and upgrade via container images

Home Page: https://containers.github.io/bootc/

License: Apache License 2.0

Shell 4.98% Rust 89.79% Makefile 0.66% Dockerfile 0.58% Jinja 0.16% Lua 1.00% Nushell 2.60% Python 0.16% Nu 0.07%

bootc's Introduction

bootc

Transactional, in-place operating system updates using OCI/Docker container images.

Motivation

The original Docker container model of using "layers" to model applications has been extremely successful. This project aims to apply the same technique for bootable host systems - using standard OCI/Docker containers as a transport and delivery format for base operating system updates.

The container image includes a Linux kernel (in e.g. /usr/lib/modules), which is used to boot. At runtime on a target system, the base userspace is not itself running in a "container" by default. For example, assuming systemd is in use, systemd acts as pid1 as usual - there's no "outer" process. More about this in the docs; see below.

Status

NOTE: At the current time, bootc has not reached 1.0, and it is possible that some APIs and CLIs may change.

Documentation

See the project documentation; there are also operating systems and distributions using bootc; here are some examples:

Developing bootc

Are you interested in working on bootc? Great! See our CONTRIBUTING.md guide.

bootc's People

Contributors

bcrochet avatar cgwalters avatar chuanchang avatar ckyrouac avatar ericcurtin avatar henrywang avatar ipanova avatar jeckersb avatar jistr avatar jlebon avatar jmarrero avatar lmilbaum avatar lukewarmtemp avatar m8ram avatar miabbott avatar mrguitar avatar mvo5 avatar omertuc avatar ondrejbudai avatar pablomh avatar paulyuuu avatar platform-engineering-bot avatar prestist avatar renovate[bot] avatar rhatdan avatar runcom avatar sacsant avatar stefwalter avatar vrothberg avatar yoheiueda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bootc's Issues

install TODO

  • Implement ppc64le and s390x (needs code from both coreos-assembler and coreos-installer)
  • support installing from SELinux-disabled hosts (needs ostreedev/ostree#2804 )
  • Support overwriting disk we used to boot (systemctl switch-root to container image?)
  • Support basic partition configs (e.g. /var)
  • Related, investigate https://www.freedesktop.org/software/systemd/man/systemd-repart.html - maybe we just limit root size and have users embed repart.d configs?
  • Also support running Ignition from the "install boot"; this would make debugging Ignition - configs dramatically nicer
  • Ensure we lock devices when partitioning https://systemd.io/BLOCK_DEVICE_LOCKING/
  • Support config files (e.g. distro default filesystem specified in /usr/lib/bootc/install.cfg)
  • Support (or default) to only installing EFI if booted that way, etc.
  • Support systemd-boot
  • Clean up/refactor install code to be more distribution-independent in general
  • Consider merging bootupd in here as e.g. bootc bootloader update or so?
  • Documentation
  • More explicitly test host/target kernel compatibility (e.g. filesystems)

Failed to run skopeo --version

# podman run --privileged --pid=host --net=none --security-opt label=type:unconfined_t ghcr.io/cgwalters/c9s-oscore bootc install --target-no-signature-verification --disable-selinux /dev/sdb
notice: Target has SELinux enabled, overriding to disable
Mounting devtmpfs
Initializing partitions
Creating filesystem
mke2fs 1.46.5 (30-Dec-2021)
Creating filesystem
Mounting /run/bootc/mounts/rootfs
Mounting /run/bootc/mounts/rootfs/boot
Creating ESP filesystem
Mounting /run/bootc/mounts/rootfs/boot/efi
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree root
ERROR Creating ostree deployment: Querying skopeo version: Failed to run skopeo --version: ExitStatus(unix_wait_status(32512))

Error: Installing to devices ending in numbers fails

When I attempted to install on a loop device it fails before formatting:

Using device: /dev/loop0
Mounting selinuxfs
Mounting devtmpfs
Initializing partitions
Creating filesystem
mke2fs 1.46.5 (30-Dec-2021)
The file /run/bootc/mounts/dev/loop03 does not exist and no size was specified.

I think naming logic for devices does not account for the extra 'p' added to devices ending with a number:

let base_rootdev = format!("{device}{ROOTPN}");

ERROR Linking tmp mounts to host: Exchanging /tmp <=> /tmp.tmp: Invalid cross-device link (os error 18)

I have hit this error ERROR Linking tmp mounts to host: Exchanging /tmp <=> /tmp.tmp: Invalid cross-device link (os error 18) with example image in install.md file, i am currently using ubuntu 23.04 as OS from which i run podman command using bootc built from source in git repo
if i mount /tmp to /tmp.tmp with podman run flag -v error changes to ERROR Linking tmp mounts to host: Symlinking /proc/1/root//tmp to /tmp.tmp: File exists (os error 17) mounting /tmp to /tmp changes error to ERROR Linking tmp mounts to host: Exchanging /tmp <=> /tmp.tmp: Device or resource busy (os error 16)

There might be some commonality with issue #91.

podman version 4.3.1

command: sudo podman run --privileged --pid=host --net=none --security-opt label=type:unconfined_t ghcr.io/cgwalters/c9s-oscore bootc install --target-no-signature-verification /dev/vda

Add `install-to-filesystem`

This aims to support external systems provisioning more complex storage stacks for the rootfs (for example, Stratis, LVM, iSCSI, etc.)

Doc build fails

Run actions/configure-pages@v3
Error: Get Pages site failed
Error: HttpError: Not Found

really fix `install_t` in containers

I'm not sure if something changed here, or my code never actually worked but basically we currently fail to bootc install with unknown types.

Unified storage

Today this project uses ostree-containers for the base image.

Much more recently, we landed logically bound app images which ended up requiring a bootc-owned containers/storage instance.

This left us in a confusing situation where we have two image storage backends used by bootc - one for the booted host, and one for LBIs.

The code they don't share is mostly containers/storage.

This issue tracks unification: Having bootc leverage more of containers/storage (which is going to likely require containers/storage improvements.

Making a bootable image

[copied from https://github.com/vrothberg/bootc-playground/issues/1]
Vivek asked me to look this one over and I have a few suggestions which I'll attach here.

First of all, nbdkit could really do with a plugin that automates everything below. The hard part (for me) is to find and download the container data. Surely there are APIs to do that, but I've no idea how. So we could collaborate on that.

Secondly you can make a bootable disk image directly in various ways. You need to choose your poison^Wboot method. Possibilities include extlinux or direct kernel booting (qemu -kernel). But here I'm going to use UEFI -> shim -> kernel which has the advantage that it is compatible with Secure Boot. Note this requires a newish feature in Fedora 38 called Unified Kernel Image.

So say you've got a *.tar.gz containing the files in your operating system. The following script will create a bootable qcow2 file from it. Note the script does not need to be run as root (and ideally should not be run as root).

#!/usr/bin/python3
import guestfs
import re
import codecs

# This must contain shim-x64 and kernel-uki-virt:
input="/var/tmp/fedora-38.tar.gz"
output="/var/tmp/fedora.qcow2"
format="qcow2"
MB=1024*1024
GB=1024*1024*1024
disk_size=6*GB
efi_part_size=512*MB
boot_part_size=512*MB

# https://uapi-group.org/specifications/specs/discoverable_partitions_specification/
root_guid="4f68bce3-e8cd-4db1-96e7-fbcaf984b709"
# Standard UEFI ESP GUID
esp_guid="C12A7328-F81F-11D2-BA4B-00A0C93EC93B"

g = guestfs.GuestFS(python_return_dict=True)
g.set_trace(1)
g.disk_create(output, format, disk_size)
g.add_drive_opts(output, format=format, readonly=0)
g.launch()

# Make the partition table for UEFI.
g.part_init("/dev/sda", "gpt")
n=128; e=int(n + efi_part_size / 512)
g.part_add("/dev/sda", "p", n, e - 1)
g.part_set_gpt_type("/dev/sda", 1, esp_guid)
n=e; e=int(n + boot_part_size / 512)
g.part_add("/dev/sda", "p", n, e - 1)
n=e
g.part_add("/dev/sda", "p", n, -128)
g.part_set_gpt_type("/dev/sda", 3, root_guid)

# Create the filesystems and mount them up.
g.mkfs("vfat", "/dev/sda1")
g.mkfs("ext4", "/dev/sda2")
g.mkfs("xfs", "/dev/sda3")
g.mount("/dev/sda3", "/")
g.mkdir("/boot")
g.mount("/dev/sda2", "/boot")
g.mkdir("/boot/efi")
g.mount("/dev/sda1", "/boot/efi")

# Unpack the filesystem from the tarball.
g.tar_in(input, "/", xattrs=True, selinux=True, compress="gzip")

# Find the kernel UKI and copy to the ESP.
uki=next(x for x in g.ls("/boot") if re.match(r"vmlinuz-virt\.efi.*", x))
g.cp("/boot/%s" % uki, "/boot/efi/EFI/KERNEL.EFI")

# Update BOOTX64.CSV so shim will boot the kernel directly,
# without grub.
cmdline=g.cat("/etc/kernel/cmdline")
csv="shimx64.efi,KERNEL,\\EFI\\KERNEL.EFI %s" % cmdline
g.write("/boot/efi/EFI/fedora/BOOTX64.CSV",
        codecs.BOM_UTF16_LE + csv.encode('utf-16-le'))

# Fix up /etc/fstab (may not be necessary with autodetect?)
root_fs_uuid=g.vfs_uuid("/dev/sda3")
boot_fs_uuid=g.vfs_uuid("/dev/sda2")
efi_fs_uuid=g.vfs_uuid("/dev/sda1")
fstab = """
UUID=%s / xfs defaults 0 0
UUID=%s /boot ext4 defaults 0 0
UUID=%s /boot/efi vfat defaults 0 0
""" % (root_fs_uuid, boot_fs_uuid, efi_fs_uuid)
g.write("/etc/fstab", fstab)

# Unmount everything and sync image.
g.umount_all()

To boot this as a transient VM:

$ rm ~/.config/libvirt/qemu/nvram/fedora38_VARS.fd
$ virt-install --transient --import --disk path=fedora.qcow2  --boot uefi --os-variant fedora38

(That rm command is because of a bug in libvirt)

The domain should go away on its own at shut down (since it is --transient), but you could also kill it with virsh destroy <N>. However the _VARS.fd file is left over afterwards (libvirt bug) so you have to delete that manually else it will interfere with future boots of any same-named VM.

To create the *.tar.gz file for testing purposes I used this script, but I guess in your case you'd somehow get podman to give you the tarball. (See my nbdkit request above).

#!/bin/bash -
set -e
set -x
virt-builder fedora-38 \
             --install shim-x64,kernel-uki-virt \
             --root-password password:123456
guestfish --ro -a fedora-38.img -i \
          tar-out / - xattrs:true selinux:true compress:gzip > fedora-38.tar.gz
rm fedora-38.img

Note my tarball was prepared with SELinux labels. systemd will refuse to boot hard if it finds an unlabelled filesystem, so if you have that you'll need to either do selinux=0 or g.touch("/.autorelabel") in the Python script.

Add support for `update --apply`

Today we have e.g. rpm-ostree upgrade --reboot. bootc update does not have an equivalent verb.

In this proposal we add update --apply which would today by default reboot. In the future ostree should gain support for systemd soft-reboot.

A bit longer term than that, we'll want to have rich tooling for admins/tooling more directly controlling the flow here.

First, an important detail is that I think we should default to locking finalization so that a reboot for other reasons doesn't apply the update - this ensures predictability for the sysadmin. This would be exposed as a new locked: bool in the CRD for staged deployments that could be changed to false if the admin/upgrade tooling wanted a reboot to also apply by default.

Also, what I think would be helpful here is to add some sort of "diff" to the CRD as well that says what the "apply strategy" would be (reboot, soft reboot or live apply).

Producing pxe bootable kernel, initramfs, and rootfs image using bootc

Hi all, it's been exciting to see this project progress and reach a 0.1 release!

This may be completely out of scope, but do the contributors have any thoughts about how this project (or related others) might produce a set of PXE bootable images (kernel, initramfs, and rootfs) from a bootc compatible image?

consider some way to verify target OS compatibility with `install-to-filesystem`

I got caught out with using install-to-filesystem --replace=alongside against a Fedora Cloud instance (using btrfs) when targeting Fedora ELN (doesn't enable btrfs).

This problem generalizes to things like installing to LVM but not having LVM in the container, etc.

This is quite tricky to test, but I think we could at least do a dirty sanity check for having the kmod in the target kernel or so?

Build fails on s390x:

Build fails on s390x:

error[E0425]: cannot find value `EFIPN` in this scope
   --> /builddir/build/BUILD/rpm-ostree-2023.5.38.g0cbcf4ae/vendor/bootc-lib/src/install/baseline.rs:247:13
    |
247 |             EFIPN,
    |             ^^^^^ not found in this scope
error[E0425]: cannot find value `EFIPN_SIZE_MB` in this scope
   --> /builddir/build/BUILD/rpm-ostree-2023.5.38.g0cbcf4ae/vendor/bootc-lib/src/install/baseline.rs:248:26
    |
34  | pub(crate) const BOOTPN_SIZE_MB: u32 = 510;
    | ------------------------------------------- similarly named constant `BOOTPN_SIZE_MB` defined here
...
248 |             format!("0:+{EFIPN_SIZE_MB}M"),
    |                          ^^^^^^^^^^^^^ help: a constant with a similar name exists: `BOOTPN_SIZE_MB`
error[E0425]: cannot find value `EFIPN` in this scope
   --> /builddir/build/BUILD/rpm-ostree-2023.5.38.g0cbcf4ae/vendor/bootc-lib/src/install/baseline.rs:252:32
    |
252 |         Some(format!("{device}{EFIPN}"))
    |                                ^^^^^ not found in this scope
For more information about this error, try `rustc --explain E0425`.
error: could not compile `bootc-lib` due to 3 previous errors
Caused by:
  process didn't exit successfully: `rustc --crate-name bootc_lib --edition=2021 /builddir/build/BUILD/rpm-ostree-2023.5.38.g0cbcf4ae/

This blocks an rpm-ostree release as we are vendoring bootc in rpm-ostree.

`--karg` option for `install-to-filesystem` is ignored?

Starting from a Fedora 38 Cloud VM, I used the following command to install a bootable container:

$ sudo podman run --rm --privileged --pid=host -v /:/target --security-opt label=type:unconfined_t registry.gitlab.com/centos/cloud/sagano/fedora-boot-tier-1-dev:eln bootc install-to-filesystem --target-no-signature-verification --karg=console=ttyS0,115200n8 --replace=alongside /target

The operation completed successfully, however upon reboot I found that the console= karg was not included in the bootloader config.

image

Add `install` verb

We should support something as straightforward as

$ bootc install quay.io/examplecorp/mycustomos:latest /dev/nvme0n1

This would:

  • Partition the disk in an opinionated way
  • Install the bootloader
  • Install the content from the container image

Partitioning

Obviously...there's a lot of tooling here. We should consider supporting at least Ignition as well as systemd-repart for nontrivial disk image setups.

Bootloader

For now, let's require that the target OS is setup using bootupd.

container image install

All the rest of the stuff (i.e. 99%) comes from the container image itself; we just need to call out to e.g. https://github.com/coreos/coreos-assembler/blob/d039e883059c48f2310572955682b9d25f4b6d61/src/create_disk.sh#L350

(Of course the steps above are also done in the example cosa code - we could hardcode some defaults, or probably better make it configurable in the image)

One idea here is to do a multi-stage fetch; we pull the container via e.g. podman (or actually instead, assume the UX is podman run --privileged quay.io/examplecorp/mycustomos:latest bootc install /dev/vda ? Then we can use something more like the deploy-from-self logic)

install --replace-booted-block-device

We could also support bootc install --replace-booted-block-device quay.io/examplecorp/mycustomos:latest which would implement this flow - basically, we:

  • Move the running code into RAM (systemd switch root? Or just replace pid1 entirely)
  • wipefs the booted block device
  • Run the same install logic over that block device

install --switch-root

It'd make sense to support systemctl switch-root to the new target rootfs, but without using the new kernel. (But, we should be really clear when we're in a kernel != root situation)

dynamic overlays

I think it would make sense to support functionality similarly to https://www.freedesktop.org/software/systemd/man/systemd-sysext.html - we could even use that as a backend, though it needs some design around interactions eventually lowering "live apply" type flows.

A strawman could look like:

$ bootc overlay add quay.io/examplecorp/seattle-manufacturing:wifi

That would add a new container image that would be dynamically unioned with the host's rootfs - on updates, we reapply both - and we'd also verify compatibility and fail if e.g. they appeared somehow incompatible. (A lot of potential things to do in "verify compatibility...")

[Request] Arch Linux demo

Is it actually possible to use distros which aren't based on RPM in the current state of this project?

Error: Linking tmp mounts to host

I just tried to follow the basic example from install.md document. Unfortunately, this fails right at the start.

root@ishikawa ~# podman run --privileged --pid=host --net=none --security-opt label=type:unconfined_t ghcr.io/cgwalters/c9s-oscore bootc install --target-no-signature-verification /dev/nbd0
ERROR Linking tmp mounts to host: Exchanging /tmp <=> /tmp.tmp: Invalid argument (os error 22)
root@ishikawa ~# rpm -q podman
podman-4.2.0-9.el9_1.x86_64
root@ishikawa ~# cat /etc/os-release 
NAME="Red Hat Enterprise Linux"
VERSION="9.1 (Plow)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="9.1"
PLATFORM_ID="platform:el9"

drop dependency on xfs_freeze

Reported via email; we should instead just directly invoke the ioctl. ostree has code for this we could make into a public C API, including the complex dance of doing this by forking a helper process to be robust against interruption.

bootc switch failed

Build bootc, and start FCOS with --bind-ro, run ./bootc switch --no-signature-verification quay.io/fedora/fedora-coreos:next and get error.

# ./bootc status
*  e051d49946dd7685ed1fa52c924506fc997958fd002ab39f9b131d0c45b72bb0.0
    (Non-container origin type)

    Backend: ostree
    Booted: yes

# ./bootc switch --no-signature-verification quay.io/fedora/fedora-coreos:next
layers stored: 0 needed: 51 (693.8 MB)
 1.32 MiB [████████████████████] (0s) Fetched ostree chunk sha256:10b8c17c44e6
error: Invalid ref name fedora:fedora/x86_64/coreos/next

Another question, does bootc switch have the same func with rpm-ostree rebase <container image> ?

Enable renovate

Every new version of the quay.io/centos-boot/fedora-tier-1 image will generate a PR in this repo which will invoke the CI with the new version

run as `install_t`

Ah yes, this bites again 😢

$ error: Importing: Layer sha256:fef321502627eccb4eeabbc2ad0788e82c68e192d41b6677c4570202cadde69d: Importing object 23/a4d18c7b21b51a284dedd705c739584dcb690607ab831aa97bafee06f0482c.file: Processing content object 23a4d18c7b21b51a284dedd705c739584dcb690607ab831aa97bafee06f0482c: Writing content object: Setting xattrs: fsetxattr(security.selinux): Invalid argument

Today e.g. rpm-ostree runs as install_t which lets it set unknown security contexts. We need to do the same...maybe hack in a quick setexeccon() or so.

Add opt-in support to readonly `/etc`, also doc transient overlayfs for `/etc`

We want bootc to support both "generic, unlocked" systems that have persistent mutable state in /etc, as well as "sealed" systems that are only configured via bootc, and shouldn't have persistent mutable configuration.

(Similar, for /var, though in the general case for that I think people will instead want to allow-list at least some persistent paths, like the journal)

install-to-filesystem does not work when targeting LVM

Host system is Fedora 39 Server.

Using the tier1 Fedora ELN image, I created a custom variant that had bootc installed from Copr:

FROM registry.gitlab.com/centos/cloud/sagano/fedora-boot-tier-1:eln
RUN curl -L -o /etc/yum.repos.d/bootc.repo https://copr.fedorainfracloud.org/coprs/rhcontainerbot/bootc/repo/fedora-rawhide/rhcontainerbot-bootc-fedora-rawhide.repo
RUN sed -i 's|\$releasever|rawhide|' /etc/yum.repos.d/bootc.repo
RUN rpm-ostree install bootc
RUN ostree container commit

Built it successfully with sudo podman build -t sagano-bootc-copr:latest -f Containerfile.sagano .

Tried install-to-filesystem:

$ image=localhost/sagano-bootc-copr:latest
$ sudo podman run --rm --privileged --pid=host -v /:/target --security-opt label=type:unconfined_t "${image}" bootc install-to-filesystem --target-no-signature-verification --karg=console=ttyS0,115200n8 --replace=alongside /target
Mounting selinuxfs
ERROR No filesystem uuid found in target root

Hmm...looks like it might be SELinux as the culprit:

$ sudo ausearch -m avc
----
time->Fri Nov  3 12:03:42 2023
type=AVC msg=audit(1699027422.008:673): avc:  denied  { nosuid_transition } for  pid=3425 comm="bootc" scontext=system_u:system_r:unconfined_t:s0:c54,c375 tcontext=system_u:system_r:install_t:s0:c54,c375 tclass=process2 permissive=0

add support or verify support of OCI crypt

It'd be great to be sure we support OCI crypt; it'd greatly help use cases of embedding secret data inside a bootable container image. I need to dig into the flow for this.

Clearly for a bootc install style flow we'd need to have the provisioning system (e.g. cloud-init -> AMI or Anaconda) also be configured with decryption keys.

We should definitely still support secrets other ways too; I think support oci-crypt would be also be natural when we extend support for configmaps and particularly secrets.

Better handle no-op updates

It's possible today to get a no-op update where e.g. just metadata like labels change. We should definitely detect this and highlight the no-op nature of the change; probably just default to "live updating" the origin? Should we record that transition at all though (just in the journal)?.

remote config via configmap and secrets

This overlaps with #7 some.

Here, the basic idea is something like:

bootc config add [--root=/etc] https://examplecorp.com/config.yml

(OR with support for OCI Artifacts we support bootc config add [--root=/etc] registry:quay.io/examplecorp/config-server-base:latest)

Where config.yml is a standard Kubernetes ConfigMap. By default, we "mount" the keys to /etc. Then, bootc upgrade looks for updates to all provided configmaps - if any change, it triggers the same upgrade logic as the base image.

We also fetch and handle secret objects in the same way. It'd be cool though to support something like handling encrypted secrets (and configmaps) which need to be decrypted via a key (which could be in a TPM or so).

We also need to think carefully about file permissions; mode 0644 for all configmap files and 0600 for secrets may make sense. In addition we could support special annotations to override these.

(This should also work to be invoked immediately after bootc install to have it ready on the first boot, i.e. we also have a --root argument or so)

switch to being a systemd service

Right now we have a lot of simplicity by not running under systemd. But there are a lot of advantages to doing so; see e.g. https://github.com/coreos/bootupd#questions-and-answers

All of the below applies to us too:

  • Why is bootupd a daemon?

It's not, really. The name was intended to be "bootloader-upDater"
not "bootloader-updater-Daemon"; the choice of a "d" suffix is
in retrospect probably too confusing.

At a technical level, yes there is a socket-activated systemd service
which will spawn a bootupd.service. However - the service will
very quickly auto exit. There's nothing long-running, so it's
not really a daemon.

The rationale behind this design is:

  • Using a systemd service provides a robust natural "locking"
    mechanism.
  • Using a systemd service ensures that critical logging metadata
    always consistently ends up in the systemd journal, not e.g.
    a transient client SSH connection.
  • systemd services can easily have sandboxing applied, and
    while bootupd is obviously privileged we can still make use
    of some of this.
  • Ultimately, we do probably want a non-CLI API (whether that's
    DBus or Cap'n Proto or varlink or something else). Having
    a socket (without a defined stable API) is preparatory work for that.

Add `bootc apply-live`

This is probably mainly draining the logic from rpm-ostree into ostree-ext, then re-using it here.

But...a whole lot of suddenly OS-specific issues come to the fore. For example, should we try to distinguish between "new content" and "changes"?

consider supporting fetching instance metadata

One thing we could consider is streamlining the process of configuring an instance to handle instance metadata.

Here we'd have an opinionated flow to fetch from the same sources as Ignition/afterburn to describe the desired system configuration, which would here be:

  • container image reference
  • configmap and secrets
  • actually in general all bootc install options

In this flow, the OS vendor would ship e.g. AMIs that include a systemd unit that does this by default, and checks whether it has bootc-compatible metadata. If it doesn't (e.g. it looks like stock cloud-init config) then we ignore it?

Clarify relationship with container support in rpm-ostree

rpm-ostree learned how to e.g. rebase to and update from a container ref. Is bootc intended to supersede this? Will some of the bits in rpm-ostree be deprecated and removed? Will rpm-ostree call out to bootc?

Might be worth adding an entry in the README about this.

Add a RPC API

This heavily relates to https://github.com/cgwalters/bootc/issues/2 - but does not strictly depend on it.

My initial take is to add something like bootc upgrade --json which would not do interactive status prints, but write a defined stable JSON output format to stdout (or a specifically targeted fd).

But...it may be more sustainable to have a better protocol; as the issue says, whether that's DBus or Cap'n Proto or varlink, etc.

bios installs fail

One thing I haven't yet debugged is for some reason grub fails with BIOS. There must be something different around how we're creating the partitions or installing things.

Ideally also, we should switch the installation code to use the new bootupd BIOS support.

k8s-openapi vendor size

OK yeah I didn't notice this at the time, but k8s-openapi has...an absolutely incredible amount of generated source code.

$ du -sc vendor | sort -nr
197216  total
62304   k8s-openapi
6924    linux-raw-sys
5420    linux-raw-sys-0.3.6
5188    encoding_rs
4868    tokio
4400    libc
3928    vcpkg
2848    regex-automata
2636    rustix

I think we can trim this; we just want the core type definitions and CRD stuff. This issue is problematic enough that we should maybe try dropping the dependency entirely.

Upstream link: Arnavion/k8s-openapi#148

Needs a tutorial or blog on how one would use this.

Would it be possible to build a base imae with osbuilder and push it to a container registry,
Add a layer to this image using buildah or podman build and update the image.

The use bootc --install to install the image on top of a VM or cloud image?

Add support for `bootc install --with-overlay quay.io/exampleuser/someoverlay:latest`

I've been advocating general configmap and secret support for a while.

However, the way CoreOS does things today with Ignition is basically allowing the admin to inject "day 1 unmanaged state" that is decoupled from the image/host. And actually, that's how Anaconda kickstart works too.

There's a lot of advantages to configmap-in-registry around supporting "day 2" changes too. However, it will require workflow and tooling changes to adapt.

One key thing I'm narrowing in on is the basic use case of "inject a ssh key into an existing golden image" - a thing I think we really want to support. And we do today via e.g. Anaconda or other installers.

But I think we should support this more natively via bootc install too.

So here's my strawman: the --with-overlay argument takes a container image reference and operates only at install time. A very key difference from the configmap approach is that the image must drop files into /etc and /var. This gives us immediate, direct support for SSH keys by injecting e.g. /var/roothome/.ssh/authorized_keys.

Migrating from Ignition

But actually the neat thing with this is it provides a very seamless adaptation of existing Ignition configuration! We've already prototyped out support for extracting an Ignition configuration to its set of files via ignition-apply, so it'd literally look like:

FROM quay.io/coreos/butane:release as builder
COPY foo.ign foo.ign
RUN ignition-apply --root /config foo.ign # render ignition to filesystem tree, error on partitioning

FROM scratch
COPY --from=builder /config /

Then push that to a registry as quay.io/exampleuser/someoverlay:latest, and one can do bootc install --with-overlay quay.io/exampleuser/someoverlay:latest and then one can switch to installing via bootc without needing to change all the bits in the Ignition config to move to /usr content etc.

So any tools (e.g. OpenShift MCO) that act on "day 2" management for this "unmanaged state" can seamlessly continue to work - but we avoid the need to e.g. host Ignition servers.

Drop host requirement on `skopeo`

Today, bootc install runs from a privileged container, using the host kernel, but otherwise all binaries executed come from the container (including e.g. mkfs.xfs etc.).

With one glaring exception; today we need skopeo in the host. For more information on this, see containers/skopeo#1838

One thing we could do relatively easily today is detect if the host and container are compatible, and copy the skopeo binary to the host if so. This would help a lot of cases, but not all.

tracker: logically bound app images

We should support a mechanism where some container images are "lifecycle bound" to the base bootc image.

Logically bound images

A common advantage/disadvantage of the below is that the user must manage multiple container images for system installs - e.g. for a disconnected/offline install they must all be mirrored, not just one.

In this model, the app images would only be referenced from the base image as .image files or an equivalent.

This contrasts with physically bound images.

bootc logically bound flow

bootc upgrade would follow a flow like:

  • fetch new base image
  • Read its root filesystem, discover logically bound images
  • Pull those images into /var/lib/containers/storage

Proposed: symlink to .image or .container files

Introduce /usr/lib/bootc/bound-images.d that is symlinks to .image files or .container files.

Pros:

  • Straightforward to implement
  • The admin only needs to bump a :sha256 digest in one place to update

Cons:

  • Handling specifiers is tricky, we will error out on them
  • No separation of concerns: an .image file is intended to pull images not be parsed by an external tool for a separate purpose.
  • Updates to Quadlet may break the process and/or add (potential) continuous maintenance burden for bootc (i.e., "chasing/reimplementing new features").
  • Forces users to use Quadlet even if they have no use for pulling images under systemd.

Other alternatives considered

New custom config file

A new TOML/usr/lib/bootc/bound-images.d, of the form e.g. 01-myimages.toml:

images = ["quay.io/testos/someimage@sha256:12345", "quay.io/testos/otherimage@sha256:54321"]
authfile = "/etc/containers/my-custom-auth.json"

Pros:

  • Easy to just list multiple images vs one image per .image file
  • TOML format is used by other bootc tooling and some of the container config formats

Cons:

  • New file format relating to container images
  • May need in the general case to support many of the existing options in .image files
  • The admin will need to bump a :sha256 digest in two places to update in general (both in a .container or .image and the custom .toml here)

Parse existing .image files

Pros:

  • Well known
  • Spec -> pull translation exists
  • Existing spec handles most image pull fields

Cons:

  • Would need to extend the spec to include a new bootc=bound or equivalent opt-in
  • Handling specifiers is tricky
  • Implentation complicated wrt managing systemd

What would happen under the covers here is that bootc would hook into podman and:

  • disallow GC of these images even if the unit isn't running (for all deployments)
  • Fetch new images (from the new base container image) on bootc upgrade

TODO:

  • docs
  • CI test
  • PR to fedora bootc examples
  • Ensure compatibility with bootc-image-builder #715
  • install path with bootc install to-filesystem - simple scenario w/out pull secret?
  • install path w/pull secret embedded in bootc image? podman pull happens from bootc container
  • install path w/bootc-image-builder where it pre-pulls images, demonstrated e2e w/konflux, we probably need to enable a model where bound images are provided in a mirror location or OCI directory

building main branch fails

Not sure if I am missing something but trying to build main branch is failing for me with the following error,

$ cargo build --release
   Compiling ostree-sys v0.12.0
   Compiling glib v0.16.7
   Compiling containers-image-proxy v0.5.2
The following warnings were emitted during compilation:

warning: `PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" "pkg-config" "--libs" "--cflags" "ostree-1" "ostree-1 >= 2022.5"` did not exit successfully: exit status: 1

error: failed to run custom build command for `ostree-sys v0.12.0`

Caused by:
  process didn't exit successfully: `/home/harshal/Projects/bootc/target/release/build/ostree-sys-2f1decd0de242109/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=OSTREE_1_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  cargo:warning=`PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" "pkg-config" "--libs" "--cflags" "ostree-1" "ostree-1 >= 2022.5"` did not exit successfully: exit status: 1
  error: could not find system library 'ostree-1' required by the 'ostree-sys' crate

  --- stderr
  Package ostree-1 was not found in the pkg-config search path.
  Perhaps you should add the directory containing `ostree-1.pc'
  to the PKG_CONFIG_PATH environment variable
  Package 'ostree-1', required by 'virtual:world', not found
  Package 'ostree-1', required by 'virtual:world', not found

warning: build failed, waiting for other jobs to finish...

Exploring potential collaboration between Kairos and Bootc projects

hello! 👋

I've just learned today about https://github.com/containers/bootc ! I tried to reach over matrix but was redirected here - I'm the maintainer of https://github.com/kairos-io/kairos which is a project that have many crossing points with bootc, and I was wondering myself if you would be interested in having a chat! I see there are community events for podman but none for bootc. Just curious, how does it handles upgrades without ostree? is it a/b style? is the OS immutable or takes over only the upgrade process?

Best

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.