snapcore/core-initrd

initrd implementation in ubuntu core

core-initrd's Issues

Missing stock link files

stock .link files that are normally shipped in systemd do not seem to be included in the initrd, as reported elsewhere.

Make the ubuntu-core-initramfs tree more dynamic

vendoring static copies of packages is ok, but causes a lot of churn.
we should attempt to have more dynamic tree of depends, instead of static copies.

factory: handle properly snapd_recovery_mode based on modeenv value

Currently main, core23.10, core22, core20 have incomplete or incorrect handling of the factory mode.

As per #213

@kubiko are you going to provide updated merge request for main first? or do you want someone else to take over?

The likely intention is to consistently do https://github.com/snapcore/core-initrd/pull/213/files in all active code branches

ubuntu-core-initramfs v66.1 and 66 missing libs on ARM64

Good day,

It appears that building Ubuntu Core 20 image with custom kernel is missing the libpthread.so.0 library, using latest 66.1 arm64 deb package from ubuntu image PPA. (Jammy?)

Booting with snapd/edge version -> 2.59.4+git895.g5aeeeae

         Starting Wait for the Ubuntu Core chooser trigger...
[    5.446717] caam 30900000.crypto: device ID = 0x0a16040100000100 (Era 9)
[    5.455275] caam 30900000.crypto: job rings = 1, qi = 0
[    5.408604] systemd[1]: Condition check resulted in Daily Cleanup of Temporary Directories being skipped.
[FAILED] Failed to start Wait for the Ubuntu Core chooser trigger.
See 'systemctl status snapd.recovery-chooser-trigger.service' for details.
[    5.485836] systemd[1]: Reached target Basic System.
[    5.528596] systemd[1]: Reached target Timer Units.
[    5.540693] snap-bootstrap[294]: @snap-bootstrap: error while loading shared libraries: libpthread.so.0: cannot open shared object file: No such file or directory
[    5.565044] systemd[1]: Condition check resulted in Show Plymouth Boot Screen being skipped.
[    5.584506] systemd[1]: Starting Wait for the Ubuntu Core chooser trigger...
[    5.600359] systemd[1]: snapd.recovery-chooser-trigger.service: Main process exited, code=exited, status=127/n/a
[    5.620379] systemd[1]: snapd.recovery-chooser-trigger.service: Failed with result 'exit-code'.
[    5.640325] systemd[1]: Failed to start Wait for the Ubuntu Core chooser trigger.
[    5.656818] systemd-udevd[282]: Using default interface naming scheme 'v249'.
[    5.672802] systemd-udevd[283]: Using default interface naming scheme 'v249'.

Checking inside that deb, it appears that libpthread is not present, but present in a older ubuntu-core-initramfs v55 package.

@alfonsosanchezbeato Could you verify this on a ARM64 platform?

Use snap-bootstrap from master snapd branch when possible

Currently we build snap-bootstrap from https://github.com/xnox/snapd/tree/run-cloudimg-rootfs-draft-1 instead of using https://github.com/snapcore/snapd so cloud images are supported. Use master as soon as that branch is merged.

Start running spread tests

spread tests were merged.

Please add keys/actions/stuff to this repo to start running tests.

@niemeyer @anonymouse64

overlap between extra-modules.conf and ubuntu-core-initramfs.conf

version: ubuntu core 22
Using combination of

• extra modules in ${skeleton}/modules/main/extra-modules.conf
• configured modules to be loaded by systemd in ${skeleton}/main/usr/lib/modules-load.d/ubuntu-core-initramfs.conf

If those two lists contain the same kernel module(s), it makes ubuntu-core-initramfs create-initrd fail with the following error:

Traceback (most recent call last):
  File "/home/ondrak/kernel-snap/parts/kernel/build/ubuntu-core-initramfs/usr/bin/ubuntu-core-initramfs", line 490, in <module>
    main()
  File "/home/ondrak/kernel-snap/parts/kernel/build/ubuntu-core-initramfs/usr/bin/ubuntu-core-initramfs", line 486, in main
    globals()[args.subcmd.replace("-", "_")](parser, args)
  File "/home/ondrak/kernel-snap/parts/kernel/build/ubuntu-core-initramfs/usr/bin/ubuntu-core-initramfs", line 310, in create_initrd
    add_modules_from_file(main, kernel_root, modules, firmware, module_load, db,
  File "/home/ondrak/kernel-snap/parts/kernel/build/ubuntu-core-initramfs/usr/bin/ubuntu-core-initramfs", line 251, in add_modules_from_file
    db.mark_installed(module, conf_file)
  File "/home/ondrak/kernel-snap/parts/kernel/build/ubuntu-core-initramfs/usr/bin/ubuntu-core-initramfs", line 135, in mark_installed
    elif old_mode == ModuleDb.IMPLICIT:
AttributeError: type object 'ModuleDb' has no attribute 'IMPLICIT'

core24 adding sulogin

in #182 we are adding sulogin, where sh was previously used, figure out if that's all correct going forward.

emergency.target.d/core-override.conf seems to be incorrect

During the boot there are a bunch of warnings like this:

emergency.target: Requested dependency OnFailure=reboot.target ignored (target units cannot fail).

which indicates that the factory/usr/lib/systemd/system/emergency.target.d/core-override.conf file is not quite correct.

Mantic ubuntu-core-initramfs module warnings please review

Building kernel snaps in Mantic produces the following warnings:

amd64 generic
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module squashfs is builtin
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module kmod-nls-cp437 not found
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module dwc2 is builtin
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module cryptomgr is builtin
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module dm_mod is builtin
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module cbc is builtin
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module xts is builtin
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module i2c-bcm2708 not found
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module sdhci-iproc not found
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module vc4 not found
NOTE: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module =drivers/hid not found
WARNING: /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf: Module xhci-pci-renesas exports symbols:
 * symbol:renesas_xhci_check_request_fw
WARNING: Module xhci-pci-renesas installed by /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf, but is dependency of xhci-pci installed by /usr/lib/ubuntu-core-initramfs/modules/main/extra-modules.conf

Please review if above is correct and expected, and if you want to change anything.

Specifically:

1. Module =drivers/hid not found sounds like a missing validation feature
2. Module xhci-pci-renesas installed by ... dep of xhci-pci maybe can be optimized

UC20 boot error "Failed to make path absolute" on Fitlet2 device

Trying to build a custom image for the Compulab Fitlet2 (just a extended pc-amd64-gadget) and kept getting this error. Using the original ubuntu-core-20-amd64.img.xz from https://cdimage.ubuntu.com/ubuntu-core/20/stable/current/ results in the same error. The device is booting of USB key.

Already have a working Core16 image, device can run regular Ubuntu 20.04 from USB. Image works on laptop and QEMU, just not on this device.

Error reads:
the-tool[237]: error: Failed to make path /dev/disk/by-partuuid/...: No such file or directory

Partition UUID matches the ubuntu-seed partition of the official image, on the custom image it points to the ubuntu-boot partition. Have tried different USB keys, but all of them report the same error.

Device specs:

• Intel Atom X7-E3950
• 4GB Memory

Just to be sure I tried booting using UEFI and legacy mode, with and without secure boot enabled (not configured), all result in the same error.

USB is "hp v195b" which is logged at 2.7 in the screenshot, after the error has occurred. However other tests show that the error occurs after the message related to the USB key.

moved elsewhere

Kernel team should promote default modules to the main modules package

modules included by default in core initrd, should be by default in linux-modules package, not extra

Properly describe code structure in the documentation

We need to clearly describe code structure of this repository in the README.md or ARCHITECTURE.md files

Something like:

├── bin   ---> Contains ubuntu-core-initramfs to build initrd.img
├── debian ---> debian directory to build deb package 
├── factory --->  main skeleton of the initrd cpio archive
├── features
├── postinst.d
├── snakeoil
├── tests
├── vendor
├── COPYING
├── crypttab
├── grub.cfg
├── HACKING.md
├── initramfs.debug
├── LICENSE
├── README.md
└── spread.yaml

• bin ---> Contains ubuntu-core-initramfs to build initrd.img
• debian ---> debian directory to build deb package . Especially see "rules" file to see how it builds systemd and builds other relevant dependencies
• factory ---> main skeleton of the initrd cpio archive

factory: installation from initrd must apply dbx revocations

today dbx revocations are applied from the base snap upon install mode boot.

If install mode boot is never performed, dbx revocations must be applied before sealing luks key on EFI capable platforms.

Mantic ubuntu-core-initramfs depmod warnings

mantic generic amd64 builds produce the following warning

depmod: WARNING: could not open modules.builtin.modinfo at /tmp/tmpgli9l1hl.ubuntu-core-initramfs/main/lib/modules/6.3.0-7-generic: No such file or directory

Maybe this means some new additional libkmod file from the kernel build is not copied over into the initrd?

Fix the orphan snakeoil/OVMF_VARS.snakeoil.fd file

snakeoil/OVMF_VARS.snakeoil.fd might not match OVMF_CODE.secboot.fd we use in tests.

Here are some ways we could handle it:

• Use lockdown.efi from efitools to install snakeoil keys on any OVMF_VARS.fd
• Put the snakeoil keys as well as code and variables images for OVMF in a separate repo.

ADD tpm_tis_spi to default modules in initrd

for secure boot in Ubuntu core usign TPM2 it is necessary to have tpm_tis_spi module force loaded. adding in the config.txt to enable tpm does not automatically load the module.

arm stub fails to load dtb without a previous dtb present

arm stub fails to load dtb without a previous dtb present, due a bogus check for which we did not yet cherry-pick a fix for.

do not attempt loading modules that do not exist

if one failed to include a module into initrd, do not load them.

Fix cloudimg-rootfs feature in Jammy

cloudimg-rootfs feature is broken in Jammy. Needs fixing.

As pointed out in the review of #76

Run snapd spread tests in CI

If doing sensitive changes in core-initrd, we need to create a draft branch in snapcore/snapd and trigger CI to run the spread tests. It would be better if the CI for core-initrd could trigger those tests as part of its CI.

Add testcases for cloudimg-rootfs feature using nullboot and generic/kvm kernel

cloudimg-rootfs is currently not tested in core-initrd project. But it should be. Ideally by self-building kernel.efi for generic or kvm kernel, and using nullboot to deploy bootloader and reboot.

Directly support arm64 compressed kernels

Very new kernels support nested compressed EFI kernels on arm64.
I wonder if sd-boot can add support for that.

Separately, we should, for now, support generating valid arm64 kernel when a gz compressed kernel is supplied.

As at the moment, we do a lot of code in every arm64 kernel to decompress vmlinuz prior to creating kernel.efi.

must set MajorImageVersion

must set MajorImageVersion

To insure kernel.efi is compatible with pure vmlinuz and pure grub MajorImageVersiona dn MinorImageVersion from vmlinuz should be copied into the kernel.efi.

Please include "systemd-run" in initrd

The UC20 full-disk-encryption will provide a way to run helpers to support special hardware for the encryption. To support this we need to run a new "fde-reveal-key" binary as part of initramfs. We would like to run this binary with systemd-run to benefit from some of the systemd features like automatic kill after a certain timeout and doing some basic sandboxing around it. This is sketched in snapcore/snapd#9488

For this to work we would like to ask to include systemd-run inside the initramfs. If it's too much of an issue or too big we could as only the custom kernels to include it or we could drive systemd via the dbus API but for simplicity we would like to have the binary as our first choice.

weekly releases needed

currently ubuntu-core-initrd vendors in lots of binaries at build time.

to ensure they are up to date we should trigger rebuilds of ubuntu-core-initrd, or have regular schedule when they release.

also given the success of automatic builds & releases of mantic branch, we should consider enabling automatic releases upon merged to focal & jammy branches too.

and probably have github action to automatically generate weekly commits and thus core-initrd releases.

Improve server feature to include most storage drivers

In classic Ubuntu we default to "most" modules which really is a very large kitchen sync.

I wonder what is a sensible and a reasonable set for server feature:

• virtio_pci virtio_mmio
• ide (or is ide dead)
• mmc
• scsi & mptfc mptsas mptscsih mptspi zfcp
• ata
• block
• nvme
• vmd
• usb/storage

Recovery might want:

• most of the USB host and dual-role drivers ?!
• all of the hid & input/keyboard stuff
• hv_*

I am kind of concerned that in the classic & core initrd we force load lots of modules without any detection if they are needed or if any devices are present at all or not.

systemd-run seems to be not working

I tried to use systemd-run in the initramfs to run the "fde-reveal-key" binary. It seems this is not quite working yet, see the attached screenshot.

improve spread tests further

see comments in #1

missing rtc modules for RPi CM4

To set the clock from /dev/rtc0 on a CM4 the rtc-pcf85063 module needs to be in the pi-kernel initrd ... since there are issues with calling hctosys from the kernel at module load time there probably also needs to be a udev rule like:

$ cat /etc/udev/rules.d/60-rtc.rules
ACTION=="add", SUBSYSTEM=="rtc", ATTRS{hctosys}=="0", RUN+="/usr/sbin/hwclock -s --utc"

and the hwclock binary to set the clock before trying to decrypt/mount the rootfs disk

a corresponding kernel bug has been opened as:

https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/1926911