o8vm/krabs

An x86 bootloader written in Rust.

KRaBs: Kernel Reader and Booters

KRaBs is an x86/x86_64 chain loader written in pure Rust.

What is KRaBs?

KRaBs is working on booting vmlinux and other kernels formatted in ELF on 32-bit/64-bit PCs and is under the development. Krabs also aims to support only the minimal Linux x86/x86_64 boot protocol. This allows you to use the kernel command line and initrd/initramfs.

Other features:

• Supports GPT and FAT32 File System on EFI System Partition(ESP).
• You can configure KRaBs's boot option in CONFIG.TXT on FAT32 ESP.
• CONFIG.TXT is a simple matrix-oriented text file. See CONFIG.TXT formats.

News

• 2020/08: Currently, KRaBs can boot kernel-5.8.3! initrd and kernel command line also work fine!!. see details

  cargo run -- -we disk.img
  

Getting Started

To get started with KRaBs, build it from source.

Requirements

1. rust-src and llvm-tools-preview
2. GPTed disk image that has BIOS Boot Partition and EFI System Partition.
3. CONFIG.TXT, kernel image and initrd on FAT32 FileSystem on EFI System Partition.
4. (option) qemu-system-x86

Prepare rust-src and llvm-tools-preview

$ cd /path/to/krabs
$ rustup component add rust-src
$ rustup component add llvm-tools-preview

Example of GPT disk image:

$ gdisk -l disk.img 
...
Found valid GPT with protective MBR; using GPT.
Disk disk2.img: 204800 sectors, 100.0 MiB
Sector size (logical): 512 bytes
Disk identifier (GUID): 2A1F86BB-74EA-47C5-923A-7A3BAF83B5DF
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 204766
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            4095   1024.0 KiB  EF02  BIOS boot partition
   2            4096          106495   50.0 MiB    EF00  EFI system partition
   3          106496          204766   48.0 MiB    8300  Linux filesystem
$ sudo kpartx -av disk.img
$ sudo mkfs.fat -F 32 /dev/mapper/loop0p2
$ sudo mkfs.ext4 /dev/mapper/loop0p3

Prepare CONFIG.TXT, kernel, initrd:

$ sudo mount /dev/mapper/loop0p2 /mnt
$ ls /mnt
CONFIG.TXT  initramfs.cpio.gz vmlinux-5.8.3

CONFIG.TXT format

simple matrix-oriented text file

main.kernel vmlinux-5.8.3
main.initrd initramfs.cpio.gz
main.cmdlin clocksource=tsc net.ifnames=0

Build

cargo build

Burn

cargo run -- -w disk.img

Test

cargo run -- -e disk.img

Examples

Examples for x86-64 Linux is described in the docs of 'Creating Custom Linux Images and Booting'.

Contributing

KRaBs welcomes all contributions. To contribute to KRaBs, check out the getting started guide and then the KRaBs contribution guidelines.

Design

KRaBs's overall architecture is described in the design document and the specification document.

Features

1. Supports legacy BIOS.
2. Supported media are HDD and SSD which have GPT.
3. Supports 32bit protected mode and 64bit long mode.
4. Supports OS kernel formatted in ELF32/ELF64.
5. Supports minimal x86/x86_64 linux boot protocol.
6. KRaBs interprets the FAT32 file system.
7. KRaBs's boot optuon is configureset by CONFIG.TXT on the FAT32 file system.
8. KRaBs can load modules such as initramsfs/initrd according to linux boot protocol.
9. KRaBs can transmit kernel command line to the kernel according to linux boot protocol.

ToDO

1. support gziped kernel.
2. support recovery mode.

License

This project is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Serde by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Contact

You can get in touch with me in the following ways:

• Contact me on my twitter.
• Open a GitHub issue in this repository.
• Email me at [email protected].

Note: I'm on a Japan time zone.

When communicating within the Krabs community, please mind our code of conduct.

krabs's Issues

The printf command used in build.sh has a compatibility issue.

Describe the bug
The printf command used in build.sh has a compatibility issue.
In POSIX, Hexadecimal escaping can be ambiguous. This causes build.sh to write fatally incorrect values ​​when writing parameters in build process.
As a result, the following error occurs during the boot step:

Booting from Hard Disk...
Stage1: I/O error

Dash printf builtin does not honor \xNN to print hexadecimal:

# in dash
$ printf '\x41'
\x41

Bash printf builtin:

# in bash
$ printf '\x41'
A

How to fix
According to POSIX, we have to use octal escaping when we want to output bytes with the printf command.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html

"\0ddd", where ddd is a zero, one, two, or three-digit octal number that shall be converted to a byte with the numeric value specified by the octal number

unrecognized command line option '--target=i586-unknown-none'

Description
I'm trying to test krabs by following instruction.
https://github.com/ellbrid/krabs/blob/master/docs/linux-image-setup-64.md

But I've got the following error.

== Building stage_4. ==
~/Documents/krabs/src/stage_4th ~/Documents/krabs
    Updating crates.io index
   Compiling compiler_builtins v0.1.25
   Compiling core v0.0.0 (~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore)
   Compiling rustc-std-workspace-core v1.99.0 (~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/tools/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/tmp/xargo.CBbDiRhAkFj9)
    Finished release [optimized] target(s) in 9.22s
   Compiling cc v1.0.50
   Compiling plankton v0.1.0 (~/Documents/krabs/src/plankton)
   Compiling stage_4th v0.1.0 (~/Documents/krabs/src/stage_4th)
error: failed to run custom build command for `stage_4th v0.1.0 (~/Documents/krabs/src/stage_4th)`

Caused by:
  process didn't exit successfully: `~/Documents/krabs/target/release/build/stage_4th-78eda19270ba321b/build-script-build` (exit code: 1)
--- stdout
TARGET = Some("i586-stage_4th")
OPT_LEVEL = Some("z")
HOST = Some("x86_64-unknown-linux-gnu")
CC_i586-stage_4th = None
CC_i586_stage_4th = None
TARGET_CC = None
CC = None
CROSS_COMPILE = None
CFLAGS_i586-stage_4th = None
CFLAGS_i586_stage_4th = None
TARGET_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CARGO_CFG_TARGET_FEATURE = None
running: "cc" "-Os" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m32" "-Wall" "-Wextra" "-m32" "--target=i586-unknown-none" "-fno-common" "-mno-sse" "-fno-builtin" "-ffreestanding" "-o" "~/Documents/krabs/target/i586-stage_4th/release/build/stage_4th-b871f005902daf44/out/c_src/huffman.o" "-c" "c_src/huffman.c"
cargo:warning=cc: error: unrecognized command line option '--target=i586-unknown-none'
exit code: 1

Environment

$ uname -a
Linux zdm 5.3.0-28-generic #30~18.04.1-Ubuntu SMP Fri Jan 17 06:14:09 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ gcc --version
gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ rustc --version
rustc 1.43.0-nightly (433aae93e 2020-02-14)

Question
Does anyone have knowledge about the error message?

GPT & FAT support

Is your feature request related to a problem? Please describe.

1. MBR has a 2TB limit and linked-list logical partitions, which is not a good choice for scalability and usability.
2. Writing an OS image directly to disk is not a good idea from an administrative standpoint because it can destroy everything.

Describe the solution you'd like

1. Switch from MBR support to GPT support.
2. Instead of writing the OS image directly to disk, it should be stored in a FAT32 file system. It is an important step in gaining UEFI compatibility in the future.

Polish translating

Is possible add localisation (multilanguage) version krabs?
Trouble is with different chars or graphics

gzip support

Use https://github.com/chalharu/rust-compression

kernel command line is not working.

Describe the bug
The transmission function of the kernel command line is not working well.

To Reproduce
Steps to reproduce the behavior:

1. build

   ./tools/build.sh -k vmlinux -i initramfs.cpio.gz -c "console=ttyS0" disk.img 

2. cat /proc/cmdline is empty

Write support for FAT32

KRaBs supports the FAT32 filesystem, but only the read feature is available. To add the recovery mode feature #18 , we need to implement write() for FAT32 module as well.

Moving from llvm_asm! to asm! macro

Although a new inline assembly syntax asm! was announced in Rust on 2020/06/08[1], KRaBs still uses llvm_am!.

[1] New inline assembly syntax available in nightly

According to the blog, the asm! macro should be safer and easier to use. This is expected to become the mainstream of inline assembly in the future, so KRaBs will have to transition to using asm! as well.
All llvm_asm! should be rewritten to the new asm! if it is rewritable.

The new inline assembly writing method is described in [2].

[2] rfcs/0000-inline-asm.md at inline-asm · Amanieu/rfcs

LZ4 Support

bzip2 can be slow in a virtual environment, so try LZ4.

UEFI Support

Create a sub-crate src/urfi to support the UEFI environment.

Issue on BochsBIOS

Describe the bug
Stage4 hangs when using BochsBIOS.
There is probably a problem with the process of transitioning to real mode to do I/O.

OOM error

Description
I'm testing krabs bootloader by following instruction.
https://github.com/ellbrid/krabs/blob/master/docs/linux-image-setup-64.md

When I launch a vm with qemu, I got OOM errors as follows:

26790 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
32638 pages RAM
0 pages HighMem/MovableOnly
3977 pages reserved
Tasks state (memory values in pages):
[  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Out of memory and no killable processes...
Kernel panic - not syncing: System is deadlocked on memory
---[ end Kernel panic - not syncing: System is deadlocked on memory ]---

How to Fix & My Verification
The error message looks life out of memory.
So I change memory option of qemu and launch it successfully:

qemu-system-x86_64 --hda disk.img -m 1G

The default value of memory size is 128 MB according to following document.
https://wiki.gentoo.org/wiki/QEMU/Options

I tested 256M and 512M, but it didn't work.
So specifying memory size explicitly is better.

Recovery mode

Is your feature request related to a problem? Please describe.
Currently, Krabs does not have a way to recover from a boot failure, so it needs a reasonable recovery option that can be also used in a cloud environment like AWS EC2.

Describe the solution you'd like
For implementing Recovery Option, we are going to use a what should be called a startup success marker. Krabs uses this special marker by embedding it in the free space on the disk.

Then the following mechanism is used to detect startup failures and implement the fall back option.

1. After the OS start up succeeded, OS have to embed the success marker into disk with a service (like systemd services, etc).
2. If this mark is not present at the next time booting, bootloader judges that the OS failed to boot last time and tries to boot with another one.
3. Bootloader erase this marker before processing from the bootloader to the kernel.

This would make it possible to implement a simple recovery mode in Krabs that can be recovered by simply rebooting in a cloud environment.

Separate build script into build part and writer part.

Currently, build.sh is writing the image to the device node in addition to the build, so it is not easy to use. Separate this from write operations because we don't want to build with sudo.

Not present the booting view of disk.img

Describe the bug
I have compiled the source code and prepared the disk.img. The partitation for disk.img has completed.
But when I run cargo run -- -w disk.img -e,the result is:

star@node:~/krabs$ cargo run -- -w disk.img -e
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
     Running `target/debug/krabs -w disk.img -e`
Checking the first [4] stage(s) build...
    Build exists. Builds { stage_1st: Some("/home/star/krabs/target/stage_1st/i586-stage_1st/release/stage_1st.bin"), stage_2nd: Some("/home/star/krabs/target/stage_2nd/i586-stage_2nd/release/stage_2nd.bin"), stage_3rd: Some("/home/star/krabs/target/stage_3rd/i586-stage_3rd/release/stage_3rd.bin"), stage_4th: Some("/home/star/krabs/target/stage_4th/i586-stage_4th/release/stage_4th.bin") }
Checking partitions...
    BIOS boot partition: start = 2048, end = 4095
    EFI System Partition: start = 4096, end = 106495
Burn each stage to disk....
    stage_1st...  done.
    stage_2nd...  done
    stage_3rd...  done
    stage_4th...  done
Try qemu...

There is no booting view of the disk.img?

Desktop (please complete the following information):

• OS: Ubuntu 20.04.1 LTS
  What is the reason?