jakubgawron1/euralios

A hobby x86-64 operating system written in Rust

EuraliOS

A hobby x86-64 operating system written in Rust. Named for Euralia in A.A. Milne’s Once on a Time because I don’t know who this is intended for except, to borrow Milne’s introduction, “For those, young or old, who like the things which I like… Either you will enjoy it, or you won’t. It is that sort of book project.”

Disclaimer: This was written by someone who doesn’t (or didn’t) know anything about Rust or operating systems. The result is probably not idiomatic Rust or a sensible OS design. If you are looking for a usable operating system implemented in Rust then Redox OS is probably what you want.

This started with Philipp Oppermann’s amazing blog on OS development with Rust (second edition), and makes use of many of the crates that Phil maintains (e.g. x86_64 and bootloader). That blog has inspired many other operating system siblings that I’ve found useful to study including ananos, the Stupid Operating System, intermezzOS, and especially MOROS the Obscure Rust Operating System. The Redox OS kernel is very readable, and Xous has a similar focus on message passing (This is a more comprehensive comparison of Rust operating systems). Anything useful here was probably borrowed from one of these projects; mistakes and omissions are my own.

Features

There are many different directions to explore in building an OS, and with EuraliOS I’ve focussed mainly on memory management and pre-emptive multitasking so far. The result is an OS that can’t do much, but it can do it all at the same time.

• The “Merriwig” kernel
  • [X] Pre-emptive multitasking of user and kernel threads, with a round-robin scheduler
  • [X] Program memory isolation using paging
  • [X] On-demand memory allocation using page fault handler
  • [X] Stack and heap memory management
  • [X] Frame allocator based on a bitmap tree structure
  • [X] ELF loader using the object crate
  • [X] Rendezvous message passing for all inter-process communication. Messages can contain communication handles or transfer ownership of memory regions between processes and threads.
  • [X] 18 syscalls for memory and process/thread control and inter-process communication
  • [X] A vDSO-like fast mechanism for access to timing information from user programs
  • [X] Virtual file systems that can be shared between processes or per-process as a mechanism for capability-based security and user isolation
• Drivers as userspace programs (Ring 3)
  • [X] Keyboard driver using the pc-keyboard crate
  • [X] Virtual consoles and VGA text output using the vga crate
  • [X] RTL8139 network card driver
  • [X] TCP network stack using smoltcp in user space, with DHCP and DNS
• User programs
  • [X] Login process and multiple users
  • [X] A basic shell
  • [X] A simple Gopher browser
  • [X] A RAM disk for temporary files
  • [ ] Port of the Kibi text editor
  • [ ] Virtio 9P to access host filesystems

There is no disk driver yet, so user programs are compiled and then the ELF files are included in the init binary using Rust’s include_bytes macro (another idea taken from MOROS).

Building and running

Euralios depends on the vga crate, with some modifications in a branch here as a git submodule. Either clone with submodules (with e.g the --recurse-submodules option) or run:

$ git submodule update --init --recursive

After setting up Rust, Cargo and QEMU (see phil-opp blog), running

$ make run

should download dependencies, build everything, and launch qemu.

Documentation

I’ve tried to document the steps to build EuraliOS, starting from the end of post 10 (heap allocation) of Phil Opp’s blog (2nd Ed). At that point the kernel had a heap and a bump frame allocator (so could allocate but not free memory), some code to handle pages, exceptions, and VGA output. Post 11 went on to add Async/Await cooperative multitasking, but here we go in a slightly different direction…

• 1. Interrupts and processes in which kernel multi-threading is introduced by modifying the timer interrupt handler to capture and change the context (registers etc).
• 2. Userspace in which a program is loaded into memory and run in Ring 3 (user space). Syscall/sysret is used to implement a simple way to print from user programs.
• 3. Memory management in which we create separate page tables for user processes and stacks for each thread.
• 4. Syscalls for thread control which adds syscalls to spawn new threads and to exit from a thread. We also learn how to use the swapgs and wrmsr instructions to switch stacks inside syscalls.
• 5. Memory returns which implements a better frame allocator that can keep track of available frames, so that memory can be freed when threads and processes exit.
• 6. User space memory management in which we create a heap for each user process so that user programs can use Box, Vec etc, using a linked list allocator. As a bonus this allows a threading API with closures as in the Rust stdlib. We also re-organise the repository into a Cargo Workspace, with separate crates for the kernel and user program.
• 7. Inter-process communication (IPC) where a simple ”rendezvous” communication method is implemented, enabling a user program to get input from the keyboard.
• 8. Faster IPC in which we switch tasks in syscalls and keyboard interrupt handler, to minimise delays in the communication.
• 9. Sending messages which adds the send syscall, allowing the user program to send messages to a VGA output kernel thread.
• 10. A standard library in which we start a euralios_std “standard” library, moving code out of the user program into a separate crate.
• 11. More messages
• 12. PCI devices where we start working on input/output to PCI devices and discovering which devices are available.
• 13. Return to sender where we add the send_receive syscall to make remote procedure calls more reliable, and start to develop a Virtual File System (VFS) with a new open syscall.
• 14. RTL8139 network card where we develop a basic driver for the RTL8139 card, adapting the MOROS driver. In the process we wrap the send_receive system call into an rcall remote procedure call, and add frame allocation of consecutive physical frames for direct memory access.
• 15. Message error handling where we add error handling and send retries to make messaging more robust. We also add a thread_yield system call to yield control of the processor when waiting and recovering from errors.
• 16. Address Resolution Protocol implementation: Writing a simple program to send and receive an ARP packet through the network.
• 17. TCP stack where the smoltcp crate is used to provide a TCP stack in user-space which communicates with the network card driver by messaging.
• 18. Gopher: Developing a simple Gopher protocol browser, and in the process improving the Virtual File System (VFS) to handle more complicated OPEN messages, and the TCP program to READ and WRITE sockets.
• 19. Timing: Adding functions to get time since system start. To speed up access to timer calibration data we map a page read-only into every user program, as the Linux virtual dynamic shared object (vDSO) does.
• 20. Domain Name System (DNS), adding the ability to look up IP addresses from host names.
• 21. VGA driver and terminals in user space, using the vga crate. Allows separate consoles for system programs and user programs like the Gopher browser.
• 22. RAMdisk driver to store files and develop the filesystem API, starting a basic interactive shell able to list files and run programs.
• 23. Interrupts and a better keyboard: Enabling user programs to receive hardware interrupts, and moving the keyboard driver out of the kernel into a user-space driver.
• 24. Text editor. Porting the Kibi text editor to run on EuraliOS.
• 25. Directories to hierarchically organise files
• 26. Multiple users and login process, using separate virtual file systems to control user capabilities

Notes

• A shell
• USB (xHCI)
• WiFi

Bibliography

Useful reference material includes:

• The OSDev.org wiki
• AMD64 Architecture Programmer’s Manual (Vol 2: System programming)
• Linux insides by @0xAX
• The Linux Kernel documentation

Other relevant links

• The Adventures of OS: Making a RISC-V Operating System using Rust
• The resea micro-kernel based OS written in C
• The kerla monolithic kernel written in Rust
• The “Tifflin” kernel written in Rust
• Poplar operating system in Rust