josehu07/hux-kernel

The Hux x86 32-bit Toy Operating System Kernel (with full wiki pages)

The Hux Kernel

Hux - An x86 32-bit single-CPU toy operating system kernel built from scratch, following the OSTEP book structure and terminology.

Author	Kernel Src LoC (temp)	Tutorial LoC (temp)
Guanzhou Hu	C + x86 ASM	Markdown
Jan. 2022	8160	10062

Tutorial / Development Doc

I document the whole development process of Hux as a complete set of tutorials. They can be found at the WIKI pages 📝 of this GitHub repository ✭.

If there are any typos / mistakes / errors, please raise an issue!

Playing with Hux

Requires a Linux host development environment. Tested on Ubuntu Xenial, Bionic, & Focal.

Clone the repo, set up a development cross-compilation toolchain following this wiki page, then build Hux by:

$ make clean
$ make

Or, if you just want to try out Hux without a development toolchain, download both the released kernel image hux.iso and the initial file system image vsfs.img (256MB) to the folder.

To run Hux in QEMU >= v6.2.0, install QEMU & GRUB following this wiki section, then do:

$ make qemu     # opens a VGA GUI window

If you are in a non-GUI environment, it is recommended to redirect VGA output to built-in VNC server, and connect to that server from a VNC client:

$ make qemu_vnc     # redirects VGA output to VNC server
                    # from VNC client, connect to 'hostname:5901'

You will see the QEMU GUI popping up with GRUB loaded. Choose the "Hux" option with Enter to boot into Hux.

For development setup & instructions, please check out the wiki pages (recommended). I have every single detail documented there.

Goals

The main goal of Hux is to be Understandable: structured in a way that is easy to understand (not mimicking existing UNIX-like systems). OS development seems scary at first glance for beginners mostly because it involves too many hairy technical details. I admit that, in real-world systems, we must face the complexity to ensure compatibility, performance, robustness, security, etc. Yet, a toy kernel project would help demonstrate the key concepts of an operating system, including its most essential steps of development, layers of abstractions, and the ideas of virtualization, concurrency, and persistence.

Goals of the Hux kernel include:

1. Understandability: as stated above
2. Minimalism: a minimal workable core design
3. Code clarity: though monolithic kernel, I will try to keep the code structure modularized
4. Experimentalism: not strictly following UNIX flavor, not targeting at practical use

I choose to write it in C language with i386-IA32 architecture, since beginners tend to be more comfortable with this combination. More up-to-date system programming languages like Rust are great choices for modern 64-bit OS dev (Philipp is making his Rust OS kernel here), but I will start with easier settings for now to maintain better understandability. Rust itself is still "niche" (maybe not?) and you have to incorporate some of its "dark magics" to succeed in OS dev. It definitely confuses new learners.

These are general and long-term goals which I will (hopefully) follow throughout the project. I hope this can lead towards a full HuxOS which we can install on real devices and play with in the future (kept simple, of course 😁)

References

Main references:

• The OSDev Wiki (IMPORTANT ✭)
• JamesM's Kernel Tutorial by James, along with its known bugs
• Writing an OS in Rust by Philipp (still updating...)
• The xv6 Kernel x86 ver. by MIT PDOS (no longer updated)

OS conceptual materials:

• Operating Systems: Three Easy Pieces (OSTEP) by Prof. Arpaci-Dusseaus
• My reading notes on OSTEP book & lectures

Check the "References" section here for the full list.

TODO List

• The basic kernel skeleton
• VGA text mode driver
• Debugging utilities stack
• Interrupts & timer
• Keyboard input support
• Global descriptors table
• Virtual memory (paging)
• Heap memory allocators
• Process user mode execution
• Essential system calls
• Time-sharing scheduler
• Basic IDE disk driver
• Very simple file system
• File system page cache
• File system crash consistency
• Support for nographics mode
• More user-level utilities
• Adding user-level tests
• Synchronization primitives
• Multi-threading concurrency
• Move to APIC & IOAPIC
• Direct memory access
• Extend Hux to Rux with Rust

hux-kernel's Issues

Wiki Paging suggestion

Hi there it's me again 👋

In https://github.com/josehu07/hux-kernel/wiki/10.-Two-Level-Paging-Setup#frame-allocation-bitmap it feels like instead of

We create an array of uint32_t's, where each integer element is essentially an array of 32 bits.

you meant

We create an array of uint8_t's, where each integer element is essentially an array of 8 bits.

as this is how struct bitmap is defined.

By the way, is it ok to provide suggestions here? Can I ping you on IRC by any chance?

Memory question

Hi there, thank you so much as I'm learning a lot from your project! I am currently tackling paging.

I have a question about this line

hux-kernel/src/memory/paging.c

Line 52 in 7fea5c5

if (kheap_curr + size > KMEM_MAX)

Something I'm not sure to understand is: while kheap_curr is initialized to a page-aligned memory address right after the kernel, which itself is loaded to 1MiB, we compare kheap_curr to something that looks like a quantity:

#define KMEM_MAX 0x00800000     /** 8MiB reserved for the kernel. */

In practice, KMEM_MAX is used as an address, and the comment should read something like: /* Max address reserved for the kernel */.

What am I missing?