s-matyukevich/raspberry-pi-os

Learning operating system development using Linux kernel and Raspberry Pi

Learning operating system development using Linux kernel and Raspberry Pi

This repository contains a step-by-step guide that teaches how to create a simple operating system (OS) kernel from scratch. I call this OS Raspberry Pi OS or just RPi OS. The RPi OS source code is largely based on Linux kernel, but the OS has very limited functionality and supports only Raspberry PI 3.

Each lesson is designed in such a way that it first explains how some kernel feature is implemented in the RPi OS, and then it tries to demonstrate how the same functionality works in the Linux kernel. Each lesson has a corresponding folder in the src directory, which contains a snapshot of the OS source code at the time when the lesson had just been completed. This allows the introduction of new concepts gracefully and helps readers to follow the evolution of the RPi OS. Understanding this guide doesn't require any specific OS development skills.

For more information about project goals and history, please read the Introduction. The project is still under active development, if you are willing to participate - please read the Contribution guide.

Table of Contents

• Introduction
• Contribution guide
• Prerequisites
• Lesson 1: Kernel Initialization
  • 1.1 Introducing RPi OS, or bare metal "Hello, world!"
  • Linux
    • 1.2 Project structure
    • 1.3 Kernel build system
    • 1.4 Startup sequence
  • 1.5 Exercises
• Lesson 2: Processor initialization
  • 2.1 RPi OS
  • 2.2 Linux
  • 2.3 Exercises
• Lesson 3: Interrupt handling
  • 3.1 RPi OS
  • Linux
    • 3.2 Low level exception handling
    • 3.3 Interrupt controllers
    • 3.4 Timers
  • 3.5 Exercises
• Lesson 4: Process scheduler
  • 4.1 RPi OS
  • Linux
    • 4.2 Scheduler basic structures
    • 4.3 Forking a task
    • 4.4 Scheduler
  • 4.5 Exercises
• Lesson 5: User processes and system calls
  • 5.1 RPi OS
  • 5.2 Linux
  • 5.3 Exercises
• Lesson 6: Virtual memory management
  • 6.1 RPi OS
  • 6.2 Linux (In progress)
  • 6.3 Exercises
• Lesson 7: Signals and interrupt waiting (To be done)
• Lesson 8: File systems (To be done)
• Lesson 9: Executable files (ELF) (To be done)
• Lesson 10: Drivers (To be done)
• Lesson 11: Networking (To be done)

raspberry-pi-os's Issues

Bug in function free_page

The current version is:

void free_page(unsigned long p){
	mem_map[p / PAGE_SIZE] = 0;
}

I guess you forgot to subtract LOW_MEMORY here. It should be:

void free_page(unsigned long p){
	mem_map[(p - LOW_MEMORY) / PAGE_SIZE] = 0;
}

Adapt design into RTOS?

I've recently took a course about Real Time Operating Systems, and was wondering, how difficult would it be to adapt the current OS design into a bare-bones RTOS?

Need to know the ARM version to understand sctlr_el*?

I was working on the exercises for lesson 2. I notices that different arm v8 versions (v8.0, v8.1, v8.2, v8.3) have different reserve bits. From looking at the ARMv8 documentation, I think I can deduce the that the PI is arm v8.0 based on the values that are set to 1 in the example. I though it would be helpful to point this out when mentioned the link to the ARM documentation so that one could correctly interpret the table. I did try to look up the ARM version of a Raspberry PI 3+, but I didn't find any clear documentation.

Could you add a blurb in the "Switching to EL1" section stating that the ARM version is 8.0? (assuming I've got that right)?

Thanks,
Paul

Questions about Mini UART in lesson 1

I've successfully built the first lesson and run it on my Raspberry PI. But as I was reading through, two questions related to the UART configuration came up. Perhaps they can be clarified in the doc.

1. The lesson states that we want to use the Mini UART and that pins 14 and 15 have the TDX1 and RDX1 functions. So after doing the lesson I think I understand that TDX0 maps to the PL11 UART and TDX1 maps to the Mini UART, but this was not clear when I read through the first time. Also the Broadcom documentation doesn't explicitly state this either.

2. Related to #1, it mentions that "This means that if we select alternative function 5, they will use Mini UART." But the code shows that a 2 is moved into GPFSEL1, which I thought would set alt2. Instead I expect a 5 to select alt5. Assuming I'm missing something, because 2 does seem to work, I think it an explanation why 2 is set instead of 5 would make it clearer.

I think this is such a great resource, and agree completely with your introduction. Having a simple starting place to understand how an OS works is extremely useful to me. Thanks for all the work so far, and look forward to the current and future lessons.

Thanks,

Paul

Put non-primary cores to sleep instead of endless loop

I just finished lesson01 and was wondering if the proc_hang endless loop that all non-primary cores go into wouldn't put 100% load on these three cores?

I was looking into the ARM documentation trying to find an instruction that basically disables a core and found wfe (wait for event) which will completely disable the clock for that core until an interrupt occurs. I think that is a bit nicer than a loop.

So I suggest:

proc_hang: 
	wfe
	b 	proc_hang

The mini uart get some something invisible chars before I send.

In the main while loop,we send char back which receive from host.
But I find that uart already got some chars before I send.

How to get the value of UARTCLK used by UART

In lesson1 exercise 2,I find the way to caculate the baudrate like below,
BAUDDIV = (FUARTCLK/(16 Baud rate))
but I cannot find the value of FUARTCLK,can anybody help to tell me where can I find the origin about FUARTCLK.
Thank you !

Lesson 1 should add instructions on how to build a cross-compiler

I keep running into an error with locating pthread.h while building gcc cross-compiler for the target from lesson 1 (aarch64-linux-gnu) even though I've successfully built x86 and x64 compilers before. I think I'm not the only one who faced this problem and would love a detailed guide from the author himself.

P.S. I followed this guide/

getting error while running ./build.sh or ./build.bat docker command not found

Check whether the RPi OS can be successfully compiled and connected from Mac OS and Windows

Run RPi OS on qemu

Check whether the RPi OS works on Raspberry Pi 3 Model B+

I expect everything to work fine, though I don't have this device.

Printing to real screen

I was trying to implement printing to real screen using the already made code found on this repo: https://jsandler18.github.io/tutorial/hdmi.html maybe someone can help me with this!

Kernel code reading environment

Hey I'm wondering is there any recommendations for reading (editing) kernel source code.

I know this is a personal preference thing. But I still would like to get some opinions.

I have been using emacs for the last two years in terminal. C/S mode is making it super fast which is better than vim. Clion is amazing but sadly it doesn't support Makefile.

I know a Android kernel developer. His kernel is good(at least I think so) and he's productive(based on commit history). And he told me that he uses vscode.
Hum... I don't know... vscode doesn't even have built-in asm highlighting.

I switched to vim from spacemacs a month ago. But frankly they are the same in general that they rely on gtags and cscope. And I feel lag reading and jumping around function calls.

What about QT Creator? I haven't tried it.

[Lesson03] qemu Timer question

The timer is not working in qemu.
The code works perfectly on the hardware but on the qemu, after some "debug", I realized that the TIMER_CLO remain zero.

Anyone had this issue?

Thanks

Page numbers do not match up with Architecture Ref Manual

In Lesson 2, the lesson text and header file sysregs.h reference the AArch64-Reference-Manual. The page numbers do not line up with the latest revision (C.a) of the manual that is linked to in the text of the lesson.

For example:

// ***************************************
// SCTLR_EL1, System Control Register (EL1), Page 2025 of AArch64-Reference-Manual.
// ***************************************

When I download this PDF, Page 2025 is section D2.10.5 Determining the memory location that caused a Watchpoint exception. SCTLR_EL1 is described on page 2654, which is section D10.2.100 SCTLR_EL1, System Control Register (EL1)

The other page references in the header file and lesson docs similarly do not line up.

Lessons grammar and style validation

I am not a native English speaker. I did my best to verify the lessons, though they still can contain a lot of grammar mistakes. If you want to help the project, you can just select any lesson or chapter and submit a PR with the changes.

Data abort in show_invalid_entry_message

The entry_error_messages array is missing the messages for error 15 and 16 (SYNC_ERROR and SYSCALL_ERROR respectively) which causes a data abort when trying to access them in the show_invalid_entry_message function.

To reproduce:

• Use code from lesson05
• Comment out line 162 in entry.S

Typo in the Table of Contents

"Prerequsities" instead of "Prerequisites "

Honestly "Prerequsities" sounds great, but I doubt it was the initial intention of the typo.

[Lesson06] LSR instruction

There is a mistake in the explanation of this snippet of code for the macro create_table_entry:

   lsr    \tmp1, \virt, #\shift
    and    \tmp1, \tmp1, #PTRS_PER_TABLE - 1            // table index

The instruction LSR stands for logical shift right, not left as it is explained in the text? http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0497a/BABJCCDH.html

Raspberry Pi Zero support

Hi,

Is it possible to work with your OS under Raspberry Pi Zero?

Thanks,
Vasyl

Broken links on caches and multi-core processors in lesson02/linux.md

Exactly what the title says. Both the links seem to have moved. The caches one has moved to https://developer.arm.com/docs/den0024/latest/caches/cache-maintenance, I think. The multi-processor one, I don't know where it's supposed to point to, so I didn't make a PR.

lesson 1: mismatch in uart_init()

Hello,

in the code snippet for Mini UART initialization we have line setting register AUX_MU_IIR_REG:

put32(AUX_MU_IIR_REG,6); //Clear FIFO

Although, it is not described, what it does and it is not present in the source code for lesson 1.
Should it persist in the code (and what it does?) or should it be dropped?

Thank you in advance.

Work on the exercises.

You can submit a PR as soon as an exercise is ready. Other people then will be able to use your code for reference.

Prepare Linux specific chapters for lesson 6

controle ms time with precision GUI pyqt4 raspberry pi 3

hi someone can help me i need get time with precision ms with python (GUI pyqt4 interface ) in my raspberry PI3
my probleme i have random error if i test 20ms i have 20ms , 19ms , 15ms, 21ms :((((((

my code test is :

i was tested with thread and i have same error !!!!!!!!
i can use hardware timer with python ????????????????????

Baud rate and baud rate register calculation

In Introducing RPi OS, or bare-metal "Hello, World!" the formula given to calculate baud rate is:

baudrate = system_clock_freq / (8 * ( baudrate_reg + 1 ))

The system_clock_freq is 250MHz, and using 270 for the baudrate_reg, the value given is ~115314, and not 115200. If I open a screen using 115314 for the baud rate, I get gibberish.

Why is that? How can I calculate correctly the baudrate_reg given any baudrate value for exercise 1, lesson 01?

Thanks

Will this tutorial work on Raspberry Pi 4?

I've read the introduction of this tutorial and I really think this is what I want to start my journey of Linux kernel. Thank you for this material.
For now, the 4th gen of raspberry pi is out so I am not sure if this repo works on the new Pi (whose SoC is Broadcom BCM2711B0 quad-core A72 (ARMv8-A) 64-bit @ 1.5GHz).
If it won't work, I decide to buy the one mentioned in the Introduction.

Confusion in lesson 4

Here say that LOW MEMORY is 2 MB and HIGH MEMORY is 1GB - 1MB

We know the total amount of memory in the system. It is 1 GB - 1 MB (the last megabyte of memory is reserved for device registers) This value is stored in the HIGH_MEMORY constant.

First 2 MB of memory are reserved for the kernel image and init task stack. This value is stored in LOW_MEMORY constant. All memory allocations start right after this point.

But here, LOW MEMORY is said to be 4 MB.

The kernel is initialized and kernel_main function is executed. The initial stack is configured to start at LOW_MEMORY, which is 4 MB.

Is that just a slip of the pen or I get something wrong?

How could I boot my pi?

I've built the lesson01 project and got these files, but I don't know how to do next.

I got a problem about using docker to build

Hello!
I‘m using dockertools in Winsows10 .
the command in my build.bat is
docker run --rm -v %cd%:/app -w /app smatyukevich/raspberry-pi-os-builder make %1
I think it's just the same as the command in the repo. But everytime I try to run ./build.bat in powershell, it would say docker: Error response from daemon: invalid mode: /app. See 'docker run --help'. I've already search in the internet and still can't work it out.
Could anybody help me out?
Thanks a lot!

[Lesson03] Exercises questions

I have 2 questions about the exercises of Lesson03 that I'm currently trying to complete, respectively exercise 1 and 3.

1. Is the local timer the Timer (ARM side) of BCM2835 Peripherals manual p.196 ?

2. Is the ARM generic timer mentioned here the one mentioned in my previous question or The Generic Timer in AArch64 state of Aarch64-Reference-Manual p. 2321 ?

Why are we compiling assembly files using GCC instead of GAS?

Hi, wanted to know what's the reason in compiling .S files using GCC rather than GAS. Is it because of "-MMD" flag?

P. S. I think there should be some kind of IRC or Discord channel to ask these kinds of questions instead of polluting the "Issues".

Use kernel8.img naming convention

Pi 3 supports the kernel8.img naming convention. Here

I think by using kernel8.img, we can drop the arm_control flag (I drop it, turns out ok).
After all we have ARMv8. And arm_control is actually undocumented.

RPI 3 with ARM v8 Boots into EL2 not EL3

When booting to Pi for lesson 2, it begins in EL2 not EL3. I believe this was a change. The RPI firmware now changes initial boot from EL3 to EL2.

UART boot

Приветствую!

Did you think about UART booting to reduce wear of SD card, and overall development speedup, at least for development/debugging period?

When some new lessons will delight us?

Raspberry Pi 2?

Is there anyway that this can work on the Raspberry Pi 2B?
Could I run it on qemu?

[Feature Request] Create a git pages site

This is a feature request/suggestion.
The layout for this git repo is in prime shape to be setup as a git pages website. This would make viewing the lessons easier as the text would fill the screen.

Question about forking user process

I'm reading lesson05, I can build the kernel and run it on my PI. It works as expected, two user task are started and I can get different message from them. But I have a question.

When forking a user process, in function copy_process, we need allocate kernel stack for new task, and copy current task's pt_regs to new task's pt_regs. cur_regs point to current task's pt_regs and childregs point to new task's pt_regs. But why we use *cur_regs = *childregs; ,rather than *childregs = *cur_regs; ?

Thanks for this tutorial, I think it is much interesting than the OS class in college. I really control the hardware and make it work rather than talking about many algorithms.

Question about lesson 2 exception level

you mentioned "when we enter el1_entry function the execution should be already at EL1 mode.", but

int el = get_el(); printf("Exception level: %d \r\n", el);

the code shows it is exception level 3, seems self contradiction,

also, without calling "svc", how did you change the exception level from 3 to 1?

thanks in advance!

Implement source code for lesson 7

(Lesson 1) Why use 0x3F000000 as base address?

It would be great if you could add this kind of explanation to the README.md, as the manual-page link that you referred us to was showing a different base address (0x7E000000), and it didn't really click for me what was going on for a while... Here was a great/helpful analysis/explanation of what's going on, in simple: https://www.raspberrypi.org/forums/viewtopic.php?t=142439 .

[Lesson01] Base address computation

I'm trying to figure out the computation behind the given base address (in https://github.com/s-matyukevich/raspberry-pi-os/blob/master/src/lesson01/include/peripherals/base.h). Can someone explain how to go from "The Raspberry Pi 3 reserves the memory above address 0x3F000000 for devices" and "AUXENB Register (0x7E21 5004)" to "0x3F00 0000 + 0x0021 5004" ? If my research are correct, the Rapsberry Pi 2 use 0x20xx xxxx as the base for physical addresses and 0x3Fxx xxxx for the Raspberry Pi 3. It is specified in the BCM2835 manual that you need to go from the bus address (which is based at 0x7Exx xxxx) to the physical address but maybe it is worth mentioning in the course ?
Also, why we can't use RXD0 and TXD0 for GPIO seems a bit obscure for me. I found in the documentation that we have UART0 (PL011 UART) and UART1 (Mini UART from Auxiliaries). IMHO, it could be nice to mention it in the course (maybe something like "we are using RX/TX D1 because instead of RX/TX D0 that are mapped to the other UART) and maybe give a reference to https://www.raspberrypi.org/documentation/configuration/uart.md for basic differences ?

get more cheaper and better respberry pi

https://www.amazon.com/gp/product/B07BFH96M3/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B07BFH96M3&linkCode=as2&tag=a294p4x9ewv02-20&linkId=294b1e6b1bb324290f0586f3653f0e04

Linking Documentation Pages

Hey, I'm a newbie in C and OS development. I found this project so interesting and informative to learn C effectively and it's a great way to learn Linux kernel, OS development and Raspberry Pi at the same time! So I'm going to have a few contributions time to time, but I'll be staring from documentation for now.
I wish I'll be able to contribute in code in future soon.

Anyway, as I was reading docs, I made it easier to move through documentation pages for myself by setting "next page" and "previous page" at the end of each doc file to link all pages. readers would continuously have access to next and previous pages easier than before then.

I though this linking feature might be useful enough to make it a pull request, is it?

Minor README correction

In the spirit of issue #3 I propose changing "Table of Content" to "Table of Contents" in the README.

Typo on: 1.4: Linux startup sequence

In the section Device Tree in the seventh paragraph:

A device tree definition consists of the blocks nested one in another. At the top level we usually can find such blocks as cpus or memory The meaning of those blocks should be quite self-explanatory. Another interesting top-level element that we can find in the bcm283x.dtsi is sos that means System on a chip It tells us that all peripheral devices are directly mapped to some memory area via memory mapped registers. soc element serves as a parent element for all peripheral devices. One of its children is gpio element. This element defines reg = <0x7e200000 0xb4> property that tells us that GPIO memory mapped registers are located in the [0x7e200000 : 0x7e2000b4] region. One of the childern of gpio element has the following definition

sos should be SoC as Wikipedia points out:

A system on a chip or system on chip (SoC or SOC)

Typo in Introduction

Under "How Raspberry Pi comes into play"

I already mentioned that I don't want the PRi OS to support multiple computer architectures and even a lot of different devices.

PRi => RPi

Revised BCM2837(5)-ARM-Peripherals

There is a homemade revised version of BCM2837 peripherals. If you don't want to check errata page all the time.

Here is the original post
And the download link

btw, I don't know why the download link is linked to rpi official repo.

Lesson 1.1 explains the wrong ARM instruction lrd vs. adr

Under the Booting the Kernel header, the ARM instruction lrd is explained when the adr instruction is used instead. This one was confusing for me as someone familiar with asm but not with ARM.

adr (PC-relative) seems to be the correct instruction.

It might be nice to link every instruction to the proper URL as well. I'm happy to fork this and fix it!