flingos/flingos

An educational operating system written in C#. A great stepping stone from high to low level development.

FlingOS™

Welcome! This is the main repository for the FlingOS™ project. We used to be over on BitBucket but have shifted across to GitHub in the latter half of 2015. The FlingOS project is an educational operating system aiming to provide high-quality resources for learning OS and low-level development. You can find out more on our website over at www.flingos.co.uk. There you'll also find our documentation and links to our tutorial videos. If you're wondering why we use C#, take a look at this article.

The FlingOS project is a three part approach to teaching OS and low-level development. You're currently looking at just one part - the code itself. The code acts as a sample codebase for people to learn from and compare to. The second part is our conceptual articles, which explain all the OS and low-level technology in detail. The third part is our tutorials which are free, ~20min videos with complete resources, available on YouTube. You can find all these links and more on our main website.

Getting Started

Interested in learning OS/low-level development?

Take a look at our Getting Started article to learn how to write your own operating system.

How do I learn from the FlingOS source code?

The FlingOS source code is here for you to look at, read and compare to. By reading the FlingOS articles and taking a look at our implementations, you should be able to write your own fairly easily.

Interested in developing FlingOS?

Join the team and then setup for development.

Interested in our ahead-of-time compiler?

If you'd just like to use our ahead-of-time compiler to write your own C#, VB.Net or F# operating system, please take a look at our stable releases.

Current progress

Hopefully you'll be able to see from our codebase (structure is described here) that FlingOS has reached a reasonably advanced state. However, since October 2015, we've been working hard to transform FlingOS from a single-tasking OS to a multi-tasking OS. We've reached the final stages of the transformation, with all the significant system calls now working and stable. However, most of the changes are still contained in the develop branch. We'll let you know via our blog when the changes are merged and details of the improvements. We'll also be adding more articles on multi-tasking and blog articles about some of the challenges we've had to overcome during the transformation work.

A note on licenses

FlingOS is released under GPLv2 under UK law. This means you can't just copy and paste our code without keeping our copyright notice and you have to release your work as open-source if it includes our code. Our source code is also released without warranty and we accept no liability (within the restrictions of UK law). Please do not use our code for anything (particularly production or safety critical work) without testing and verifying it yourself.

Why did we choose GPLv2 not BSD, MIT or another, more permissive license?

FlingOS is here for people to learn from by reading and comparing. We are not here to just supply out-of-the-box code so we don't allow people to just reuse our work. Also, we are providing a learning resource not a reference sample. By restricitng the use of our code it helps to prevent the widercommunity accidentally treating us a reference codebase.

flingos's Issues

Restructure : Boot/Load : Virtual memory mapping in ASM

Improve the ASM mapping code so it maps the exact amount required not a random guess. Then remove the "unhack" comment.

PATA DMA

Add DMA support to the PATA driver so we can get faster R/W. This will also need to perform checking to see if DMA is supported.

Keyboard Autodetection

Add code for auto-detecting the correct keyboard layout and saving the setting to a file.

String get/set does not do array bounds checks

The String class in FOS_System library doesn't apply array bounds checks.

USB : Thorough testing & fixing

Thoroughly test the USB stack and fix any bugs found.

ATA : Thorough testing

Stress test the ATA code and fix any issues found.

Unit Testing and Static Code Analysis

Investigate, design and, if viable, implement Unit Testing and utilise Static Code Analysis where sensible. This needs a lot more thought and planning and may be a distraction from the main aim of FlingOS.

ISO9660 Write support

After PATAPI write support has been added, add Write support up the file system stack so that ISO9660 allows writing.

Compiler: Support more IL token types

Add support for more token types in Ldtoken implementations. This will allow/can be tested, using things like Array initialisation e.g.

New int[] { 5, 6, 7 }

Complete TODOs in ATA code

Go through and complete / resolve TODOs in the ATA code.

USB Mouse/keyboard support

After adding Isochronous Transfer Queue support to the EHCI driver, create USB keyboard and mouse drivers.

Self-installer

Implement a self-installer so that FlingOS can install itself and drivers onto real hardware.

New Debugger

Develop a totally new debugger that is more stable and with greater capability to make it actually useful. Reinvestigate Visual Studio integration.

To compliment the new drivers compiler, we need a new debugger. This means implementing the debug data features in the drivers compiler and then creating the debugger. New debugger should be designed to tackle some of the problems with the old debugger. Namely: Being able to set breakpoints Better stepping / tracking capability Faster Thread / process conscious Dual mode - one for debugging using int3s only, one for debugging using a debug process in the OS

Restructure : Main Process : Start GC Thread

The GC cleanup should be started as a thread of the main process not a separate task.

Restructure : Window Manager : Create & implement

A new Window Manager process must be created which uses system calls and pipes to allow output to the screen or a serial port.

Restructure : Boot/Load : Static heap size

Reduce the size of the static heap to the maximum potential required amount prior to the virtual memory manager being initialised. (The virtual memory manager allows the heap to auto-expand).

Restructure : Main Process : Remove non-critical interrupts thread

This was a bad idea in the first place and so it should be removed. Deferred interrupts will only be allowed for system calls and will be handled via a new, formal mechanism which allows any process to defer a system call and handle it later,

Driver Compiler - Virt Mem Nops

Investigate why the NOPs are required at the "start" of the VirtMemInit ASM plug. May be to do with the way the new drivers compiler works.

After further investigation and attempts using jump instructions between files limited success was achieved. The jump instructions allowed the OS to boot but seemed to cause failure in that the virtual memory was not / would not initialise properly.

NOPs added to the start of every initialisation assembly file. Still not sure what the real cause of this is.

Create a SATAPI driver

Implement/add a SATAPI driver.

Process startup arguments

Implement mechanism for passing arguments to the main method of a new process/thread.

Initial boot occasionally hits crashes

When you boot FlingOS, occasionally it hits an error and prints out "Cannot leave on null handler! Address: 0x????????". This needs to be traced and fixed. It may be a cross-thread/inter-thread/timing issue or a compiler issue.

Typing after scrolling up overwrites text

Typing in the Console/MainShell after scrolling up the view overwrites existing text. Need to add an option for making typing auto-reset to the actual current writing line (i.e. the last line.)

GDB Support

Add a C# GDB stub so FlingOS can be debugged using GDB. Unfortunately, GDB (nor its Mono variant) are going to have the C# support we need so we'll have to look in to gdb debugging at the ASM level.

Restructure : Main Process / Main task / All other tasks : Remove direct access to consoles

Nothing should make direct use of the console class (excluding the BasicConsole). A new console class must be created which uses pipes and system calls to write output via the Window Manager process.

Update Help command output

The output of the "Help" command in the Kernel.Core.MainShell needs to be updated to include the latest changes and to add information about default options/actions.

OHCI / xHCI support

Investigate and (if viable) implement OHCI and XHCI drivers in the USB stack.

Int64 IL op support

The following operations are not currently supported:

• Division of unsigned longs - div.un IL op not supported yet for 64-bits.
• Modulo of unsigned longs - rem.un IL op not supported yet for 64-bits.
• Division of signed longs - div IL op not supported yet for 64-bits.
• Modulo of signed longs - rem IL op not supported yet for 64-bits.

Restructure : Main Task method : Remove bus enumeration

The Main Task method should not be calling/handling bus enumeration nor device initialisation. Remove calls to InitPCI and InitATA.

Device handling will be shifted to the Device Manager process and the relevant drivers.

Complete TODOs in USB code

Go through and complete / resolve TODOs in the USB code.

Restructure : Boot/Load : Concatenate pre-req files

Concatenate and clean up the "PreReqs" and "Descriptor Tables" ASM files into a single ASM file called "LoadSequence.x86.asm".

Complete TODOs in ELF code

Go through and complete / resolve TODOs in the ELF code.

Old comments:

Work out why when you try to start a second ELF process, the whole OS starts page fault like crazy (Scheduler is trying to access invalid data or the process itself is causing a page fault?)

---

Fixed. The issue was: - The ELF loader wasn't adding the thread's stack page to the loading process's memory - So when the Segment Loading in the ELF loader called for VirtMemManager to allocate a free page, it allocated the same free page as the stack page - So same page was thus used for code and stack data (and added to the loaded process's mem layout twice. The data version taking priority.) - Thus when the mem layout was loaded by the scheduler, the stack and/or code were corrupted resulting in later page faults.
The OS was able to continue executing as the page faults only occurred when the scheduler allowed that process to run. Each page fault had no exception handler so IRet almost immediately thus causing an unhandled page fault and so immediate double fault or further page fault. The overall result is quick succession page faults with the rest of the OS running as normal.
At least we know each process/thread is now being kept reasonably independent :) Just need to be careful of allocations when loading a new thread/process.

---

All TODOs / notes necessary for stable ELF support completed. Further work to be done on this later.

Optimise the File System code

Consider optimising the file system code to improve its speed and memory efficiency. At the moment file searching, reading and writing all take too long.

Old comments

Improved a lot recently with multiple cluster and multiple sector read support.

Drivers Compiler legacy code cleanup

Replace chunks of code like this:

List<Type> allParams = ((MethodInfo)methodToCall).GetParameters().Select(x => x.ParameterType).ToList();
//Go through each one
if (!methodToCall.IsStatic)
{
    allParams.Insert(0, methodToCall.DeclaringType);
}
foreach (Type aParam in allParams)
{
    //Pop the paramter off our stack 
    //(Note: Return value was never pushed onto our stack. See above)
    conversionState.CurrentStackFrame.Stack.Pop();
    //Add the size of the paramter to the total number of bytes to pop
    bytesToAdd += conversionState.TheILLibrary.GetTypeInfo(aParam).SizeOnStackInBytes;
}

With something which uses methodToCallInfo.ArgumentInfos - it's a useful optimisation / improvement. Also, check whether ArgumentInfos already includes the instance parameter or not.

Restructure : Main Process : Shift out of ManagedMain

Create an actual task/process for the main process instead of a ManagedMain method inside the Kernel class.

Can FlingOS run C# programs, like from .NET or Mono?

Sorry if the answer is in the documentation and I didn't noticed it.

Will I be able to compile a C# program against Mono, for example, and run it on FlingOS ?

Restructure : Main Process : Initialise other processes

The Main Process should start the Device Manager and Window Manager processes.

Optimised IL return/break inside try/finally

Investigate what happens in optimised IL if you put a return (or break) inside a try-finally. May need to check for this case and handle appropriately.

Add metrics to Drivers Compiler

Add metrics to the Drivers Compiler.

Initially, use it to track which IL ops are used during a compile. This will allow us to determine if all the IL ops are used by the compiler verification kernel.

Restructure : Boot/Load : Virtual Memory Manager initialisation

Make the virtual memory manager initialise much earlier (in the static constructor calls) so that the heap can do system calls to auto-expand.

Thread/Process Safety

Go through all OS code and creating a list of potential thread-unsafe code.

e.g. the entire PATA driver is thread-unsafe as there is no locking on the methods. (It works at the moment because only the MainShell uses the PATA driver.)

After creating the list, create "Investigate and Resolve" issues for each item on the list so they can solved one by one.

USB Eject/cleanup

Proper Eject and cleanup in the USB stack. At the moment, ejected devices aren't removed from the devices list and unplugged devices aren't removed either. Also, we need to handle proper cleanup / err detection and handling when a device error occurs or a device is removed during initialisation or an operation.

EHCI Isochronous Transfer Queue support

Implement support for the Isochronous Transfer Queue in the EHCI driver.

Exception handling for externally loaded drivers / applications

Add exception handling subsystem for processes. This will involve additions to the system calls (i.e. ABI) to allow registering of an exception handler.

Restructure : Boot/Load : IDT initialisation

• Remove the assembly code IDT initialisation
• Add methods for loading the processor IDT pointer from C#
• Add methods for getting/setting IDT entries from C#
• Add methods for initialising the IDT from C#
• Call the new C# IDT init functions from the Kernel Main method.

Add PATAPI Write support

Add Write support to the PATAPI driver. This will also require detecting if the hardware/physical drive supports writing and if the disc is writeable (if that's even possible).

SATA driver

Create a SATA driver.

Restructure : Main Process : Take control of static heap

Make the main process claim the static heap as its dynamic heap (this should be the first thing done by the process).

Network stack

Design and begin to implement a TCP/IP network stack, initially focusing on whatever the network card is that VMWare virtualises. This will need to occur after FlingOS has finished restructuring.

CPUs Article - Add x86 Protected Mode info

Add sample code for switching from real to protected mode for x86 architecture along with explanation of the difference.

USB Low and Full Speed Device support

Complete the UHCI implementation and then test/bug fix it. So far, a lot of the implementation is there but it isn't quite working.