michaelkuc6/blog_os
1
0
Fork 0
mirror of https://github.com/phil-opp/blog_os.git synced 2025-12-16 22:37:49 +00:00
Code Issues Projects Releases Wiki Activity

Update to new internal link syntax

Browse Source
This commit is contained in:
Philipp Oppermann
2019-09-29 15:24:21 +02:00
parent 0b971c9128
commit a133ea82cf
29 changed files with 117 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
blog/content/first-edition/extra/naked-exceptions/01-catching-exceptions/index.md
View File

@@ -20,8 +20,8 @@ As always, the complete source code is on [GitHub]. Please file [issues] for any
> **Note**: This post describes how to handle exceptions using naked functions (see [“Handling Exceptions with Naked Functions”] for an overview). Our new way of handling exceptions can be found in the [“Handling Exceptions”] post.
[“Handling Exceptions with Naked Functions”]: ./first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions”]: ./first-edition/posts/09-handling-exceptions/index.md
[“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions”]: @/first-edition/posts/09-handling-exceptions/index.md
## Exceptions
An exception signals that something is wrong with the current instruction. For example, the CPU issues an exception if the current instruction tries to divide by 0. When an exception occurs, the CPU interrupts its current work and immediately calls a specific exception handler function, depending on the exception type.

4
blog/content/first-edition/extra/naked-exceptions/02-better-exception-messages/index.md
View File

@@ -21,8 +21,8 @@ As always, the complete source code is on [GitHub]. Please file [issues] for any
> **Note**: This post describes how to handle exceptions using naked functions (see [“Handling Exceptions with Naked Functions”] for an overview). Our new way of handling exceptions can be found in the [“Handling Exceptions”] post.
[“Handling Exceptions with Naked Functions”]: ./first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions”]: ./first-edition/posts/09-handling-exceptions/index.md
[“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions”]: @/first-edition/posts/09-handling-exceptions/index.md
## Exceptions in Detail
An exception signals that something is wrong with the currently-executed instruction. Whenever an exception occurs, the CPU interrupts its current work and starts an internal exception routine.

12
blog/content/first-edition/extra/naked-exceptions/03-returning-from-exceptions/index.md
View File

@@ -21,8 +21,8 @@ As always, the complete source code is on [GitHub]. Please file [issues] for any
> **Note**: This post describes how to handle exceptions using naked functions (see [“Handling Exceptions with Naked Functions”] for an overview). Our new way of handling exceptions can be found in the [“Handling Exceptions”] post.
[“Handling Exceptions with Naked Functions”]: ./first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions”]: ./first-edition/posts/09-handling-exceptions/index.md
[“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions”]: @/first-edition/posts/09-handling-exceptions/index.md
## Introduction
Most exceptions are fatal and can't be resolved. For example, we can't return from a divide-by-zero exception in a reasonable way. However, there are some exceptions that we can resolve:
@@ -42,7 +42,7 @@ The breakpoint exception is commonly used in debuggers: When the user sets a bre
For our use case, we don't need to overwrite any instructions (it wouldn't even be possible since we [set the page table flags] to read-only). Instead, we just want to print a message when the breakpoint instruction is executed and then continue the program.
[set the page table flags]: ./first-edition/posts/07-remap-the-kernel/index.md#using-the-correct-flags
[set the page table flags]: @/first-edition/posts/07-remap-the-kernel/index.md#using-the-correct-flags
### Catching Breakpoints
Let's start by defining a handler function for the breakpoint exception:
@@ -216,7 +216,7 @@ Instead of the expected _“It did not crash”_ message after the breakpoint ex
### Debugging
Let's debug it using GDB. For that we execute `make debug` in one terminal (which starts QEMU with the `-s -S` flags) and then `make gdb` (which starts and connects GDB) in a second terminal. For more information about GDB debugging, check out our [Set Up GDB] guide.
[Set Up GDB]: ./first-edition/extra/set-up-gdb/index.md
[Set Up GDB]: @/first-edition/extra/set-up-gdb/index.md
First we want to check if our `iretq` was successful. Therefore we set a breakpoint on the `println!("It did not crash line!")` statement in `src/lib.rs`. Let's assume that it's on line 61:
@@ -304,7 +304,7 @@ Unfortunately, Rust does not support such a calling convention. It was [proposed
[interrupt calling conventions]: https://github.com/rust-lang/rfcs/pull/1275
[Naked functions]: https://github.com/rust-lang/rfcs/blob/master/text/1201-naked-fns.md
[naked fn post]: ./first-edition/extra/naked-exceptions/02-better-exception-messages/index.md#naked-functions
[naked fn post]: @/first-edition/extra/naked-exceptions/02-better-exception-messages/index.md#naked-functions
### A naked wrapper function
@@ -574,7 +574,7 @@ It doesn't compile anymore. The error tells us that the Rust compiler no longer
The [core library] is implicitly linked to all `no_std` crates and contains things such as `Result`, `Option`, and iterators. We've used that library without problems since [the very beginning], so why is it no longer available?
[core library]: https://doc.rust-lang.org/nightly/core/index.html
[the very beginning]: ./first-edition/posts/03-set-up-rust/index.md
[the very beginning]: @/first-edition/posts/03-set-up-rust/index.md
The problem is that the core library is distributed together with the Rust compiler as a _precompiled_ library. So it is only valid for the host triple, which is `x86_64-unknown-linux-gnu` in our case. If we want to compile code for other targets, we need to recompile `core` for these targets first.

4
blog/content/first-edition/posts/01-multiboot-kernel/index.md
View File

@@ -187,7 +187,7 @@ Idx Name Size VMA LMA File off Algn
```
_Note_: The `ld` and `objdump` commands are platform specific. If you're _not_ working on x86_64 architecture, you will need to [cross compile binutils]. Then use `x86_64‑elf‑ld` and `x86_64‑elf‑objdump` instead of `ld` and `objdump`.
[cross compile binutils]: ./first-edition/extra/cross-compile-binutils.md
[cross compile binutils]: @/first-edition/extra/cross-compile-binutils.md
## Creating the ISO
All PC BIOSes know how to boot from a CD-ROM, so we want to create a bootable CD-ROM image, containing our kernel and the GRUB bootloader's files, in a single file called an [ISO](https://en.wikipedia.org/wiki/ISO_image). Make the following directory structure and copy the `kernel.bin` to the right place:
@@ -315,7 +315,7 @@ Now we can invoke `make` and all updated assembly files are compiled and linked.
In the [next post] we will create a page table and do some CPU configuration to switch to the 64-bit [long mode].
[next post]: ./first-edition/posts/02-entering-longmode/index.md
[next post]: @/first-edition/posts/02-entering-longmode/index.md
[long mode]: https://en.wikipedia.org/wiki/Long_mode
## Footnotes

10
blog/content/first-edition/posts/02-entering-longmode/index.md
View File

@@ -11,7 +11,7 @@ updated = "2015-10-29"
In the [previous post] we created a minimal multiboot kernel. It just prints `OK` and hangs. The goal is to extend it and call 64-bit [Rust] code. But the CPU is currently in [protected mode] and allows only 32-bit instructions and up to 4GiB memory. So we need to set up _Paging_ and switch to the 64-bit [long mode] first.
[previous post]: ./first-edition/posts/01-multiboot-kernel/index.md
[previous post]: @/first-edition/posts/01-multiboot-kernel/index.md
[Rust]: http://www.rust-lang.org/
[protected mode]: https://en.wikipedia.org/wiki/Protected_mode
[long mode]: https://en.wikipedia.org/wiki/Long_mode
@@ -39,7 +39,7 @@ error:
At address `0xb8000` begins the so-called [VGA text buffer]. It's an array of screen characters that are displayed by the graphics card. A [future post] will cover the VGA buffer in detail and create a Rust interface to it. But for now, manual bit-fiddling is the easiest option.
[VGA text buffer]: https://en.wikipedia.org/wiki/VGA-compatible_text_mode
[future post]: ./first-edition/posts/04-printing-to-screen/index.md
[future post]: @/first-edition/posts/04-printing-to-screen/index.md
A screen character consists of a 8 bit color code and a 8 bit [ASCII] character. We used the color code `4f` for all characters, which means white text on red background. `0x52` is an ASCII `R`, `0x45` is an `E`, `0x3a` is a `:`, and `0x20` is a space. The second space is overwritten by the given ASCII byte. Finally the CPU is stopped with the `hlt` instruction.
@@ -492,8 +492,8 @@ _Congratulations_! You have successfully wrestled through this CPU configuration
#### One Last Thing
Above, we reloaded the code segment register `cs` with the new GDT offset. However, the data segment registers `ss`, `ds`, `es`, `fs`, and `gs` still contain the data segment offsets of the old GDT. This isn't necessarily bad, since they're ignored by almost all instructions in 64-bit mode. However, there are a few instructions that expect a valid data segment descriptor _or the null descriptor_ in those registers. An example is the the [iretq] instruction that we'll need in the [_Returning from Exceptions_] post.
[iretq]: ./first-edition/extra/naked-exceptions/03-returning-from-exceptions/index.md#the
[_Returning from Exceptions_]: ./first-edition/extra/naked-exceptions/03-returning-from-exceptions/index.md
[iretq]: @/first-edition/extra/naked-exceptions/03-returning-from-exceptions/index.md#the
[_Returning from Exceptions_]: @/first-edition/extra/naked-exceptions/03-returning-from-exceptions/index.md
To avoid future problems, we reload all data segment registers with null:
@@ -515,7 +515,7 @@ long_mode_start:
It's time to finally leave assembly behind and switch to [Rust]. Rust is a systems language without garbage collections that guarantees memory safety. Through a real type system and many abstractions it feels like a high-level language but can still be low-level enough for OS development. The [next post] describes the Rust setup.
[Rust]: https://www.rust-lang.org/
[next post]: ./first-edition/posts/03-set-up-rust/index.md
[next post]: @/first-edition/posts/03-set-up-rust/index.md
## Footnotes
[^hardware_lookup]: In the x86 architecture, the page tables are _hardware walked_, so the CPU will look at the table on its own when it needs a translation. Other architectures, for example MIPS, just throw an exception and let the OS translate the virtual address.

10
blog/content/first-edition/posts/03-set-up-rust/index.md
View File

@@ -11,8 +11,8 @@ updated = "2017-04-12"
In the previous posts we created a [minimal Multiboot kernel][multiboot post] and [switched to Long Mode][long mode post]. Now we can finally switch to [Rust] code. Rust is a high-level language without runtime. It allows us to not link the standard library and write bare metal code. Unfortunately the setup is not quite hassle-free yet.
[multiboot post]: ./first-edition/posts/01-multiboot-kernel/index.md
[long mode post]: ./first-edition/posts/02-entering-longmode/index.md
[multiboot post]: @/first-edition/posts/01-multiboot-kernel/index.md
[long mode post]: @/first-edition/posts/02-entering-longmode/index.md
[Rust]: https://www.rust-lang.org/
<!-- more -->
@@ -92,7 +92,7 @@ Let's define some properties of our target system:
- **No SSE**: Our target might not have [SSE] support. Even if it does, we probably don't want to use SSE instructions in our kernel, because it makes interrupt handling much slower. We will explain this in detail in the [“Handling Exceptions”] post.
- **No hardware floats**: The `x86_64` architecture uses SSE instructions for floating point operations, which we don't want to use (see the previous point). So we also need to avoid hardware floating point operations in our kernel. Instead, we will use _soft floats_, which are basically software functions that emulate floating point operations using normal integers.
[“Handling Exceptions”]: ./first-edition/posts/09-handling-exceptions/index.md
[“Handling Exceptions”]: @/first-edition/posts/09-handling-exceptions/index.md
### Target Specifications
Rust allows us to define [custom targets] through a JSON configuration file. A minimal target specification equal to `x86_64-unknown-linux-gnu` (the default 64-bit Linux target) looks like this:
@@ -486,10 +486,10 @@ Some notes:
### Stack Overflows
Since we still use the small 64 byte [stack from the last post], we must be careful not to [overflow] it. Normally, Rust tries to avoid stack overflows through _guard pages_: The page below the stack isn't mapped and such a stack overflow triggers a page fault (instead of silently overwriting random memory). But we can't unmap the page below our stack right now since we currently use only a single big page. Fortunately the stack is located just above the page tables. So some important page table entry would probably get overwritten on stack overflow and then a page fault occurs, too.
[stack from the last post]: ./first-edition/posts/02-entering-longmode/index.md#creating-a-stack
[stack from the last post]: @/first-edition/posts/02-entering-longmode/index.md#creating-a-stack
[overflow]: https://en.wikipedia.org/wiki/Stack_overflow
## What's next?
Until now we write magic bits to some memory location when we want to print something to screen. In the [next post] we create a abstraction for the VGA text buffer that allows us to print strings in different colors and provides a simple interface.
[next post]: ./first-edition/posts/04-printing-to-screen/index.md
[next post]: @/first-edition/posts/04-printing-to-screen/index.md

6
blog/content/first-edition/posts/04-printing-to-screen/index.md
View File

@@ -11,7 +11,7 @@ updated = "2016-10-31"
In the [previous post] we switched from assembly to [Rust], a systems programming language that provides great safety. But so far we are using unsafe features like [raw pointers] whenever we want to print to screen. In this post we will create a Rust module that provides a safe and easy-to-use interface for the VGA text buffer. It will support Rust's [formatting macros], too.
[previous post]: ./first-edition/posts/03-set-up-rust/index.md
[previous post]: @/first-edition/posts/03-set-up-rust/index.md
[Rust]: https://www.rust-lang.org/
[raw pointers]: https://doc.rust-lang.org/book/raw-pointers.html
[formatting macros]: https://doc.rust-lang.org/std/fmt/#related-macros
@@ -642,7 +642,7 @@ In the next posts we will map the kernel pages correctly so that accessing `0x0`
The [next post] describes the Multiboot information structure and creates a frame allocator using the information about memory areas.
[next post]: ./first-edition/posts/05-allocating-frames/index.md
[next post]: @/first-edition/posts/05-allocating-frames/index.md
## Other Rust OS Projects
Now that you know the very basics of OS development in Rust, you should also check out the following projects:
@@ -658,7 +658,7 @@ _Note_: You need to [cross compile binutils] to build it (or you create some sym
[Rust Bare-Bones Kernel]: https://github.com/thepowersgang/rust-barebones-kernel
[higher half]: http://wiki.osdev.org/Higher_Half_Kernel
[cross compile binutils]: ./first-edition/extra/cross-compile-binutils.md
[cross compile binutils]: @/first-edition/extra/cross-compile-binutils.md
[RustOS]: https://github.com/RustOS-Fork-Holding-Ground/RustOS
["Tifflin" Experimental Kernel]:https://github.com/thepowersgang/rust_os
[Redox]: https://github.com/redox-os/redox

4
blog/content/first-edition/posts/05-allocating-frames/index.md
View File

@@ -430,10 +430,10 @@ Now we have a working frame allocator. It is a bit rudimentary and cannot free f
## What's next?
The [next post] will be about paging again. We will use the frame allocator to create a safe module that allows us to switch page tables and map pages. Then we will use this module and the information from the Elf-sections tag to remap the kernel correctly.
[next post]: ./first-edition/posts/06-page-tables/index.md
[next post]: @/first-edition/posts/06-page-tables/index.md
## Recommended Posts
Eric Kidd started the [Bare Metal Rust] series last week. Like this post, it builds upon the code from [Printing to Screen], but tries to support keyboard input instead of wrestling through memory management details.
[Bare Metal Rust]: http://www.randomhacks.net/bare-metal-rust/
[Printing to Screen]: ./first-edition/posts/04-printing-to-screen/index.md
[Printing to Screen]: @/first-edition/posts/04-printing-to-screen/index.md

6
blog/content/first-edition/posts/06-page-tables/index.md
View File

@@ -52,7 +52,7 @@ pub struct Page {
```
We import the `PAGE_SIZE` and define a constant for the number of entries per table. To make future function signatures more expressive, we can use the type aliases `PhysicalAddress` and `VirtualAddress`. The `Page` struct is similar to the `Frame` struct in the [previous post], but represents a virtual page instead of a physical frame.
[previous post]: ./first-edition/posts/05-allocating-frames/index.md#a-memory-module
[previous post]: @/first-edition/posts/05-allocating-frames/index.md#a-memory-module
### Page Table Entries
To model page table entries, we create a new `entry` submodule:
@@ -652,7 +652,7 @@ pub struct ActivePageTable {
```
We can't store the `Table<Level4>` directly because it needs to be at a special memory location (like the [VGA text buffer]). We could use a raw pointer or `&mut` instead of [Unique], but Unique indicates ownership better.
[VGA text buffer]: ./first-edition/posts/04-printing-to-screen/index.md#the-text-buffer
[VGA text buffer]: @/first-edition/posts/04-printing-to-screen/index.md#the-text-buffer
[Unique]: https://doc.rust-lang.org/1.10.0/core/ptr/struct.Unique.html
Because the `ActivePageTable` owns the unique recursive mapped P4 table, there must be only one `ActivePageTable` instance. Thus we make the constructor function unsafe:
@@ -881,7 +881,7 @@ This post has become pretty long. So let's summarize what we've done:
## What's next?
In the [next post] we will extend this module and add a function to modify inactive page tables. Through that function, we will create a new page table hierarchy that maps the kernel correctly using 4KiB pages. Then we will switch to the new table to get a safer kernel environment.
[next post]: ./first-edition/posts/07-remap-the-kernel/index.md
[next post]: @/first-edition/posts/07-remap-the-kernel/index.md
Afterwards, we will use this paging module to build a heap allocator. This will allow us to use allocation and collection types such as `Box` and `Vec`.

20
blog/content/first-edition/posts/07-remap-the-kernel/index.md
View File

@@ -21,18 +21,18 @@ As always, you can find the source code on [GitHub]. Don't hesitate to file issu
In the [previous post], we had a strange bug in the `unmap` function. Its reason was a silent stack overflow, which corrupted the page tables. Fortunately, our kernel stack is right above the page tables so that we noticed the overflow relatively quickly. This won't be the case when we add threads with new stacks in the future. Then a silent stack overflow could overwrite some data without us noticing. But eventually some completely unrelated function fails because a variable changed its value.
[previous post]: ./first-edition/posts/06-page-tables/index.md
[previous post]: @/first-edition/posts/06-page-tables/index.md
As you can imagine, these kinds of bugs are horrendous to debug. For that reason we will create a new hierarchical page table in this post, which has _guard page_ below the stack. A guard page is basically an unmapped page that causes a page fault when accessed. Thus we can catch stack overflows right when they happen.
Also, we will use the [information about kernel sections] to map the various sections individually instead of blindly mapping the first gigabyte. To improve safety even further, we will set the correct page table flags for the various sections. Thus it won't be possible to modify the contents of `.text` or to execute code from `.data` anymore.
[information about kernel sections]: ./first-edition/posts/05-allocating-frames/index.md#kernel-elf-sections
[information about kernel sections]: @/first-edition/posts/05-allocating-frames/index.md#kernel-elf-sections
## Preparation
There are many things that can go wrong when we switch to a new table. Therefore it's a good idea to [set up a debugger][set up gdb]. You should not need it when you follow this post, but it's good to know how to debug a problem when it occurs[^fn-debug-notes].
[set up gdb]: ./first-edition/extra/set-up-gdb/index.md
[set up gdb]: @/first-edition/extra/set-up-gdb/index.md
We also update the `Page` and `Frame` types to make our lives easier. The `Page` struct gets some derived traits:
@@ -281,7 +281,7 @@ pub fn map_table_frame(&mut self,
```
This function interprets the given frame as a page table frame and returns a `Table` reference. We return a table of level 1 because it [forbids calling the `next_table` methods][some clever solution]. Calling `next_table` must not be possible since it's not a page of the recursive mapping. To be able to return a `Table<Level1>`, we need to make the `Level1` enum in `memory/paging/table.rs` public.
[some clever solution]: ./first-edition/posts/06-page-tables/index.md#some-clever-solution
[some clever solution]: @/first-edition/posts/06-page-tables/index.md#some-clever-solution
The `unsafe` block is safe since the `VirtualAddress` returned by the `map` function is always valid and the type cast just reinterprets the frame's content.
@@ -554,7 +554,7 @@ First, we create a temporary page at page number `0xcafebabe`. We could use `0xd
Then we use the `with` function to temporary change the recursive mapping and execute the closure as if the `new_table` were active. This allows us to map the sections in the new table without changing the active mapping. To get the kernel sections, we use the [Multiboot information structure].
[Multiboot information structure]: ./first-edition/posts/05-allocating-frames/index.md#the-multiboot-information-structure
[Multiboot information structure]: @/first-edition/posts/05-allocating-frames/index.md#the-multiboot-information-structure
Let's resolve the above `TODO` by identity mapping the sections:
@@ -806,7 +806,7 @@ Let's cross our fingers and run it…
### Debugging
A QEMU boot loop indicates that some CPU exception occurred. We can see all thrown CPU exception by starting QEMU with `-d int` (as described [here][qemu debugging]):
[qemu debugging]: ./first-edition/posts/03-set-up-rust/index.md#debugging
[qemu debugging]: @/first-edition/posts/03-set-up-rust/index.md#debugging
```bash
> qemu-system-x86_64 -d int -no-reboot -cdrom build/os-x86_64.iso
@@ -831,8 +831,8 @@ These lines are the important ones. We can read many useful information from the
[osdev exception overview]: http://wiki.osdev.org/Exceptions
[page fault]: http://wiki.osdev.org/Exceptions#Page_Fault
[page fault error code]: http://wiki.osdev.org/Exceptions#Error_code
[GDT segment]: ./first-edition/posts/02-entering-longmode/index.md#loading-the-gdt
[VGA text buffer]: ./first-edition/posts/04-printing-to-screen/index.md#the-vga-text-buffer
[GDT segment]: @/first-edition/posts/02-entering-longmode/index.md#loading-the-gdt
[VGA text buffer]: @/first-edition/posts/04-printing-to-screen/index.md#the-vga-text-buffer
So let's find out which function caused the exception:
@@ -1030,7 +1030,7 @@ The final step is to create a guard page for our kernel stack.
The decision to place the kernel stack right above the page tables was already useful to detect a silent stack overflow in the [previous post][silent stack overflow]. Now we profit from it again. Let's look at our assembly `.bss` section again to understand why:
[silent stack overflow]: ./first-edition/posts/06-page-tables/index.md#translate
[silent stack overflow]: @/first-edition/posts/06-page-tables/index.md#translate
```nasm
; in src/arch/x86_64/boot.asm
@@ -1091,7 +1091,7 @@ Unfortunately stack probes require compiler support. They already work on Window
## What's next?
Now that we have a (mostly) safe kernel stack and a working page table module, we can add a virtual memory allocator. The [next post] will explore Rust's allocator API and create a very basic allocator. At the end of that post, we will be able to use Rust's allocation and collections types such as [Box], [Vec], or even [BTreeMap].
[next post]: ./first-edition/posts/08-kernel-heap/index.md
[next post]: @/first-edition/posts/08-kernel-heap/index.md
[Box]: https://doc.rust-lang.org/nightly/alloc/boxed/struct.Box.html
[Vec]: https://doc.rust-lang.org/1.10.0/collections/vec/struct.Vec.html
[BTreeMap]: https://doc.rust-lang.org/1.10.0/collections/btree_map/struct.BTreeMap.html

10
blog/content/first-edition/posts/08-kernel-heap/index.md
View File

@@ -10,8 +10,8 @@ template = "first-edition/page.html"
In the previous posts we created a [frame allocator] and a [page table module]. Now we are ready to create a kernel heap and a memory allocator. Thus, we will unlock `Box`, `Vec`, `BTreeMap`, and the rest of the [alloc] crate.
[frame allocator]: ./first-edition/posts/05-allocating-frames/index.md
[page table module]: ./first-edition/posts/06-page-tables/index.md
[frame allocator]: @/first-edition/posts/05-allocating-frames/index.md
[page table module]: @/first-edition/posts/06-page-tables/index.md
[alloc]: https://doc.rust-lang.org/nightly/alloc/index.html
<!-- more -->
@@ -460,8 +460,8 @@ That's it. Now our `memory::init` function can only be called once. The macro wo
### Mapping the Heap
Now we're ready to map the heap pages. In order to do it, we need access to the `ActivePageTable` or `Mapper` instance (see the [page table] and [kernel remapping] posts). For that we return it from the `paging::remap_the_kernel` function:
[page table]: ./first-edition/posts/06-page-tables/index.md
[kernel remapping]: ./first-edition/posts/07-remap-the-kernel/index.md
[page table]: @/first-edition/posts/06-page-tables/index.md
[kernel remapping]: @/first-edition/posts/07-remap-the-kernel/index.md
```rust
// in src/memory/paging/mod.rs
@@ -736,4 +736,4 @@ Now we're able to use heap storage in our kernel without leaking memory. This al
## What's next?
This post concludes the section about memory management for now. We will revisit this topic eventually, but now it's time to explore other topics. The upcoming posts will be about CPU exceptions and interrupts. We will catch all page, double, and triple faults and create a driver to read keyboard input. The [next post] starts by setting up a so-called _Interrupt Descriptor Table_.
[next post]: ./first-edition/posts/09-handling-exceptions/index.md
[next post]: @/first-edition/posts/09-handling-exceptions/index.md

4
blog/content/first-edition/posts/09-handling-exceptions/index.md
View File

@@ -229,7 +229,7 @@ The breakpoint exception is commonly used in debuggers: When the user sets a bre
For our use case, we don't need to overwrite any instructions (it wouldn't even be possible since we [set the page table flags] to read-only). Instead, we just want to print a message when the breakpoint instruction is executed and then continue the program.
[set the page table flags]: ./first-edition/posts/07-remap-the-kernel/index.md#using-the-correct-flags
[set the page table flags]: @/first-edition/posts/07-remap-the-kernel/index.md#using-the-correct-flags
So let's create a simple `breakpoint_handler` function and add it to our IDT:
@@ -461,7 +461,7 @@ The documentation of the [`Idt`] struct and the [OSDev Wiki][osdev wiki exceptio
## Too much Magic?
The `x86-interrupt` calling convention and the [`Idt`] type made the exception handling process relatively straightforward and painless. If this was too much magic for you and you like to learn all the gory details of exception handling, we got you covered: Our [“Handling Exceptions with Naked Functions”] series shows how to handle exceptions without the `x86-interrupt` calling convention and also creates its own `Idt` type. Historically, these posts were the main exception handling posts before the `x86-interrupt` calling convention and the `x86_64` crate existed.
[“Handling Exceptions with Naked Functions”]: ./first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md
## What's next?
We've successfully caught our first exception and returned from it! The next step is to add handlers for other common exceptions such as page faults. We also need to make sure that we never cause a [triple fault], since it causes a complete system reset. The next post explains how we can avoid this by correctly catching [double faults].

16
blog/content/first-edition/posts/10-double-faults/index.md
View File

@@ -20,7 +20,7 @@ As always, the complete source code is available on [GitHub]. Please file [issue
## What is a Double Fault?
In simplified terms, a double fault is a special exception that occurs when the CPU fails to invoke an exception handler. For example, it occurs when a page fault is triggered but there is no page fault handler registered in the [Interrupt Descriptor Table][IDT] (IDT). So it's kind of similar to catch-all blocks in programming languages with exceptions, e.g. `catch(...)` in C++ or `catch(Exception e)` in Java or C#.
[IDT]: ./first-edition/posts/09-handling-exceptions/index.md#the-interrupt-descriptor-table
[IDT]: @/first-edition/posts/09-handling-exceptions/index.md#the-interrupt-descriptor-table
A double fault behaves like a normal exception. It has the vector number `8` and we can define a normal handler function for it in the IDT. It is really important to provide a double fault handler, because if a double fault is unhandled a fatal _triple fault_ occurs. Triple faults can't be caught and most hardware reacts with a system reset.
@@ -119,7 +119,7 @@ For example, what happens if… :
3. a divide-by-zero handler causes a breakpoint exception, but the breakpoint handler is swapped out?
4. our kernel overflows its stack and the [guard page] is hit?
[guard page]: ./first-edition/posts/07-remap-the-kernel/index.md#creating-a-guard-page
[guard page]: @/first-edition/posts/07-remap-the-kernel/index.md#creating-a-guard-page
Fortunately, the AMD64 manual ([PDF][AMD64 manual]) has an exact definition (in Section 8.2.9). According to it, a “double fault exception _can_ occur when a second exception occurs during the handling of a prior (first) exception handler”. The _“can”_ is important: Only very specific combinations of exceptions lead to a double fault. These combinations are:
@@ -199,7 +199,7 @@ struct InterruptStackTable {
For each exception handler, we can choose a stack from the IST through the `options` field in the corresponding [IDT entry]. For example, we could use the first stack in the IST for our double fault handler. Then the CPU would automatically switch to this stack whenever a double fault occurs. This switch would happen before anything is pushed, so it would prevent the triple fault.
[IDT entry]: ./first-edition/posts/09-handling-exceptions/index.md#the-interrupt-descriptor-table
[IDT entry]: @/first-edition/posts/09-handling-exceptions/index.md#the-interrupt-descriptor-table
### Allocating a new Stack
In order to fill an Interrupt Stack Table later, we need a way to allocate new stacks. Therefore we extend our `memory` module with a new `stack_allocator` submodule:
@@ -230,7 +230,7 @@ impl StackAllocator {
```
We create a simple `StackAllocator` that allocates stacks from a given range of pages (`PageIter` is an Iterator over a range of pages; we introduced it [in the kernel heap post].).
[in the kernel heap post]: ./first-edition/posts/08-kernel-heap/index.md#mapping-the-heap
[in the kernel heap post]: @/first-edition/posts/08-kernel-heap/index.md#mapping-the-heap
We add a `alloc_stack` method that allocates a new stack:
@@ -285,8 +285,8 @@ impl StackAllocator {
```
The method takes mutable references to the [ActivePageTable] and a [FrameAllocator], since it needs to map the new virtual stack pages to physical frames. We define that the stack size is a multiple of the page size.
[ActivePageTable]: ./first-edition/posts/06-page-tables/index.md#page-table-ownership
[FrameAllocator]: ./first-edition/posts/05-allocating-frames/index.md#a-frame-allocator
[ActivePageTable]: @/first-edition/posts/06-page-tables/index.md#page-table-ownership
[FrameAllocator]: @/first-edition/posts/05-allocating-frames/index.md#a-frame-allocator
Instead of operating directly on `self.range`, we [clone] it and only write it back on success. This way, subsequent stack allocations can still succeed if there are pages left (e.g., a call with `size_in_pages = 3` can still succeed after a failed call with `size_in_pages = 100`).
@@ -297,7 +297,7 @@ In order to be able to clone `PageIter`, we add a `#[derive(Clone)]` to its defi
The actual allocation is straightforward: First, we choose the next page as [guard page]. Then we choose the next `size_in_pages` pages as stack pages using [Iterator::nth]. If all three variables are `Some`, the allocation succeeded and we map the stack pages to physical frames using [ActivePageTable::map]. The guard page remains unmapped.
[Iterator::nth]: https://doc.rust-lang.org/nightly/core/iter/trait.Iterator.html#method.nth
[ActivePageTable::map]: ./first-edition/posts/06-page-tables/index.md#more-mapping-functions
[ActivePageTable::map]: @/first-edition/posts/06-page-tables/index.md#more-mapping-functions
Finally, we create and return a new `Stack`, which we define as follows:
@@ -505,7 +505,7 @@ The Global Descriptor Table (GDT) is a relict that was used for [memory segmenta
We already created a GDT [when switching to long mode]. Back then, we used assembly to create valid code and data segment descriptors, which were required to enter 64-bit mode. We could just edit that assembly file and add an additional TSS descriptor. However, we now have the expressiveness of Rust, so let's do it in Rust instead.
[when switching to long mode]: ./first-edition/posts/02-entering-longmode/index.md#the-global-descriptor-table
[when switching to long mode]: @/first-edition/posts/02-entering-longmode/index.md#the-global-descriptor-table
We start by creating a new `interrupts::gdt` submodule. For that we need to rename the `src/interrupts.rs` file to `src/interrupts/mod.rs`. Then we can create a new submodule:

6
blog/content/news/2018-03-09-pure-rust.md
View File

@@ -12,7 +12,7 @@ Over the past six months we've been working on a second edition of this blog. Ou
The [first edition] required several C-tools for building:
[first edition]: ./first-edition/_index.md
[first edition]: @/first-edition/_index.md
- We used the [`GRUB`] bootloader for booting our kernel. To create a bootable disk/CD image we used the [`grub-mkrescue`] tool, which is very difficult to get to run on Windows.
- The [`xorriso`] program was also required, because it is used by `grub-mkrescue`.
@@ -23,7 +23,7 @@ The [first edition] required several C-tools for building:
[`GRUB`]: https://www.gnu.org/software/grub/
[`grub-mkrescue`]: https://www.gnu.org/software/grub/manual/grub/html_node/Invoking-grub_002dmkrescue.html
[`xorriso`]: https://www.gnu.org/software/xorriso/
[entering long mode]: ./first-edition/posts/02-entering-longmode/index.md
[entering long mode]: @/first-edition/posts/02-entering-longmode/index.md
[`nasm`]: http://www.nasm.us/xdoc/2.13.03/html/nasmdoc1.html
[`ld`]: https://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_3.html
[linker script]: http://www.scoberlin.de/content/media/http/informatik/gcc_docs/ld_3.html
@@ -58,6 +58,6 @@ With our custom bootloader in place, the last remaining problem is platform inde
## The new Posts
The second edition is already live at [https://os.phil-opp.com/second-edition]. Please tell us if you have any feedback on the new posts! We're planning to move over the content from the [first edition] iteratively, in a different order and with various other improvements.
[https://os.phil-opp.com/second-edition]: ./second-edition/_index.md
[https://os.phil-opp.com/second-edition]: @/second-edition/_index.md
Many thanks to everyone who helped to make Rust an even better language for OS development!

2
blog/content/second-edition/posts/01-freestanding-rust-binary/index.md
View File

@@ -513,4 +513,4 @@ Note that this is just a minimal example of a freestanding Rust binary. This bin
The [next post] explains the steps needed for turning our freestanding binary into a minimal operating system kernel. This includes creating a custom target, combining our executable with a bootloader, and learning how to print something to the screen.
[next post]: ./second-edition/posts/02-minimal-rust-kernel/index.md
[next post]: @/second-edition/posts/02-minimal-rust-kernel/index.md

12
blog/content/second-edition/posts/02-minimal-rust-kernel/index.md
View File

@@ -8,7 +8,7 @@ date = 2018-02-10
In this post we create a minimal 64-bit Rust kernel for the x86 architecture. We build upon the [freestanding Rust binary] from the previous post to create a bootable disk image, that prints something to the screen.
[freestanding Rust binary]: ./second-edition/posts/01-freestanding-rust-binary/index.md
[freestanding Rust binary]: @/second-edition/posts/01-freestanding-rust-binary/index.md
<!-- more -->
@@ -74,7 +74,7 @@ To make a kernel Multiboot compliant, one just needs to insert a so-called [Mult
Because of these drawbacks we decided to not use GRUB or the Multiboot standard. However, we plan to add Multiboot support to our [bootimage] tool, so that it's possible to load your kernel on a GRUB system too. If you're interested in writing a Multiboot compliant kernel, check out the [first edition] of this blog series.
[first edition]: ./first-edition/_index.md
[first edition]: @/first-edition/_index.md
### UEFI
@@ -169,7 +169,7 @@ This setting specifies that the target doesn't support [stack unwinding] on pani
We're writing a kernel, so we'll need to handle interrupts at some point. To do that safely, we have to disable a certain stack pointer optimization called the _“red zone”_, because it would cause stack corruptions otherwise. For more information, see our separate post about [disabling the red zone].
[disabling the red zone]: ./second-edition/extra/disable-red-zone/index.md
[disabling the red zone]: @/second-edition/extra/disable-red-zone/index.md
```json
"features": "-mmx,-sse,+soft-float",
@@ -183,7 +183,7 @@ The `mmx` and `sse` features determine support for [Single Instruction Multiple
A problem with disabling SIMD is that floating point operations on `x86_64` require SIMD registers by default. To solve this problem, we add the `soft-float` feature, which emulates all floating point operations through software functions based on normal integers.
For more information, see our post on [disabling SIMD](./second-edition/extra/disable-simd/index.md).
For more information, see our post on [disabling SIMD](@/second-edition/extra/disable-simd/index.md).
#### Putting it Together
Our target specification file now looks like this:
@@ -209,7 +209,7 @@ Our target specification file now looks like this:
### Building our Kernel
Compiling for our new target will use Linux conventions (I'm not quite sure why, I assume that it's just LLVM's default). This means that we need an entry point named `_start` as described in the [previous post]:
[previous post]: ./second-edition/posts/01-freestanding-rust-binary/index.md
[previous post]: @/second-edition/posts/01-freestanding-rust-binary/index.md
```rust
// src/main.rs
@@ -275,7 +275,7 @@ Now we can rerun the above command with `xbuild` instead of `build`:
We see that `cargo xbuild` cross-compiles the `core`, `compiler_builtin`, and `alloc` libraries for our new custom target. Since these libraries use a lot of unstable features internally, this only works with a [nightly Rust compiler]. Afterwards, `cargo xbuild` successfully compiles our `blog_os` crate.
[nightly Rust compiler]: ./second-edition/posts/01-freestanding-rust-binary/index.md#installing-rust-nightly
[nightly Rust compiler]: @/second-edition/posts/01-freestanding-rust-binary/index.md#installing-rust-nightly
Now we are able to build our kernel for a bare metal target. However, our `_start` entry point, which will be called by the boot loader, is still empty. So let's output something to screen from it.

14
blog/content/second-edition/posts/04-testing/index.md
View File

@@ -22,11 +22,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
This post replaces the (now deprecated) [_Unit Testing_] and [_Integration Tests_] posts. It assumes that you have followed the [_A Minimal Rust Kernel_] post after 2019-04-27. Mainly, it requires that you have a `.cargo/config` file that [sets a default target] and [defines a runner executable].
[_Unit Testing_]: ./second-edition/posts/deprecated/04-unit-testing/index.md
[_Integration Tests_]: ./second-edition/posts/deprecated/05-integration-tests/index.md
[_A Minimal Rust Kernel_]: ./second-edition/posts/02-minimal-rust-kernel/index.md
[sets a default target]: ./second-edition/posts/02-minimal-rust-kernel/index.md#set-a-default-target
[defines a runner executable]: ./second-edition/posts/02-minimal-rust-kernel/index.md#using-cargo-run
[_Unit Testing_]: @/second-edition/posts/deprecated/04-unit-testing/index.md
[_Integration Tests_]: @/second-edition/posts/deprecated/05-integration-tests/index.md
[_A Minimal Rust Kernel_]: @/second-edition/posts/02-minimal-rust-kernel/index.md
[sets a default target]: @/second-edition/posts/02-minimal-rust-kernel/index.md#set-a-default-target
[defines a runner executable]: @/second-edition/posts/02-minimal-rust-kernel/index.md#using-cargo-run
## Testing in Rust
@@ -150,7 +150,7 @@ Together with the device name (`isa-debug-exit`), we pass the two parameters `io
There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the VGA device.
[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md
[VGA text buffer]: @/second-edition/posts/03-vga-text-buffer/index.md
In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`).
@@ -302,7 +302,7 @@ Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spi
Like the `isa-debug-exit` device, the UART is programmed using port I/O. Since the UART is more complex, it uses multiple I/O ports for programming different device registers. The unsafe `SerialPort::new` function expects the address of the first I/O port of the UART as argument, from which it can calculate the addresses of all needed ports. We're passing the port address `0x3F8`, which is the standard port number for the first serial interface.
[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
[vga lazy-static]: @/second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros:

8
blog/content/second-edition/posts/05-cpu-exceptions/index.md
View File

@@ -349,7 +349,7 @@ Fortunately the `lazy_static` macro exists. Instead of evaluating a `static` at
We already imported the `lazy_static` crate when we [created an abstraction for the VGA text buffer][vga text buffer lazy static]. So we can directly use the `lazy_static!` macro to create our static IDT:
[vga text buffer lazy static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
[vga text buffer lazy static]: @/second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
```rust
// in src/interrupts.rs
@@ -454,16 +454,16 @@ fn test_breakpoint_exception() {
Apart from printing status messages through the [serial port], the test invokes the `int3` function to trigger a breakpoint exception. By checking that the execution continues afterwards, we verify that our breakpoint handler is working correctly.
[serial port]: ./second-edition/posts/04-testing/index.md#serial-port
[serial port]: @/second-edition/posts/04-testing/index.md#serial-port
You can try this new test by running `cargo xtest` (all tests) or `cargo xtest --lib` (only tests of `lib.rs` and its modules). You should see `test_breakpoint_exception...[ok]` in the output.
## Too much Magic?
The `x86-interrupt` calling convention and the [`InterruptDescriptorTable`] type made the exception handling process relatively straightforward and painless. If this was too much magic for you and you like to learn all the gory details of exception handling, we got you covered: Our [“Handling Exceptions with Naked Functions”] series shows how to handle exceptions without the `x86-interrupt` calling convention and also creates its own IDT type. Historically, these posts were the main exception handling posts before the `x86-interrupt` calling convention and the `x86_64` crate existed. Note that these posts are based on the [first edition] of this blog and might be out of date.
[“Handling Exceptions with Naked Functions”]: ./first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md
[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html
[first edition]: ./first-edition/_index.md
[first edition]: @/first-edition/_index.md
## What's next?
We've successfully caught our first exception and returned from it! The next step is to ensure that we catch all exceptions, because an uncaught exception causes a fatal [triple fault], which leads to a system reset. The next post explains how we can avoid this by correctly catching [double faults].

6
blog/content/second-edition/posts/06-double-faults/index.md
View File

@@ -21,7 +21,7 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
## What is a Double Fault?
In simplified terms, a double fault is a special exception that occurs when the CPU fails to invoke an exception handler. For example, it occurs when a page fault is triggered but there is no page fault handler registered in the [Interrupt Descriptor Table][IDT] (IDT). So it's kind of similar to catch-all blocks in programming languages with exceptions, e.g. `catch(...)` in C++ or `catch(Exception e)` in Java or C#.
[IDT]: ./second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
[IDT]: @/second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
A double fault behaves like a normal exception. It has the vector number `8` and we can define a normal handler function for it in the IDT. It is really important to provide a double fault handler, because if a double fault is unhandled a fatal _triple fault_ occurs. Triple faults can't be caught and most hardware reacts with a system reset.
@@ -196,7 +196,7 @@ struct InterruptStackTable {
For each exception handler, we can choose a stack from the IST through the `options` field in the corresponding [IDT entry]. For example, we could use the first stack in the IST for our double fault handler. Then the CPU would automatically switch to this stack whenever a double fault occurs. This switch would happen before anything is pushed, so it would prevent the triple fault.
[IDT entry]: ./second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
[IDT entry]: @/second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
### The IST and TSS
The Interrupt Stack Table (IST) is part of an old legacy structure called _[Task State Segment]_ \(TSS). The TSS used to hold various information (e.g. processor register state) about a task in 32-bit mode and was for example used for [hardware context switching]. However, hardware context switching is no longer supported in 64-bit mode and the format of the TSS changed completely.
@@ -438,7 +438,7 @@ name = "stack_overflow"
harness = false
```
[without a test harness]: ./second-edition/posts/04-testing/index.md#no-harness
[without a test harness]: @/second-edition/posts/04-testing/index.md#no-harness
Now `cargo xtest --test stack_overflow` should compile successfully. The test fails of course, since the `unimplemented` macro panics.

6
blog/content/second-edition/posts/07-hardware-interrupts/index.md
View File

@@ -66,7 +66,7 @@ This graphic shows the typical assignment of interrupt lines. We see that most o
Each controller can be configured through two [I/O ports], one “command” port and one “data” port. For the primary controller these ports are `0x20` (command) and `0x21` (data). For the secondary controller they are `0xa0` (command) and `0xa1` (data). For more information on how the PICs can be configured see the [article on osdev.org].
[I/O ports]: ./second-edition/posts/04-testing/index.md#i-o-ports
[I/O ports]: @/second-edition/posts/04-testing/index.md#i-o-ports
[article on osdev.org]: https://wiki.osdev.org/8259_PIC
### Implementation
@@ -255,7 +255,7 @@ We now have a form of concurrency in our kernel: The timer interrupts occur asyn
We can already provoke a deadlock in our kernel. Remember, our `println` macro calls the `vga_buffer::_print` function, which [locks a global `WRITER`][vga spinlock] using a spinlock:
[vga spinlock]: ./second-edition/posts/03-vga-text-buffer/index.md#spinlocks
[vga spinlock]: @/second-edition/posts/03-vga-text-buffer/index.md#spinlocks
```rust
// in src/vga_buffer.rs
@@ -579,7 +579,7 @@ We now see that a `k` appears on the screen when we press a key. However, this o
To find out _which_ key was pressed, we need to query the keyboard controller. We do this by reading from the data port of the PS/2 controller, which is the [I/O port] with number `0x60`:
[I/O port]: ./second-edition/posts/04-testing/index.md#i-o-ports
[I/O port]: @/second-edition/posts/04-testing/index.md#i-o-ports
```rust
// in src/interrupts.rs

6
blog/content/second-edition/posts/08-paging-introduction/index.md
View File

@@ -247,7 +247,7 @@ It is important to remember flushing the TLB on each page table modification bec
One thing that we did not mention yet: **Our kernel already runs on paging**. The bootloader that we added in the ["A minimal Rust Kernel"] post already set up a 4-level paging hierarchy that maps every page of our kernel to a physical frame. The bootloader does this because paging is mandatory in 64-bit mode on x86_64.
["A minimal Rust kernel"]: ./second-edition/posts/02-minimal-rust-kernel/index.md#creating-a-bootimage
["A minimal Rust kernel"]: @/second-edition/posts/02-minimal-rust-kernel/index.md#creating-a-bootimage
This means that every memory address that we used in our kernel was a virtual address. Accessing the VGA buffer at address `0xb8000` only worked because the bootloader _identity mapped_ that memory page, which means that it mapped the virtual page `0xb8000` to the physical frame `0xb8000`.
@@ -257,7 +257,7 @@ Paging makes our kernel already relatively safe, since every memory access that
Let's try to cause a page fault by accessing some memory outside of our kernel. First, we create a page fault handler and register it in our IDT, so that we see a page fault exception instead of a generic [double fault] :
[double fault]: ./second-edition/posts/06-double-faults/index.md
[double fault]: @/second-edition/posts/06-double-faults/index.md
```rust
// in src/interrupts.rs
@@ -296,7 +296,7 @@ The [`CR2`] register is automatically set by the CPU on a page fault and contain
[`Cr2::read`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr2.html#method.read
[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html
[LLVM bug]: https://github.com/rust-lang/rust/issues/57270
[`hlt_loop`]: ./second-edition/posts/07-hardware-interrupts/index.md#the
[`hlt_loop`]: @/second-edition/posts/07-hardware-interrupts/index.md#the
Now we can try to access some memory outside our kernel:

18
blog/content/second-edition/posts/09-paging-implementation/index.md
View File

@@ -21,11 +21,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
The [previous post] gave an introduction to the concept of paging. It motivated paging by comparing it with segmentation, explained how paging and page tables work, and then introduced the 4-level page table design of `x86_64`. We found out that the bootloader already set up a page table hierarchy for our kernel, which means that our kernel already runs on virtual addresses. This improves safety since illegal memory accesses cause page fault exceptions instead of modifying arbitrary physical memory.
[previous post]: ./second-edition/posts/08-paging-introduction/index.md
[previous post]: @/second-edition/posts/08-paging-introduction/index.md
The post ended with the problem that we [can't access the page tables from our kernel][end of previous post] because they are stored in physical memory and our kernel already runs on virtual addresses. This post continues at this point and explores different approaches of making the page table frames accessible to our kernel. We will discuss the advantages and drawbacks of each approach and then decide for an approach for our kernel.
[end of previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
[end of previous post]: @/second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
To implement the approach, we will need support from the bootloader, so we'll configure it first. Afterward, we will implement a function that traverses the page table hierarchy in order to translate virtual to physical addresses. Finally, we learn how to create new mappings in the page tables and how to find unused memory frames for creating new page tables.
@@ -65,7 +65,7 @@ In this example, we see various identity-mapped page table frames. This way the
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000 KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `28 KiB` because it would collide with the already mapped page at `1004 KiB`. So we have to look further until we find a large enough unmapped area, for example at `1008 KiB`. This is a similar fragmentation problem as with [segmentation].
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
[segmentation]: ./second-edition/posts/08-paging-introduction/index.md#fragmentation
[segmentation]: @/second-edition/posts/08-paging-introduction/index.md#fragmentation
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000 KiB memory region starting at `1008 KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000 KiB` and `2008 KiB` anymore, because we can't identity map it.
@@ -192,7 +192,7 @@ Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 i
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
[sign extension]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86
[sign extension]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86
We use [octal] numbers for representing the addresses since each octal character represents three bits, which allows us to clearly separate the 9-bit indexes of the different page table levels. This isn't possible with the hexadecimal system where each character represents four bits.
@@ -380,7 +380,7 @@ For the module we create an empty `src/memory.rs` file.
At the [end of the previous post], we tried to take a look at the page tables our kernel runs on, but failed since we couldn't access the physical frame that the `CR3` register points to. We're now able to continue from there by creating an `active_level_4_table` function that returns a reference to the active level 4 page table:
[end of the previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
[end of the previous post]: @/second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
```rust
// in src/memory.rs
@@ -603,7 +603,7 @@ When we run it, we see the following output:
As expected, the identity-mapped address `0xb8000` translates to the same physical address. The code page and the stack page translate to some arbitrary physical addresses, which depend on how the bootloader created the initial mapping for our kernel. It's worth noting that the last 12 bits always stay the same after translation, which makes sense because these bits are the [_page offset_] and not part of the translation.
[_page offset_]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
[_page offset_]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
Since each physical address can be accessed by adding the `physical_memory_offset`, the translation of the `physical_memory_offset` address itself should point to physical address `0`. However, the translation fails because the mapping uses huge pages for efficiency, which is not supported in our implementation yet.
@@ -748,7 +748,7 @@ In addition to the `page` that should be mapped, the function expects a mutable
For the mapping, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. Calling [`map_to`] is unsafe because it's possible to break memory safety with invalid arguments, so we need to use an `unsafe` block. For a list of all possible flags, see the [_Page Table Format_] section of the previous post.
[_Page Table Format_]: ./second-edition/posts/08-paging-introduction/index.md#page-table-format
[_Page Table Format_]: @/second-edition/posts/08-paging-introduction/index.md#page-table-format
The [`map_to`] function can fail, so it returns a [`Result`]. Since this is just some example code that does not need to be robust, we just use [`expect`] to panic when an error occurs. On success, the function returns a [`MapperFlush`] type that provides an easy way to flush the newly mapped page from the translation lookaside buffer (TLB) with its [`flush`] method. Like `Result`, the type uses the [`#[must_use]`][must_use] attribute to emit a warning when we accidentally forget to use it.
@@ -791,7 +791,7 @@ Additionally, the graphic shows the physical frame of the VGA text buffer in red
The graphic shows two canditate pages in the virtual address space, both marked in yellow. One page is at address `0x803fdfd000`, which is 3 pages before the mapped page (in blue). While the level 4 and level 3 page table indices are the same as for the blue page, the level 2 and level 1 indices are different (see the [previous post][page-table-indices]). The different index into the level 2 table means that a different level 1 table is used for this page. Since this level 1 table does not exist yet, we would need to create it if we chose that page for our example mapping, which would require an additional unused physical frame. In contrast, the second candidate page at address `0x803fe02000` does not have this problem because it uses the same level 1 page table than the blue page. Thus, all required page tables already exist.
[page-table-indices]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
[page-table-indices]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
In summary, the difficulty of creating a new mapping depends on the virtual page that we want to map. In the easiest case, the level 1 page table for the page already exists and we just need to write a single entry. In the most difficult case, the page is in a memory region for that no level 3 exists yet so that we need to create new level 3, level 2 and level 1 page tables first.
@@ -830,7 +830,7 @@ We first create the mapping for the page at address `0` by calling our `create_e
Then we convert the page to a raw pointer and write a value to offset `400`. We don't write to the start of the page because the top line of the VGA buffer is directly shifted off the screen by the next `println`. We write the value `0x_f021_f077_f065_f04e`, which represents the string _"New!"_ on white background. As we learned [in the _“VGA Text Mode”_ post], writes to the VGA buffer should be volatile, so we use the [`write_volatile`] method.
[in the _“VGA Text Mode”_ post]: ./second-edition/posts/03-vga-text-buffer/index.md#volatile
[in the _“VGA Text Mode”_ post]: @/second-edition/posts/03-vga-text-buffer/index.md#volatile
[`write_volatile`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.write_volatile
When we run it in QEMU, we see the following output:

18
blog/content/second-edition/posts/10-heap-allocation/index.md
View File

@@ -49,7 +49,7 @@ fn inner(i: usize) -> &'static u32 {
While returning a reference makes no sense in this example, there are cases where we want a variable to live longer than the function. We already saw such a case in our kernel when we tried to [load an interrupt descriptor table] and had to use a `static` variable to extend the lifetime.
[load an interrupt descriptor table]: ./second-edition/posts/05-cpu-exceptions/index.md#loading-the-idt
[load an interrupt descriptor table]: @/second-edition/posts/05-cpu-exceptions/index.md#loading-the-idt
### Static Variables
@@ -61,14 +61,14 @@ When the `inner` function returns in the above example, it's part of the call st
Apart from the `'static` lifetime, static variables also have the useful property that their location is known at compile time, so that no reference is needed for accessing it. We utilized that property for our `println` macro: By using a [static `Writer`] internally there is no `&mut Writer` reference needed to invoke the macro, which is very useful in [exception handlers] where we don't have access to any additional variables.
[static `Writer`]: ./second-edition/posts/03-vga-text-buffer/index.md#a-global-interface
[exception handlers]: ./second-edition/posts/05-cpu-exceptions/index.md#implementation
[static `Writer`]: @/second-edition/posts/03-vga-text-buffer/index.md#a-global-interface
[exception handlers]: @/second-edition/posts/05-cpu-exceptions/index.md#implementation
However, this property of static variables brings a crucial drawback: They are read-only by default. Rust enforces this because a [data race] would occur if e.g. two threads modify a static variable at the same time. The only way to modify a static variable is to encapsulate it in a [`Mutex`] type, which ensures that only a single `&mut` reference exists at any point in time. We already used a `Mutex` for our [static VGA buffer `Writer`][vga mutex].
[data race]: https://doc.rust-lang.org/nomicon/races.html
[`Mutex`]: https://docs.rs/spin/0.5.2/spin/struct.Mutex.html
[vga mutex]: ./second-edition/posts/03-vga-text-buffer/index.md#spinlocks
[vga mutex]: @/second-edition/posts/03-vga-text-buffer/index.md#spinlocks
## Dynamic Memory
@@ -377,7 +377,7 @@ The error handler is called because the `Box::new` function implicitly calls the
Before we can create a proper allocator, we first need to create a heap memory region from which the allocator can allocate memory. To do this, we need to define a virtual memory range for the heap region and then map this region to physical frames. See the [_"Introduction To Paging"_] post for an overview of virtual memory and page tables.
[_"Introduction To Paging"_]: ./second-edition/posts/08-paging-introduction/index.md
[_"Introduction To Paging"_]: @/second-edition/posts/08-paging-introduction/index.md
The first step is to define a virtual memory region for the heap. We can choose any virtual address range that we like, as long as it is not already used for a different memory region. Let's define it as the memory starting at address `0x_4444_4444_0000` so that we can easily recognize a heap pointer later:
@@ -392,8 +392,8 @@ We set the heap size to 100 KiB for now. If we need more space in the future, we
If we tried to use this heap region now, a page fault would occur since the virtual memory region is not mapped to physical memory yet. To resolve this, we create an `init_heap` function that maps the heap pages using the [`Mapper` API] that we introduced in the [_"Paging Implementation"_] post:
[`Mapper` API]: ./second-edition/posts/09-paging-implementation/index.md#using-mappedpagetable
[_"Paging Implementation"_]: ./second-edition/posts/09-paging-implementation/index.md
[`Mapper` API]: @/second-edition/posts/09-paging-implementation/index.md#using-mappedpagetable
[_"Paging Implementation"_]: @/second-edition/posts/09-paging-implementation/index.md
```rust
// in src/allocator.rs
@@ -460,7 +460,7 @@ The implementation can be broken down into two parts:
[`Option::ok_or`]: https://doc.rust-lang.org/core/option/enum.Option.html#method.ok_or
[question mark operator]: https://doc.rust-lang.org/edition-guide/rust-2018/error-handling-and-panics/the-question-mark-operator-for-easier-error-handling.html
[`MapperFlush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html
[_translation lookaside buffer_]: ./second-edition/posts/08-paging-introduction/index.md#the-translation-lookaside-buffer
[_translation lookaside buffer_]: @/second-edition/posts/08-paging-introduction/index.md#the-translation-lookaside-buffer
[`flush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush
The final step is to call this function from our `kernel_main`:
@@ -668,7 +668,7 @@ fn panic(info: &PanicInfo) -> ! {
We reuse the `test_runner` and `test_panic_handler` functions from our `lib.rs`. Since we want to test allocations, we enable the `alloc` crate through the `extern crate alloc` statement. For more information about the test boilerplate check out the [_Testing_] post.
[_Testing_]: ./second-edition/posts/04-testing/index.md
[_Testing_]: @/second-edition/posts/04-testing/index.md
The implementation of the `main` function looks like this:

4
blog/content/second-edition/posts/deprecated/04-unit-testing/index.md
View File

@@ -25,11 +25,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
In this post we explore how to execute `cargo test` on the host system (as a normal Linux/Windows/macOS executable). This only works if you don't have a `.cargo/config` file that sets a default target. If you followed the [_Minimal Rust Kernel_] post before 2019-04-27, you should be fine. If you followed it after that date, you need to remove the `build.target` key from your `.cargo/config` file and explicitly pass a target argument to `cargo xbuild`.
[_Minimal Rust Kernel_]: ./second-edition/posts/02-minimal-rust-kernel/index.md
[_Minimal Rust Kernel_]: @/second-edition/posts/02-minimal-rust-kernel/index.md
Alternatively, consider reading the new [_Testing_] post instead. It sets up a similar functionality as this post, but instead of running the tests on your host system, they are run in a realistic environment inside QEMU.
[_Testing_]: ./second-edition/posts/04-testing/index.md
[_Testing_]: @/second-edition/posts/04-testing/index.md
## Unit Tests for `no_std` Binaries
Rust has a [built-in test framework] that is capable of running unit tests without the need to set anything up. Just create a function that checks some results through assertions and add the `#[test]` attribute to the function header. Then `cargo test` will automatically find and execute all test functions of your crate.

8
blog/content/second-edition/posts/deprecated/05-integration-tests/index.md
View File

@@ -25,8 +25,8 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
This post builds upon the [_Unit Testing_] post, so you need to follow it first. Alternatively, consider reading the new [_Testing_] post instead, which replaces both _Unit Testing_ and this post. The new posts implements similar functionality, but integrates it directly in `cargo xtest`, so that both unit and integration tests run in a realistic environment inside QEMU.
[_Unit Testing_]: ./second-edition/posts/deprecated/04-unit-testing/index.md
[_Testing_]: ./second-edition/posts/04-testing/index.md
[_Unit Testing_]: @/second-edition/posts/deprecated/04-unit-testing/index.md
[_Testing_]: @/second-edition/posts/04-testing/index.md
## Overview
@@ -60,7 +60,7 @@ The chips implementing a serial interface are called [UARTs]. There are [lots of
### Port I/O
There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the GPU.
[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md
[VGA text buffer]: @/second-edition/posts/03-vga-text-buffer/index.md
In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`).
@@ -105,7 +105,7 @@ lazy_static! {
Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static`. However, this time we use `lazy_static` to ensure that the `init` method is called before first use. We're using the port address `0x3F8`, which is the standard port number for the first serial interface.
[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
[vga lazy-static]: @/second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros:

10
blog/content/second-edition/posts/deprecated/10-advanced-paging/index.md
View File

@@ -23,11 +23,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
In the [previous post] we learned about the principles of paging and how the 4-level page tables on x86_64 work. We also found out that the bootloader already set up a page table hierarchy for our kernel, which means that our kernel already runs on virtual addresses. This improves safety since illegal memory accesses cause page fault exceptions instead of modifying arbitrary physical memory.
[previous post]: ./second-edition/posts/08-paging-introduction/index.md
[previous post]: @/second-edition/posts/08-paging-introduction/index.md
However, it also causes a problem when we try to access the page tables from our kernel because we can't directly access the physical addresses that are stored in page table entries or the `CR3` register. We experienced that problem already [at the end of the previous post] when we tried to inspect the active page tables.
[at the end of the previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
[at the end of the previous post]: @/second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
The next section discusses the problem in detail and provides different approaches to a solution. Afterward, we implement a function that traverses the page table hierarchy in order to translate virtual to physical addresses. Finally, we learn how to create new mappings in the page tables and how to find unused memory frames for creating new page tables.
@@ -63,7 +63,7 @@ So in order to access page table frames, we need to map some virtual pages to th
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000 KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `28 KiB` because it would collide with the already mapped page at `1004 MiB`. So we have to look further until we find a large enough unmapped area, for example at `1008 KiB`. This is a similar fragmentation problem as with [segmentation].
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
[segmentation]: ./second-edition/posts/08-paging-introduction/index.md#fragmentation
[segmentation]: @/second-edition/posts/08-paging-introduction/index.md#fragmentation
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000 KiB memory region starting at `1008 KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000 KiB` and `2008 KiB` anymore, because we can't identity map it.
@@ -156,7 +156,7 @@ Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 i
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
[sign extension]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86
[sign extension]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86
We use [octal] numbers for representing the addresses since each octal character represents three bits, which allows us to clearly separate the 9-bit indexes of the different page table levels. This isn't possible with the hexadecimal system where each character represents four bits.
@@ -505,7 +505,7 @@ We first create the mapping for the page at `0x1000` by calling our `create_exam
Then we write the value `0xf021_f077_f065_f04e` to this page, which represents the string _"New!"_ on white background. We don't write directly to the beginning of the page at `0x1000` since the top line is directly shifted off the screen by the next `println`. Instead, we write to offset `0x900`, which is about in the middle of the screen. As we learned [in the _“VGA Text Mode”_ post], writes to the VGA buffer should be volatile, so we use the [`write_volatile`] method.
[in the _“VGA Text Mode”_ post]: ./second-edition/posts/03-vga-text-buffer/index.md#volatile
[in the _“VGA Text Mode”_ post]: @/second-edition/posts/03-vga-text-buffer/index.md#volatile
[`write_volatile`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.write_volatile
When we run it in QEMU, we see the following output:

2
blog/content/status-update/2019-07-06.md
View File

@@ -6,7 +6,7 @@ aliases = ["status-update/2019-06-04/index.html"]
This post gives an overview of the recent updates to the _Writing an OS in Rust_ blog and the used libraries and tools.
My focus this month was to finish the [_Heap Allocation_](./second-edition/posts/10-heap-allocation/index.md) post, on which I had been working since March. I originally wanted to include a section about different allocator designs (bump, linked list, slab, …) and how to implement them, but I decided to split it out into a separate post because it became much too long. I try to release this half-done post soon.
My focus this month was to finish the [_Heap Allocation_](@/second-edition/posts/10-heap-allocation/index.md) post, on which I had been working since March. I originally wanted to include a section about different allocator designs (bump, linked list, slab, …) and how to implement them, but I decided to split it out into a separate post because it became much too long. I try to release this half-done post soon.
Apart from the new post, there were some minor updates to the `x86_64`, `bootloader` and `cargo-xbuild` crates. The following gives a short overview of notable changes to the different projects.

2
blog/templates/first-edition/index.html
View File

@@ -21,7 +21,7 @@
</div>
<div class="warning">
<b>No longer updated!</b> You are viewing the first edition of “Writing an OS in Rust”, which is no longer updated. You can find the second edition <a href="{{ get_url(path = "./second-edition/_index.md") }}">here</a>.
<b>No longer updated!</b> You are viewing the first edition of “Writing an OS in Rust”, which is no longer updated. You can find the second edition <a href="{{ get_url(path = "@/second-edition/_index.md") }}">here</a>.
</div>
<div id="bare-bones" class="post-category bare-bones">Bare Bones</div>

4
blog/templates/first-edition/page.html
View File

@@ -26,7 +26,7 @@
</aside>
<div class="warning">
<b>No longer updated!</b> You are viewing the a post of the first edition of “Writing an OS in Rust”, which is no longer updated. You can find the second edition <a href="{{ get_url(path = "./second-edition/_index.md") }}">here</a>.
<b>No longer updated!</b> You are viewing the a post of the first edition of “Writing an OS in Rust”, which is no longer updated. You can find the second edition <a href="{{ get_url(path = "@/second-edition/_index.md") }}">here</a>.
</div>
{{ page.content | safe }}
@@ -50,5 +50,3 @@
</section>
{% endblock after_main %}