Update to new internal link syntax

This commit is contained in:
Philipp Oppermann
2019-09-29 15:24:21 +02:00
parent 0b971c9128
commit a133ea82cf
29 changed files with 117 additions and 119 deletions

View File

@@ -513,4 +513,4 @@ Note that this is just a minimal example of a freestanding Rust binary. This bin
The [next post] explains the steps needed for turning our freestanding binary into a minimal operating system kernel. This includes creating a custom target, combining our executable with a bootloader, and learning how to print something to the screen.
[next post]: ./second-edition/posts/02-minimal-rust-kernel/index.md
[next post]: @/second-edition/posts/02-minimal-rust-kernel/index.md

View File

@@ -8,7 +8,7 @@ date = 2018-02-10
In this post we create a minimal 64-bit Rust kernel for the x86 architecture. We build upon the [freestanding Rust binary] from the previous post to create a bootable disk image, that prints something to the screen.
[freestanding Rust binary]: ./second-edition/posts/01-freestanding-rust-binary/index.md
[freestanding Rust binary]: @/second-edition/posts/01-freestanding-rust-binary/index.md
<!-- more -->
@@ -74,7 +74,7 @@ To make a kernel Multiboot compliant, one just needs to insert a so-called [Mult
Because of these drawbacks we decided to not use GRUB or the Multiboot standard. However, we plan to add Multiboot support to our [bootimage] tool, so that it's possible to load your kernel on a GRUB system too. If you're interested in writing a Multiboot compliant kernel, check out the [first edition] of this blog series.
[first edition]: ./first-edition/_index.md
[first edition]: @/first-edition/_index.md
### UEFI
@@ -169,7 +169,7 @@ This setting specifies that the target doesn't support [stack unwinding] on pani
We're writing a kernel, so we'll need to handle interrupts at some point. To do that safely, we have to disable a certain stack pointer optimization called the _“red zone”_, because it would cause stack corruptions otherwise. For more information, see our separate post about [disabling the red zone].
[disabling the red zone]: ./second-edition/extra/disable-red-zone/index.md
[disabling the red zone]: @/second-edition/extra/disable-red-zone/index.md
```json
"features": "-mmx,-sse,+soft-float",
@@ -183,7 +183,7 @@ The `mmx` and `sse` features determine support for [Single Instruction Multiple
A problem with disabling SIMD is that floating point operations on `x86_64` require SIMD registers by default. To solve this problem, we add the `soft-float` feature, which emulates all floating point operations through software functions based on normal integers.
For more information, see our post on [disabling SIMD](./second-edition/extra/disable-simd/index.md).
For more information, see our post on [disabling SIMD](@/second-edition/extra/disable-simd/index.md).
#### Putting it Together
Our target specification file now looks like this:
@@ -209,7 +209,7 @@ Our target specification file now looks like this:
### Building our Kernel
Compiling for our new target will use Linux conventions (I'm not quite sure why, I assume that it's just LLVM's default). This means that we need an entry point named `_start` as described in the [previous post]:
[previous post]: ./second-edition/posts/01-freestanding-rust-binary/index.md
[previous post]: @/second-edition/posts/01-freestanding-rust-binary/index.md
```rust
// src/main.rs
@@ -275,7 +275,7 @@ Now we can rerun the above command with `xbuild` instead of `build`:
We see that `cargo xbuild` cross-compiles the `core`, `compiler_builtin`, and `alloc` libraries for our new custom target. Since these libraries use a lot of unstable features internally, this only works with a [nightly Rust compiler]. Afterwards, `cargo xbuild` successfully compiles our `blog_os` crate.
[nightly Rust compiler]: ./second-edition/posts/01-freestanding-rust-binary/index.md#installing-rust-nightly
[nightly Rust compiler]: @/second-edition/posts/01-freestanding-rust-binary/index.md#installing-rust-nightly
Now we are able to build our kernel for a bare metal target. However, our `_start` entry point, which will be called by the boot loader, is still empty. So let's output something to screen from it.

View File

@@ -22,11 +22,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
This post replaces the (now deprecated) [_Unit Testing_] and [_Integration Tests_] posts. It assumes that you have followed the [_A Minimal Rust Kernel_] post after 2019-04-27. Mainly, it requires that you have a `.cargo/config` file that [sets a default target] and [defines a runner executable].
[_Unit Testing_]: ./second-edition/posts/deprecated/04-unit-testing/index.md
[_Integration Tests_]: ./second-edition/posts/deprecated/05-integration-tests/index.md
[_A Minimal Rust Kernel_]: ./second-edition/posts/02-minimal-rust-kernel/index.md
[sets a default target]: ./second-edition/posts/02-minimal-rust-kernel/index.md#set-a-default-target
[defines a runner executable]: ./second-edition/posts/02-minimal-rust-kernel/index.md#using-cargo-run
[_Unit Testing_]: @/second-edition/posts/deprecated/04-unit-testing/index.md
[_Integration Tests_]: @/second-edition/posts/deprecated/05-integration-tests/index.md
[_A Minimal Rust Kernel_]: @/second-edition/posts/02-minimal-rust-kernel/index.md
[sets a default target]: @/second-edition/posts/02-minimal-rust-kernel/index.md#set-a-default-target
[defines a runner executable]: @/second-edition/posts/02-minimal-rust-kernel/index.md#using-cargo-run
## Testing in Rust
@@ -150,7 +150,7 @@ Together with the device name (`isa-debug-exit`), we pass the two parameters `io
There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the VGA device.
[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md
[VGA text buffer]: @/second-edition/posts/03-vga-text-buffer/index.md
In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`).
@@ -302,7 +302,7 @@ Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spi
Like the `isa-debug-exit` device, the UART is programmed using port I/O. Since the UART is more complex, it uses multiple I/O ports for programming different device registers. The unsafe `SerialPort::new` function expects the address of the first I/O port of the UART as argument, from which it can calculate the addresses of all needed ports. We're passing the port address `0x3F8`, which is the standard port number for the first serial interface.
[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
[vga lazy-static]: @/second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros:

View File

@@ -349,7 +349,7 @@ Fortunately the `lazy_static` macro exists. Instead of evaluating a `static` at
We already imported the `lazy_static` crate when we [created an abstraction for the VGA text buffer][vga text buffer lazy static]. So we can directly use the `lazy_static!` macro to create our static IDT:
[vga text buffer lazy static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
[vga text buffer lazy static]: @/second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
```rust
// in src/interrupts.rs
@@ -454,16 +454,16 @@ fn test_breakpoint_exception() {
Apart from printing status messages through the [serial port], the test invokes the `int3` function to trigger a breakpoint exception. By checking that the execution continues afterwards, we verify that our breakpoint handler is working correctly.
[serial port]: ./second-edition/posts/04-testing/index.md#serial-port
[serial port]: @/second-edition/posts/04-testing/index.md#serial-port
You can try this new test by running `cargo xtest` (all tests) or `cargo xtest --lib` (only tests of `lib.rs` and its modules). You should see `test_breakpoint_exception...[ok]` in the output.
## Too much Magic?
The `x86-interrupt` calling convention and the [`InterruptDescriptorTable`] type made the exception handling process relatively straightforward and painless. If this was too much magic for you and you like to learn all the gory details of exception handling, we got you covered: Our [“Handling Exceptions with Naked Functions”] series shows how to handle exceptions without the `x86-interrupt` calling convention and also creates its own IDT type. Historically, these posts were the main exception handling posts before the `x86-interrupt` calling convention and the `x86_64` crate existed. Note that these posts are based on the [first edition] of this blog and might be out of date.
[“Handling Exceptions with Naked Functions”]: ./first-edition/extra/naked-exceptions/_index.md
[“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md
[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html
[first edition]: ./first-edition/_index.md
[first edition]: @/first-edition/_index.md
## What's next?
We've successfully caught our first exception and returned from it! The next step is to ensure that we catch all exceptions, because an uncaught exception causes a fatal [triple fault], which leads to a system reset. The next post explains how we can avoid this by correctly catching [double faults].

View File

@@ -21,7 +21,7 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
## What is a Double Fault?
In simplified terms, a double fault is a special exception that occurs when the CPU fails to invoke an exception handler. For example, it occurs when a page fault is triggered but there is no page fault handler registered in the [Interrupt Descriptor Table][IDT] (IDT). So it's kind of similar to catch-all blocks in programming languages with exceptions, e.g. `catch(...)` in C++ or `catch(Exception e)` in Java or C#.
[IDT]: ./second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
[IDT]: @/second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
A double fault behaves like a normal exception. It has the vector number `8` and we can define a normal handler function for it in the IDT. It is really important to provide a double fault handler, because if a double fault is unhandled a fatal _triple fault_ occurs. Triple faults can't be caught and most hardware reacts with a system reset.
@@ -196,7 +196,7 @@ struct InterruptStackTable {
For each exception handler, we can choose a stack from the IST through the `options` field in the corresponding [IDT entry]. For example, we could use the first stack in the IST for our double fault handler. Then the CPU would automatically switch to this stack whenever a double fault occurs. This switch would happen before anything is pushed, so it would prevent the triple fault.
[IDT entry]: ./second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
[IDT entry]: @/second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
### The IST and TSS
The Interrupt Stack Table (IST) is part of an old legacy structure called _[Task State Segment]_ \(TSS). The TSS used to hold various information (e.g. processor register state) about a task in 32-bit mode and was for example used for [hardware context switching]. However, hardware context switching is no longer supported in 64-bit mode and the format of the TSS changed completely.
@@ -438,7 +438,7 @@ name = "stack_overflow"
harness = false
```
[without a test harness]: ./second-edition/posts/04-testing/index.md#no-harness
[without a test harness]: @/second-edition/posts/04-testing/index.md#no-harness
Now `cargo xtest --test stack_overflow` should compile successfully. The test fails of course, since the `unimplemented` macro panics.

View File

@@ -66,7 +66,7 @@ This graphic shows the typical assignment of interrupt lines. We see that most o
Each controller can be configured through two [I/O ports], one “command” port and one “data” port. For the primary controller these ports are `0x20` (command) and `0x21` (data). For the secondary controller they are `0xa0` (command) and `0xa1` (data). For more information on how the PICs can be configured see the [article on osdev.org].
[I/O ports]: ./second-edition/posts/04-testing/index.md#i-o-ports
[I/O ports]: @/second-edition/posts/04-testing/index.md#i-o-ports
[article on osdev.org]: https://wiki.osdev.org/8259_PIC
### Implementation
@@ -255,7 +255,7 @@ We now have a form of concurrency in our kernel: The timer interrupts occur asyn
We can already provoke a deadlock in our kernel. Remember, our `println` macro calls the `vga_buffer::_print` function, which [locks a global `WRITER`][vga spinlock] using a spinlock:
[vga spinlock]: ./second-edition/posts/03-vga-text-buffer/index.md#spinlocks
[vga spinlock]: @/second-edition/posts/03-vga-text-buffer/index.md#spinlocks
```rust
// in src/vga_buffer.rs
@@ -579,7 +579,7 @@ We now see that a `k` appears on the screen when we press a key. However, this o
To find out _which_ key was pressed, we need to query the keyboard controller. We do this by reading from the data port of the PS/2 controller, which is the [I/O port] with number `0x60`:
[I/O port]: ./second-edition/posts/04-testing/index.md#i-o-ports
[I/O port]: @/second-edition/posts/04-testing/index.md#i-o-ports
```rust
// in src/interrupts.rs

View File

@@ -247,7 +247,7 @@ It is important to remember flushing the TLB on each page table modification bec
One thing that we did not mention yet: **Our kernel already runs on paging**. The bootloader that we added in the ["A minimal Rust Kernel"] post already set up a 4-level paging hierarchy that maps every page of our kernel to a physical frame. The bootloader does this because paging is mandatory in 64-bit mode on x86_64.
["A minimal Rust kernel"]: ./second-edition/posts/02-minimal-rust-kernel/index.md#creating-a-bootimage
["A minimal Rust kernel"]: @/second-edition/posts/02-minimal-rust-kernel/index.md#creating-a-bootimage
This means that every memory address that we used in our kernel was a virtual address. Accessing the VGA buffer at address `0xb8000` only worked because the bootloader _identity mapped_ that memory page, which means that it mapped the virtual page `0xb8000` to the physical frame `0xb8000`.
@@ -257,7 +257,7 @@ Paging makes our kernel already relatively safe, since every memory access that
Let's try to cause a page fault by accessing some memory outside of our kernel. First, we create a page fault handler and register it in our IDT, so that we see a page fault exception instead of a generic [double fault] :
[double fault]: ./second-edition/posts/06-double-faults/index.md
[double fault]: @/second-edition/posts/06-double-faults/index.md
```rust
// in src/interrupts.rs
@@ -296,7 +296,7 @@ The [`CR2`] register is automatically set by the CPU on a page fault and contain
[`Cr2::read`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr2.html#method.read
[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html
[LLVM bug]: https://github.com/rust-lang/rust/issues/57270
[`hlt_loop`]: ./second-edition/posts/07-hardware-interrupts/index.md#the
[`hlt_loop`]: @/second-edition/posts/07-hardware-interrupts/index.md#the
Now we can try to access some memory outside our kernel:

View File

@@ -21,11 +21,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
The [previous post] gave an introduction to the concept of paging. It motivated paging by comparing it with segmentation, explained how paging and page tables work, and then introduced the 4-level page table design of `x86_64`. We found out that the bootloader already set up a page table hierarchy for our kernel, which means that our kernel already runs on virtual addresses. This improves safety since illegal memory accesses cause page fault exceptions instead of modifying arbitrary physical memory.
[previous post]: ./second-edition/posts/08-paging-introduction/index.md
[previous post]: @/second-edition/posts/08-paging-introduction/index.md
The post ended with the problem that we [can't access the page tables from our kernel][end of previous post] because they are stored in physical memory and our kernel already runs on virtual addresses. This post continues at this point and explores different approaches of making the page table frames accessible to our kernel. We will discuss the advantages and drawbacks of each approach and then decide for an approach for our kernel.
[end of previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
[end of previous post]: @/second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
To implement the approach, we will need support from the bootloader, so we'll configure it first. Afterward, we will implement a function that traverses the page table hierarchy in order to translate virtual to physical addresses. Finally, we learn how to create new mappings in the page tables and how to find unused memory frames for creating new page tables.
@@ -65,7 +65,7 @@ In this example, we see various identity-mapped page table frames. This way the
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `28KiB` because it would collide with the already mapped page at `1004KiB`. So we have to look further until we find a large enough unmapped area, for example at `1008KiB`. This is a similar fragmentation problem as with [segmentation].
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
[segmentation]: ./second-edition/posts/08-paging-introduction/index.md#fragmentation
[segmentation]: @/second-edition/posts/08-paging-introduction/index.md#fragmentation
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000KiB memory region starting at `1008KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000KiB` and `2008KiB` anymore, because we can't identity map it.
@@ -192,7 +192,7 @@ Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 i
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
[sign extension]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86
[sign extension]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86
We use [octal] numbers for representing the addresses since each octal character represents three bits, which allows us to clearly separate the 9-bit indexes of the different page table levels. This isn't possible with the hexadecimal system where each character represents four bits.
@@ -380,7 +380,7 @@ For the module we create an empty `src/memory.rs` file.
At the [end of the previous post], we tried to take a look at the page tables our kernel runs on, but failed since we couldn't access the physical frame that the `CR3` register points to. We're now able to continue from there by creating an `active_level_4_table` function that returns a reference to the active level 4 page table:
[end of the previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
[end of the previous post]: @/second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
```rust
// in src/memory.rs
@@ -603,7 +603,7 @@ When we run it, we see the following output:
As expected, the identity-mapped address `0xb8000` translates to the same physical address. The code page and the stack page translate to some arbitrary physical addresses, which depend on how the bootloader created the initial mapping for our kernel. It's worth noting that the last 12 bits always stay the same after translation, which makes sense because these bits are the [_page offset_] and not part of the translation.
[_page offset_]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
[_page offset_]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
Since each physical address can be accessed by adding the `physical_memory_offset`, the translation of the `physical_memory_offset` address itself should point to physical address `0`. However, the translation fails because the mapping uses huge pages for efficiency, which is not supported in our implementation yet.
@@ -748,7 +748,7 @@ In addition to the `page` that should be mapped, the function expects a mutable
For the mapping, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. Calling [`map_to`] is unsafe because it's possible to break memory safety with invalid arguments, so we need to use an `unsafe` block. For a list of all possible flags, see the [_Page Table Format_] section of the previous post.
[_Page Table Format_]: ./second-edition/posts/08-paging-introduction/index.md#page-table-format
[_Page Table Format_]: @/second-edition/posts/08-paging-introduction/index.md#page-table-format
The [`map_to`] function can fail, so it returns a [`Result`]. Since this is just some example code that does not need to be robust, we just use [`expect`] to panic when an error occurs. On success, the function returns a [`MapperFlush`] type that provides an easy way to flush the newly mapped page from the translation lookaside buffer (TLB) with its [`flush`] method. Like `Result`, the type uses the [`#[must_use]`][must_use] attribute to emit a warning when we accidentally forget to use it.
@@ -791,7 +791,7 @@ Additionally, the graphic shows the physical frame of the VGA text buffer in red
The graphic shows two canditate pages in the virtual address space, both marked in yellow. One page is at address `0x803fdfd000`, which is 3 pages before the mapped page (in blue). While the level 4 and level 3 page table indices are the same as for the blue page, the level 2 and level 1 indices are different (see the [previous post][page-table-indices]). The different index into the level 2 table means that a different level 1 table is used for this page. Since this level 1 table does not exist yet, we would need to create it if we chose that page for our example mapping, which would require an additional unused physical frame. In contrast, the second candidate page at address `0x803fe02000` does not have this problem because it uses the same level 1 page table than the blue page. Thus, all required page tables already exist.
[page-table-indices]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
[page-table-indices]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86-64
In summary, the difficulty of creating a new mapping depends on the virtual page that we want to map. In the easiest case, the level 1 page table for the page already exists and we just need to write a single entry. In the most difficult case, the page is in a memory region for that no level 3 exists yet so that we need to create new level 3, level 2 and level 1 page tables first.
@@ -830,7 +830,7 @@ We first create the mapping for the page at address `0` by calling our `create_e
Then we convert the page to a raw pointer and write a value to offset `400`. We don't write to the start of the page because the top line of the VGA buffer is directly shifted off the screen by the next `println`. We write the value `0x_f021_f077_f065_f04e`, which represents the string _"New!"_ on white background. As we learned [in the _“VGA Text Mode”_ post], writes to the VGA buffer should be volatile, so we use the [`write_volatile`] method.
[in the _“VGA Text Mode”_ post]: ./second-edition/posts/03-vga-text-buffer/index.md#volatile
[in the _“VGA Text Mode”_ post]: @/second-edition/posts/03-vga-text-buffer/index.md#volatile
[`write_volatile`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.write_volatile
When we run it in QEMU, we see the following output:

View File

@@ -49,7 +49,7 @@ fn inner(i: usize) -> &'static u32 {
While returning a reference makes no sense in this example, there are cases where we want a variable to live longer than the function. We already saw such a case in our kernel when we tried to [load an interrupt descriptor table] and had to use a `static` variable to extend the lifetime.
[load an interrupt descriptor table]: ./second-edition/posts/05-cpu-exceptions/index.md#loading-the-idt
[load an interrupt descriptor table]: @/second-edition/posts/05-cpu-exceptions/index.md#loading-the-idt
### Static Variables
@@ -61,14 +61,14 @@ When the `inner` function returns in the above example, it's part of the call st
Apart from the `'static` lifetime, static variables also have the useful property that their location is known at compile time, so that no reference is needed for accessing it. We utilized that property for our `println` macro: By using a [static `Writer`] internally there is no `&mut Writer` reference needed to invoke the macro, which is very useful in [exception handlers] where we don't have access to any additional variables.
[static `Writer`]: ./second-edition/posts/03-vga-text-buffer/index.md#a-global-interface
[exception handlers]: ./second-edition/posts/05-cpu-exceptions/index.md#implementation
[static `Writer`]: @/second-edition/posts/03-vga-text-buffer/index.md#a-global-interface
[exception handlers]: @/second-edition/posts/05-cpu-exceptions/index.md#implementation
However, this property of static variables brings a crucial drawback: They are read-only by default. Rust enforces this because a [data race] would occur if e.g. two threads modify a static variable at the same time. The only way to modify a static variable is to encapsulate it in a [`Mutex`] type, which ensures that only a single `&mut` reference exists at any point in time. We already used a `Mutex` for our [static VGA buffer `Writer`][vga mutex].
[data race]: https://doc.rust-lang.org/nomicon/races.html
[`Mutex`]: https://docs.rs/spin/0.5.2/spin/struct.Mutex.html
[vga mutex]: ./second-edition/posts/03-vga-text-buffer/index.md#spinlocks
[vga mutex]: @/second-edition/posts/03-vga-text-buffer/index.md#spinlocks
## Dynamic Memory
@@ -377,7 +377,7 @@ The error handler is called because the `Box::new` function implicitly calls the
Before we can create a proper allocator, we first need to create a heap memory region from which the allocator can allocate memory. To do this, we need to define a virtual memory range for the heap region and then map this region to physical frames. See the [_"Introduction To Paging"_] post for an overview of virtual memory and page tables.
[_"Introduction To Paging"_]: ./second-edition/posts/08-paging-introduction/index.md
[_"Introduction To Paging"_]: @/second-edition/posts/08-paging-introduction/index.md
The first step is to define a virtual memory region for the heap. We can choose any virtual address range that we like, as long as it is not already used for a different memory region. Let's define it as the memory starting at address `0x_4444_4444_0000` so that we can easily recognize a heap pointer later:
@@ -392,8 +392,8 @@ We set the heap size to 100 KiB for now. If we need more space in the future, we
If we tried to use this heap region now, a page fault would occur since the virtual memory region is not mapped to physical memory yet. To resolve this, we create an `init_heap` function that maps the heap pages using the [`Mapper` API] that we introduced in the [_"Paging Implementation"_] post:
[`Mapper` API]: ./second-edition/posts/09-paging-implementation/index.md#using-mappedpagetable
[_"Paging Implementation"_]: ./second-edition/posts/09-paging-implementation/index.md
[`Mapper` API]: @/second-edition/posts/09-paging-implementation/index.md#using-mappedpagetable
[_"Paging Implementation"_]: @/second-edition/posts/09-paging-implementation/index.md
```rust
// in src/allocator.rs
@@ -460,7 +460,7 @@ The implementation can be broken down into two parts:
[`Option::ok_or`]: https://doc.rust-lang.org/core/option/enum.Option.html#method.ok_or
[question mark operator]: https://doc.rust-lang.org/edition-guide/rust-2018/error-handling-and-panics/the-question-mark-operator-for-easier-error-handling.html
[`MapperFlush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html
[_translation lookaside buffer_]: ./second-edition/posts/08-paging-introduction/index.md#the-translation-lookaside-buffer
[_translation lookaside buffer_]: @/second-edition/posts/08-paging-introduction/index.md#the-translation-lookaside-buffer
[`flush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush
The final step is to call this function from our `kernel_main`:
@@ -668,7 +668,7 @@ fn panic(info: &PanicInfo) -> ! {
We reuse the `test_runner` and `test_panic_handler` functions from our `lib.rs`. Since we want to test allocations, we enable the `alloc` crate through the `extern crate alloc` statement. For more information about the test boilerplate check out the [_Testing_] post.
[_Testing_]: ./second-edition/posts/04-testing/index.md
[_Testing_]: @/second-edition/posts/04-testing/index.md
The implementation of the `main` function looks like this:

View File

@@ -25,11 +25,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
In this post we explore how to execute `cargo test` on the host system (as a normal Linux/Windows/macOS executable). This only works if you don't have a `.cargo/config` file that sets a default target. If you followed the [_Minimal Rust Kernel_] post before 2019-04-27, you should be fine. If you followed it after that date, you need to remove the `build.target` key from your `.cargo/config` file and explicitly pass a target argument to `cargo xbuild`.
[_Minimal Rust Kernel_]: ./second-edition/posts/02-minimal-rust-kernel/index.md
[_Minimal Rust Kernel_]: @/second-edition/posts/02-minimal-rust-kernel/index.md
Alternatively, consider reading the new [_Testing_] post instead. It sets up a similar functionality as this post, but instead of running the tests on your host system, they are run in a realistic environment inside QEMU.
[_Testing_]: ./second-edition/posts/04-testing/index.md
[_Testing_]: @/second-edition/posts/04-testing/index.md
## Unit Tests for `no_std` Binaries
Rust has a [built-in test framework] that is capable of running unit tests without the need to set anything up. Just create a function that checks some results through assertions and add the `#[test]` attribute to the function header. Then `cargo test` will automatically find and execute all test functions of your crate.

View File

@@ -25,8 +25,8 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
This post builds upon the [_Unit Testing_] post, so you need to follow it first. Alternatively, consider reading the new [_Testing_] post instead, which replaces both _Unit Testing_ and this post. The new posts implements similar functionality, but integrates it directly in `cargo xtest`, so that both unit and integration tests run in a realistic environment inside QEMU.
[_Unit Testing_]: ./second-edition/posts/deprecated/04-unit-testing/index.md
[_Testing_]: ./second-edition/posts/04-testing/index.md
[_Unit Testing_]: @/second-edition/posts/deprecated/04-unit-testing/index.md
[_Testing_]: @/second-edition/posts/04-testing/index.md
## Overview
@@ -60,7 +60,7 @@ The chips implementing a serial interface are called [UARTs]. There are [lots of
### Port I/O
There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the GPU.
[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md
[VGA text buffer]: @/second-edition/posts/03-vga-text-buffer/index.md
In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`).
@@ -105,7 +105,7 @@ lazy_static! {
Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static`. However, this time we use `lazy_static` to ensure that the `init` method is called before first use. We're using the port address `0x3F8`, which is the standard port number for the first serial interface.
[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
[vga lazy-static]: @/second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros:

View File

@@ -23,11 +23,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
In the [previous post] we learned about the principles of paging and how the 4-level page tables on x86_64 work. We also found out that the bootloader already set up a page table hierarchy for our kernel, which means that our kernel already runs on virtual addresses. This improves safety since illegal memory accesses cause page fault exceptions instead of modifying arbitrary physical memory.
[previous post]: ./second-edition/posts/08-paging-introduction/index.md
[previous post]: @/second-edition/posts/08-paging-introduction/index.md
However, it also causes a problem when we try to access the page tables from our kernel because we can't directly access the physical addresses that are stored in page table entries or the `CR3` register. We experienced that problem already [at the end of the previous post] when we tried to inspect the active page tables.
[at the end of the previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
[at the end of the previous post]: @/second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
The next section discusses the problem in detail and provides different approaches to a solution. Afterward, we implement a function that traverses the page table hierarchy in order to translate virtual to physical addresses. Finally, we learn how to create new mappings in the page tables and how to find unused memory frames for creating new page tables.
@@ -63,7 +63,7 @@ So in order to access page table frames, we need to map some virtual pages to th
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `28KiB` because it would collide with the already mapped page at `1004MiB`. So we have to look further until we find a large enough unmapped area, for example at `1008KiB`. This is a similar fragmentation problem as with [segmentation].
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
[segmentation]: ./second-edition/posts/08-paging-introduction/index.md#fragmentation
[segmentation]: @/second-edition/posts/08-paging-introduction/index.md#fragmentation
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000KiB memory region starting at `1008KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000KiB` and `2008KiB` anymore, because we can't identity map it.
@@ -156,7 +156,7 @@ Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 i
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
[sign extension]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86
[sign extension]: @/second-edition/posts/08-paging-introduction/index.md#paging-on-x86
We use [octal] numbers for representing the addresses since each octal character represents three bits, which allows us to clearly separate the 9-bit indexes of the different page table levels. This isn't possible with the hexadecimal system where each character represents four bits.
@@ -505,7 +505,7 @@ We first create the mapping for the page at `0x1000` by calling our `create_exam
Then we write the value `0xf021_f077_f065_f04e` to this page, which represents the string _"New!"_ on white background. We don't write directly to the beginning of the page at `0x1000` since the top line is directly shifted off the screen by the next `println`. Instead, we write to offset `0x900`, which is about in the middle of the screen. As we learned [in the _“VGA Text Mode”_ post], writes to the VGA buffer should be volatile, so we use the [`write_volatile`] method.
[in the _“VGA Text Mode”_ post]: ./second-edition/posts/03-vga-text-buffer/index.md#volatile
[in the _“VGA Text Mode”_ post]: @/second-edition/posts/03-vga-text-buffer/index.md#volatile
[`write_volatile`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.write_volatile
When we run it in QEMU, we see the following output: