mirror of
https://github.com/phil-opp/blog_os.git
synced 2025-12-16 14:27:49 +00:00
302 lines
22 KiB
Markdown
302 lines
22 KiB
Markdown
+++
|
||
title = "Advanced Paging"
|
||
order = 10
|
||
path = "advanced-paging"
|
||
date = 0000-01-01
|
||
template = "second-edition/page.html"
|
||
+++
|
||
|
||
This post TODO
|
||
|
||
<!-- more -->
|
||
|
||
This blog is openly developed on [Github]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom].
|
||
|
||
[Github]: https://github.com/phil-opp/blog_os
|
||
[at the bottom]: #comments
|
||
|
||
## Introduction
|
||
|
||
In the [previous post] we learned about the principles of paging and how the 4-level page tables on the x86_64 architecture work. We also found out that the bootloader already set up a 4-level page table hierarchy for our kernel, since paging is mandatory on x86_64 in 64-bit mode. This means that our kernel already runs on virtual addresses.
|
||
|
||
[previous post]: ./second-edition/posts/09-paging-introduction/index.md
|
||
|
||
This makes our kernel much safer, since every memory access that is out of bounds causes a page fault exception instead of writing to random physical memory. The bootloader even set the correct access permissions for each page, which means that only the pages containing code are executable and only data pages are writable.
|
||
|
||
### Page Faults
|
||
|
||
Let's try to cause a page fault by accessing some memory outside of our kernel! First, we create a page fault handler and register it in our IDT, so that we see a page fault exception instead of a generic [double fault] :
|
||
|
||
[double fault]: ./second-edition/posts/07-double-faults/index.md
|
||
|
||
```rust
|
||
// in src/interrupts.rs
|
||
|
||
lazy_static! {
|
||
static ref IDT: InterruptDescriptorTable = {
|
||
let mut idt = InterruptDescriptorTable::new();
|
||
|
||
[…]
|
||
|
||
idt.page_fault.set_handler_fn(page_fault_handler); // new
|
||
|
||
idt
|
||
};
|
||
}
|
||
|
||
use x86_64::structures::idt::PageFaultErrorCode;
|
||
|
||
extern "x86-interrupt" fn page_fault_handler(
|
||
stack_frame: &mut ExceptionStackFrame,
|
||
_error_code: PageFaultErrorCode,
|
||
) {
|
||
use crate::hlt_loop;
|
||
use x86_64::registers::control::Cr2;
|
||
|
||
println!("EXCEPTION: PAGE FAULT");
|
||
println!("Accessed Address: {:?}", Cr2::read());
|
||
println!("{:#?}", stack_frame);
|
||
hlt_loop();
|
||
}
|
||
```
|
||
|
||
The [`CR2`] register is automatically set by the CPU on a page fault and contains the accessed virtual address that caused the page fault. We use the [`Cr2::read`] function of the `x86_64` crate to read and print it. Normally the [`PageFaultErrorCode`] type would provide more information about the type of memory access that caused the page fault, but there is currently an [LLVM bug] that passes an invalid error code, so we ignore it for now. We can't continue execution without resolving the page fault, so we enter a [`hlt_loop`] at the end.
|
||
|
||
[`CR2`]: https://en.wikipedia.org/wiki/Control_register#CR2
|
||
[`Cr2::read`]: https://docs.rs/x86_64/0.3.5/x86_64/registers/control/struct.Cr2.html#method.read
|
||
[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.3.4/x86_64/structures/idt/struct.PageFaultErrorCode.html
|
||
[LLVM bug]: https://github.com/rust-lang/rust/issues/57270
|
||
[`hlt_loop`]: ./second-edition/posts/08-hardware-interrupts/index.md#the
|
||
|
||
Now we can try to access some memory outside our kernel:
|
||
|
||
```rust
|
||
// in src/main.rs
|
||
|
||
#[cfg(not(test))]
|
||
#[no_mangle]
|
||
pub extern "C" fn _start() -> ! {
|
||
use blog_os::interrupts::PICS;
|
||
|
||
println!("Hello World{}", "!");
|
||
|
||
// set up the IDT first, otherwise we would enter a boot loop instead of
|
||
// invoking our page fault handler
|
||
blog_os::gdt::init();
|
||
blog_os::interrupts::init_idt();
|
||
unsafe { PICS.lock().initialize() };
|
||
x86_64::instructions::interrupts::enable();
|
||
|
||
// new
|
||
let ptr = 0xdeadbeaf as *mut u32;
|
||
unsafe { *ptr = 42; }
|
||
|
||
println!("It did not crash!");
|
||
blog_os::hlt_loop();
|
||
}
|
||
```
|
||
|
||
When we run it, we see that our page fault handler is called:
|
||
|
||
|
||
|
||
The `CR2` register indeed contains `0xdeadbeaf`, the address that we tried to access. This virtual address has no mapping in the page tables, so a page fault occured.
|
||
|
||
We see that the current instruction pointer is `0x20430a`, so we know that this address points to a code page. Code pages are mapped read-only by the bootloader, so reading from this address works but writing causes a page fault. You can try this by changing the `0xdeadbeaf` pointer:
|
||
|
||
```rust
|
||
// Note: The actual address might be different for you. Use the address that
|
||
// your page fault handler reports.
|
||
let ptr = 0x20430a as *mut u32;
|
||
// read from a code page -> works
|
||
unsafe { let x = *ptr; }
|
||
// write to a code page -> page fault
|
||
unsafe { *ptr = 42; }
|
||
```
|
||
|
||
### The Problem
|
||
|
||
We just saw that it is a good thing that our kernel already runs on virtual addresses, as it improves safety. However, it also leads to a problem when we try to access the page tables from our kernel: Page tables use physical addresses internally that we can't access directly.
|
||
|
||
We experienced that problem already [at the end of the previous post] when we tried to inspect the page tables from our kernel. The next section discusses the problem in detail and provides different approaches to a solution.
|
||
|
||
[at the end of the previous post]: ./second-edition/posts/09-paging-introduction/index.md#try-it-out
|
||
|
||
## Accessing Page Tables
|
||
|
||
Accessing the page tables from our kernel is not as easy as it may seem. To understand the problem let's take a look at the example 4-level page table hierarchy of the previous post again:
|
||
|
||
|
||
|
||
The important thing here is that each page entry stores the _physical_ address of the next table. This avoids the need to run a translation for these addresses too, which would be bad for performance and could easily cause endless translation loops.
|
||
|
||
The problem for us is that we can't directly access physical addresses from our kernel, since our kernel also runs on top of virtual addresses. For example when we access address `4 KiB`, we access the _virtual_ address `4 KiB`, not the _physical_ address `4 KiB` where the level 4 page table lives. When we want to acccess the physical address `4 KiB`, we can only do so through some virtual address that maps to it.
|
||
|
||
So in order access page table frames, we need to map some virtual pages to them. There are different ways to create these mappings that all allow us to access arbitrary page table frames:
|
||
|
||
|
||
- A simple solution is to **identity map all page tables**:
|
||
|
||
|
||
|
||
In this example we see various identity-mapped page table frames. This way the physical addresses in the page tables are also valid virtual addresses, so that we can easily access the page tables of all levels starting from the CR3 register.
|
||
|
||
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000 KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `26 KiB` because it would collide with the already mapped page at `1004 MiB`. So we have to look further until we find a large enough unmapped area, for example at `1008 KiB`. This is a similar fragmentation problem as with [segmentation].
|
||
|
||
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
|
||
[segmentation]: ./second-edition/posts/09-paging-introduction/index.md#fragmentation
|
||
|
||
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000 KiB memory region starting at `1008 KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000 KiB` and `2008 KiB` anymore, because we can't identity map it.
|
||
|
||
- Alternatively, we could **map the page tables frames only temporarily** when we need to access them. To be able to create the temporary mappings we only need a single identity-mapped level 1 table:
|
||
|
||
|
||
|
||
The level 1 table in this graphic controls the first 2 MiB of the virtual address space. This is because it is reachable by starting at the CR3 register and following the 0th entry in the level 4, level 3, and level 2 page tables. The entry with index `8` maps the virtual page at address `32 KiB` to the physical frame at address `32 KiB`, thereby identity mapping the level 1 table itself. The graphic shows this identity-mapping by the horizontal arrow at `32 KiB`.
|
||
|
||
By writing to the identity-mapped level 1 table, our kernel can create up to 511 temporary mappings (512 minus the entry required for the identity mapping). In the above example, the kernel mapped the 0th entry of the level 1 table to the frame with address `24 KiB`. This created a temporary mapping of the virtual page at `0 KiB` to the physical frame of the level 2 page table, indicated by the dashed arrow. Now the kernel can access the level 2 page table by writing to the page starting at `0 KiB`.
|
||
|
||
The process for accessing an arbitrary page table frame with temporary mappings would be:
|
||
|
||
- Search for a free entry in the identity mapped level 1 table.
|
||
- Map that entry to the physical frame of the page table that we want to access.
|
||
- Access the target frame through the virtual page that maps to the entry.
|
||
- Set the entry back to unused thereby removing the temporary mapping again.
|
||
|
||
This approach keeps the virtual address space clean, since it reuses the same 512 virtual pages for creating the mappings. The drawback is that it is a bit cumbersome, especially since a new mapping might require modifications of multiple table levels, which means that we would need to repeat the above process multiple times.
|
||
|
||
- While both of the above approaches work, there is a third technique called **recursive page tables** that combines their advantages: It keeps all page table frames mapped at all times so that no temporary mappings are needed, and also keeps the mapped pages together to avoid fragmentation of the virtual address space. This is the technique that we will use for our implementation, therefore it is described in detail in the following section.
|
||
|
||
## Recursive Page Tables
|
||
|
||
The idea behind this approach sounds simple: _Map some entry of the level 4 page table to the frame of level 4 table itself_. By doing this, we effectively reserve a part of the virtual address space and map all current and future page table frames to that space. Thus, the single entry makes every table of every level accessible through a calculatable address.
|
||
|
||
Let's go through an example to understand how this all works:
|
||
|
||
|
||
|
||
The only difference to the [example at the beginning of this post] is the additional entry at index `511` in the level 4 table, which is mapped to physical frame `4 KiB`, the frame of the level 4 table itself.
|
||
|
||
[example at the beginning of this post]: #accessing-page-tables
|
||
|
||
By letting the CPU follow this entry on a translation, it doesn't reach a level 3 table, but the same level 4 table again. This is similar to a recursive function that calls itself, therefore this table is called a _recursive page table_. The important thing is that the CPU assumes that every entry in the level 4 table points to a level 3 table, so it now treats the level 4 table as a level 3 table. This works because tables of all levels have the exact same layout on x86_64.
|
||
|
||
By following the recursive entry one or multiple times before we start the actual translation, we can effectively shorten the number of levels that the CPU traverses. For example, if we follow the recursive entry once and then proceed to the level 3 table, the CPU thinks that the level 3 table is a level 2 table. Going further, it treats the level 2 table as a level 1 table, and the level 1 table as the mapped frame. This means that we can now read and write the level 1 page table because the CPU thinks that it is the mapped frame. The graphic below illustrates the 5 translation steps:
|
||
|
||
|
||
|
||
Similarly, we can follow the recursive entry twice before starting the translation to reduce the number of traversed levels to two:
|
||
|
||
|
||
|
||
Let's go through it step by step: First the CPU follows the recursive entry on the level 4 table and thinks that it reaches a level 3 table. Then it follows the recursive entry again and thinks that it reaches a level 2 table. But in reality, it is still on the level 4 table. When the CPU now follows a different entry, it lands on a level 3 table, but thinks it is already on a level 1 table. So while the next entry points at a level 2 table, the CPU thinks that it points to the mapped frame, which allows us to read and write the level 2 table.
|
||
|
||
Accessing the tables of levels 3 and 4 works in the same way. For accessing the level 3 table, we follow the recursive entry entry three times, tricking the CPU into thinking it is already on a level 1 table. Then we follow another entry and reach a level 3 table, which the CPU treats as a mapped frame. For accessing the level 4 table itself, we just follow the recursive entry four times until the CPU treats the level 4 table itself as mapped frame (in blue in the graphic below).
|
||
|
||
|
||
|
||
It might take some time to wrap your head around the concept, but it works quite well in practice.
|
||
|
||
### Address Calculation
|
||
|
||
We saw that we can access tables of all levels by following the recursive entry once or multiple times before the actual translation. Since the indexes into the tables of the four levels are derived directly from the virtual address, we need to construct special virtual addresses for this technique. Remember, the page table indexes are derived from the address in the following way:
|
||
|
||
|
||
|
||
Let's assume that we want to access the level 1 page table that maps a specific page. As we learned above, this means that we have to follow the recursive entry one time before continuing with the level 4, level 3, and level 2 indexes. To do that we move each block of the address one block to the right and set the original level 4 index to the index of the recursive entry:
|
||
|
||
|
||
|
||
For accessing the level 2 table of that page, we move each index block two blocks to the right and set both the blocks of the original level 4 index and the original level 3 index to the index of the recursive entry:
|
||
|
||
|
||
|
||
Accessing the level 3 table works by moving each block three blocks to the right and using the recursive index for the original level 4, level 3, and level 2 address blocks:
|
||
|
||
|
||
|
||
Finally, we can access the level 4 table by moving each block four blocks to the right and using the recursive index for all address blocks except for the offset:
|
||
|
||
|
||
|
||
The page table index blocks are 9 bits, so moving each block one block to the right means a bitshift by 9 bits: `address >> 9`. To derive the 12-bit offset field from the shifted index, we need to multiply it by 8, the size of a page table entry. Through this operation, we can calculate virtual addresses for the page tables of all four levels.
|
||
|
||
The table below summarizes the address structure for accessing the different kinds of frames:
|
||
|
||
Mapped Frame for | Address Structure ([octal])
|
||
---------------- | -------------------------------
|
||
Page | `0o_SSSSSS_AAA_BBB_CCC_DDD_EEEE`
|
||
Level 1 Table | `0o_SSSSSS_RRR_AAA_BBB_CCC_DDDD`
|
||
Level 2 Table | `0o_SSSSSS_RRR_RRR_AAA_BBB_CCCC`
|
||
Level 3 Table | `0o_SSSSSS_RRR_RRR_RRR_AAA_BBBB`
|
||
Level 4 Table | `0o_SSSSSS_RRR_RRR_RRR_RRR_AAAA`
|
||
|
||
[octal]: https://en.wikipedia.org/wiki/Octal
|
||
|
||
Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 index, and `DDD` the level 1 index of the mapped frame, and `EEEE` the offset into it. `RRR` is the index of the recursive entry. When an index (three digits) is transformed to an offset (four digits), it is done by multiplying it by 8 (the size of a page table entry). With this offset, the resulting address directly points to the respective page table entry.
|
||
|
||
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
|
||
|
||
[sign extension]: ./second-edition/posts/09-paging-introduction/index.md#paging-on-x86
|
||
|
||
## Implementation
|
||
|
||
After all this theory we can finally start our implementation. We already mentioned that the bootloader created page tables for our kernel, but it also created a recursive mapping in the last entry of the level 4 table for us. The bootloader did this, because otherwise there would be a [chicken or egg problem]: We need to access the level 4 table to create a recursive mapping, but we can't access it without some kind of mapping.
|
||
|
||
[chicken or egg problem]: https://en.wikipedia.org/wiki/Chicken_or_the_egg
|
||
|
||
We already used this recursive mapping [at the end of the previous post] to access the level 4 table. We did this through the hardcoded address `0xffff_ffff_ffff_f000`. When we convert this address to [octal] and compare it with the above table, we can see that it exactly follows the structure: with `RRR` = `0o777` = 511, `AAAA` = 0, and the sign extension bits set to `1` each:
|
||
|
||
```
|
||
structure: 0o_SSSSSS_RRR_RRR_RRR_RRR_AAAA
|
||
address: 0o_177777_777_777_777_777_0000
|
||
```
|
||
|
||
Using hardcoded addresses is rarely a good idea, since they might become outdated. For example, our code would break if a future bootloader version uses a different entry for the recursive mapping. Fortunately the bootloader tells us the recursive entry by passing a _boot information structure_ to our kernel.
|
||
|
||
### Boot Information
|
||
|
||
To communicate the index of the recursive entry and other information to our kernel, the bootloader passes a reference to a boot information structure as an argument when calling our `_start` function. Right now we don't have this argument declared in our function, so we just ignore it. Let's add the argument to the signature of our `_start` function:
|
||
|
||
```rust
|
||
// in src/main.rs
|
||
|
||
use bootloader::bootinfo::BootInfo;
|
||
|
||
#[cfg(not(test))]
|
||
#[no_mangle]
|
||
pub extern "C" fn _start(boot_info: &'static BootInfo) -> ! {
|
||
println!("boot_info: {:x?}", boot_info);
|
||
|
||
[…]
|
||
}
|
||
```
|
||
|
||
The [`BootInfo`] struct is still in an early stage, so expect some breakage when updating to future [semver-incompatible] bootloader versions. When we print it, we see that it currently has the three fields `p4_table_addr`, `memory_map`, and `package`:
|
||
|
||
[`BootInfo`]: https://docs.rs/bootloader/0.3.11/bootloader/bootinfo/struct.BootInfo.html
|
||
[semver-incompatible]: https://doc.rust-lang.org/stable/cargo/reference/specifying-dependencies.html#caret-requirements
|
||
|
||
![QEMU printing a `BootInfo` struct: "boot_info: Bootlnfo { p4_table_addr: fffffffffffff000. memory_map: […]. package: […]"](qemu-bootinfo-print.png)
|
||
|
||
The most interesting field for us right now is `p4_table_addr`, as it contains a virtual address that is mapped to the physical frame of the level 4 page table. As we see this address is `0xfffffffffffff000`, which is the same as the hardcoded address we used before.
|
||
|
||
The `memory_map` field will become relevant later in this post. The `package` field is an in-progress feature to bundle additional data with the bootloader. The implementation is not finished, so we can ignore this field for now.
|
||
|
||
### The `RecursivePageTable` Type
|
||
|
||
TODO:
|
||
|
||
- Map adddress 0 to the vga buffer
|
||
- We need free physical frames for creating new page tables -> memory map
|
||
|
||
## A Physical Memory Map
|
||
|
||
## Allocating Stacks
|
||
|
||
## Summary
|
||
|
||
## What's next?
|
||
|
||
---
|
||
TODO update post date |