diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 6e66feab..28c855dc 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -166,18 +166,18 @@ The functionality of the `isa-debug-exit` device is very simple. When a `value` Instead of manually invoking the `in` and `out` assembly instructions, we use the abstractions provided by the [`x86_64`] crate. To add a dependency on that crate, we add it to the `dependencies` section in our `Cargo.toml`: -[`x86_64`]: https://docs.rs/x86_64/0.7.5/x86_64/ +[`x86_64`]: https://docs.rs/x86_64/0.8.1/x86_64/ ```toml # in Cargo.toml [dependencies] -x86_64 = "0.7.5" +x86_64 = "0.8.1" ``` Now we can use the [`Port`] type provided by the crate to create an `exit_qemu` function: -[`Port`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/port/struct.Port.html +[`Port`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/port/struct.Port.html ```rust // in src/main.rs diff --git a/blog/content/second-edition/posts/05-cpu-exceptions/index.md b/blog/content/second-edition/posts/05-cpu-exceptions/index.md index 0fa6862b..840117a6 100644 --- a/blog/content/second-edition/posts/05-cpu-exceptions/index.md +++ b/blog/content/second-edition/posts/05-cpu-exceptions/index.md @@ -82,7 +82,7 @@ Don't worry about steps 4 and 5 for now, we will learn about the global descript ## An IDT Type Instead of creating our own IDT type, we will use the [`InterruptDescriptorTable` struct] of the `x86_64` crate, which looks like this: -[`InterruptDescriptorTable` struct]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html +[`InterruptDescriptorTable` struct]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html ``` rust #[repr(C)] @@ -113,10 +113,10 @@ pub struct InterruptDescriptorTable { The fields have the type [`idt::Entry`], which is a struct that represents the fields of an IDT entry (see the table above). The type parameter `F` defines the expected handler function type. We see that some entries require a [`HandlerFunc`] and some entries require a [`HandlerFuncWithErrCode`]. The page fault even has its own special type: [`PageFaultHandlerFunc`]. -[`idt::Entry`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.Entry.html -[`HandlerFunc`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/type.HandlerFunc.html -[`HandlerFuncWithErrCode`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/type.HandlerFuncWithErrCode.html -[`PageFaultHandlerFunc`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/type.PageFaultHandlerFunc.html +[`idt::Entry`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.Entry.html +[`HandlerFunc`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/type.HandlerFunc.html +[`HandlerFuncWithErrCode`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/type.HandlerFuncWithErrCode.html +[`PageFaultHandlerFunc`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/type.PageFaultHandlerFunc.html Let's look at the `HandlerFunc` type first: @@ -191,7 +191,7 @@ So the _interrupt stack frame_ looks like this: In the `x86_64` crate, the interrupt stack frame is represented by the [`InterruptStackFrame`] struct. It is passed to interrupt handlers as `&mut` and can be used to retrieve additional information about the exception's cause. The struct contains no error code field, since only some few exceptions push an error code. These exceptions use the separate [`HandlerFuncWithErrCode`] function type, which has an additional `error_code` argument. -[`InterruptStackFrame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptStackFrame.html +[`InterruptStackFrame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptStackFrame.html Note that there is currently [a bug in LLVM] that leads to wrong error code arguments. The cause of the issue is already known and a solution is [being worked on]. @@ -279,7 +279,7 @@ This error occurs because the `x86-interrupt` calling convention is still unstab In order that the CPU uses our new interrupt descriptor table, we need to load it using the [`lidt`] instruction. The `InterruptDescriptorTable` struct of the `x86_64` provides a [`load`][InterruptDescriptorTable::load] method function for that. Let's try to use it: [`lidt`]: https://www.felixcloutier.com/x86/lgdt:lidt -[InterruptDescriptorTable::load]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html#method.load +[InterruptDescriptorTable::load]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html#method.load ```rust // in src/interrupts.rs @@ -462,7 +462,7 @@ You can try this new test by running `cargo xtest` (all tests) or `cargo xtest - The `x86-interrupt` calling convention and the [`InterruptDescriptorTable`] type made the exception handling process relatively straightforward and painless. If this was too much magic for you and you like to learn all the gory details of exception handling, we got you covered: Our [“Handling Exceptions with Naked Functions”] series shows how to handle exceptions without the `x86-interrupt` calling convention and also creates its own IDT type. Historically, these posts were the main exception handling posts before the `x86-interrupt` calling convention and the `x86_64` crate existed. Note that these posts are based on the [first edition] of this blog and might be out of date. [“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md -[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html +[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html [first edition]: @/first-edition/_index.md ## What's next? diff --git a/blog/content/second-edition/posts/06-double-faults/index.md b/blog/content/second-edition/posts/06-double-faults/index.md index a0f587d9..42c0acf4 100644 --- a/blog/content/second-edition/posts/06-double-faults/index.md +++ b/blog/content/second-edition/posts/06-double-faults/index.md @@ -79,13 +79,15 @@ lazy_static! { // new extern "x86-interrupt" fn double_fault_handler( - stack_frame: &mut InterruptStackFrame, _error_code: u64) + stack_frame: &mut InterruptStackFrame, _error_code: u64) -> ! { panic!("EXCEPTION: DOUBLE FAULT\n{:#?}", stack_frame); } ``` -Our handler prints a short error message and dumps the exception stack frame. The error code of the double fault handler is always zero, so there's no reason to print it. +Our handler prints a short error message and dumps the exception stack frame. The error code of the double fault handler is always zero, so there's no reason to print it. One difference to the breakpoint handler is that the double fault handler is [_diverging_]. The reason is that the `x86_64` architecture does not permit returning from a double fault exception. + +[_diverging_]: https://doc.rust-lang.org/stable/rust-by-example/fn/diverging.html When we start our kernel now, we should see that the double fault handler is invoked: @@ -225,7 +227,7 @@ The _Privilege Stack Table_ is used by the CPU when the privilege level changes. ### Creating a TSS Let's create a new TSS that contains a separate double fault stack in its interrupt stack table. For that we need a TSS struct. Fortunately, the `x86_64` crate already contains a [`TaskStateSegment` struct] that we can use. -[`TaskStateSegment` struct]: https://docs.rs/x86_64/0.7.5/x86_64/structures/tss/struct.TaskStateSegment.html +[`TaskStateSegment` struct]: https://docs.rs/x86_64/0.8.1/x86_64/structures/tss/struct.TaskStateSegment.html We create the TSS in a new `gdt` module (the name will make sense later): @@ -371,8 +373,8 @@ pub fn init() { We reload the code segment register using [`set_cs`] and to load the TSS using [`load_tss`]. The functions are marked as `unsafe`, so we need an `unsafe` block to invoke them. The reason is that it might be possible to break memory safety by loading invalid selectors. -[`set_cs`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/segmentation/fn.set_cs.html -[`load_tss`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/tables/fn.load_tss.html +[`set_cs`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/segmentation/fn.set_cs.html +[`load_tss`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/tables/fn.load_tss.html Now that we loaded a valid TSS and interrupt stack table, we can set the stack index for our double fault handler in the IDT: @@ -517,7 +519,7 @@ use x86_64::structures::idt::InterruptStackFrame; extern "x86-interrupt" fn test_double_fault_handler( _stack_frame: &mut InterruptStackFrame, _error_code: u64, -) { +) -> ! { serial_println!("[ok]"); exit_qemu(QemuExitCode::Success); loop {} diff --git a/blog/content/second-edition/posts/07-hardware-interrupts/index.md b/blog/content/second-edition/posts/07-hardware-interrupts/index.md index 886c38b8..9f1cd8f1 100644 --- a/blog/content/second-edition/posts/07-hardware-interrupts/index.md +++ b/blog/content/second-edition/posts/07-hardware-interrupts/index.md @@ -206,7 +206,7 @@ extern "x86-interrupt" fn timer_interrupt_handler( Our `timer_interrupt_handler` has the same signature as our exception handlers, because the CPU reacts identically to exceptions and external interrupts (the only difference is that some exceptions push an error code). The [`InterruptDescriptorTable`] struct implements the [`IndexMut`] trait, so we can access individual entries through array indexing syntax. -[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html +[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html [`IndexMut`]: https://doc.rust-lang.org/core/ops/trait.IndexMut.html In our timer interrupt handler, we print a dot to the screen. As the timer interrupt happens periodically, we would expect to see a dot appearing on each timer tick. However, when we run it we see that only a single dot is printed: @@ -331,7 +331,7 @@ pub fn _print(args: fmt::Arguments) { The [`without_interrupts`] function takes a [closure] and executes it in an interrupt-free environment. We use it to ensure that no interrupt can occur as long as the `Mutex` is locked. When we run our kernel now we see that it keeps running without hanging. (We still don't notice any dots, but this is because they're scrolling by too fast. Try to slow down the printing, e.g. by putting a `for _ in 0..10000 {}` inside the loop.) -[`without_interrupts`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/interrupts/fn.without_interrupts.html +[`without_interrupts`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/interrupts/fn.without_interrupts.html [closure]: https://doc.rust-lang.org/book/second-edition/ch13-01-closures.html We can apply the same change to our serial printing function to ensure that no deadlocks occur with it either: @@ -586,7 +586,7 @@ extern "x86-interrupt" fn keyboard_interrupt_handler( We use the [`Port`] type of the `x86_64` crate to read a byte from the keyboard's data port. This byte is called the [_scancode_] and is a number that represents the key press/release. We don't do anything with the scancode yet, we just print it to the screen: -[`Port`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/port/struct.Port.html +[`Port`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/port/struct.Port.html [_scancode_]: https://en.wikipedia.org/wiki/Scancode ![QEMU printing scancodes to the screen when keys are pressed](qemu-printing-scancodes.gif) diff --git a/blog/content/second-edition/posts/08-paging-introduction/index.md b/blog/content/second-edition/posts/08-paging-introduction/index.md index 4ed55aff..29f8784c 100644 --- a/blog/content/second-edition/posts/08-paging-introduction/index.md +++ b/blog/content/second-edition/posts/08-paging-introduction/index.md @@ -229,8 +229,8 @@ Let's take a closer look at the available flags: The `x86_64` crate provides types for [page tables] and their [entries], so we don't need to create these structures ourselves. -[page tables]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page_table/struct.PageTable.html -[entries]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page_table/struct.PageTableEntry.html +[page tables]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page_table/struct.PageTable.html +[entries]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page_table/struct.PageTableEntry.html ### The Translation Lookaside Buffer @@ -239,7 +239,7 @@ A 4-level page table makes the translation of virtual addresses expensive, becau Unlike the other CPU caches, the TLB is not fully transparent and does not update or remove translations when the contents of page tables change. This means that the kernel must manually update the TLB whenever it modifies a page table. To do this, there is a special CPU instruction called [`invlpg`] (“invalidate page”) that removes the translation for the specified page from the TLB, so that it is loaded again from the page table on the next access. The TLB can also be flushed completely by reloading the `CR3` register, which simulates an address space switch. The `x86_64` crate provides Rust functions for both variants in the [`tlb` module]. [`invlpg`]: https://www.felixcloutier.com/x86/INVLPG.html -[`tlb` module]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/tlb/index.html +[`tlb` module]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/tlb/index.html It is important to remember flushing the TLB on each page table modification because otherwise the CPU might keep using the old translation, which can lead to non-deterministic bugs that are very hard to debug. @@ -294,8 +294,8 @@ extern "x86-interrupt" fn page_fault_handler( The [`CR2`] register is automatically set by the CPU on a page fault and contains the accessed virtual address that caused the page fault. We use the [`Cr2::read`] function of the `x86_64` crate to read and print it. The [`PageFaultErrorCode`] type provides more information about the type of memory access that caused the page fault, for example whether it was caused by a read or write operation. For this reason we print it too. We can't continue execution without resolving the page fault, so we enter a [`hlt_loop`] at the end. [`CR2`]: https://en.wikipedia.org/wiki/Control_register#CR2 -[`Cr2::read`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr2.html#method.read -[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html +[`Cr2::read`]: https://docs.rs/x86_64/0.8.1/x86_64/registers/control/struct.Cr2.html#method.read +[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.PageFaultErrorCode.html [LLVM bug]: https://github.com/rust-lang/rust/issues/57270 [`hlt_loop`]: @/second-edition/posts/07-hardware-interrupts/index.md#the-hlt-instruction @@ -329,7 +329,7 @@ When we run it, we see that our page fault handler is called: The `CR2` register indeed contains `0xdeadbeaf`, the address that we tried to access. The error code tells us through the [`CAUSED_BY_WRITE`] that the fault occurred while trying to perform a write operation. It tells us even more through the [bits that are _not_ set][`PageFaultErrorCode`]. For example, the fact that the `PROTECTION_VIOLATION` flag is not set means that the page fault occurred because the target page wasn't present. -[`CAUSED_BY_WRITE`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.CAUSED_BY_WRITE +[`CAUSED_BY_WRITE`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.CAUSED_BY_WRITE We see that the current instruction pointer is `0x2031b2`, so we know that this address points to a code page. Code pages are mapped read-only by the bootloader, so reading from this address works but writing causes a page fault. You can try this by changing the `0xdeadbeaf` pointer to `0x2031b2`: @@ -353,7 +353,7 @@ By commenting out the last line, we see that the read access works, but the writ We see that the _"read worked"_ message is printed, which indicates that the read operation did not cause any errors. However, instead of the _"write worked"_ message a page fault occurs. This time the [`PROTECTION_VIOLATION`] flag is set in addition to the [`CAUSED_BY_WRITE`] flag, which indicates that the page was present, but the operation was not allowed on it. In this case, writes to the page are not allowed since code pages are mapped as read-only. -[`PROTECTION_VIOLATION`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.PROTECTION_VIOLATION +[`PROTECTION_VIOLATION`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.PROTECTION_VIOLATION ### Accessing the Page Tables @@ -379,9 +379,9 @@ pub extern "C" fn _start() -> ! { The [`Cr3::read`] function of the `x86_64` returns the currently active level 4 page table from the `CR3` register. It returns a tuple of a [`PhysFrame`] and a [`Cr3Flags`] type. We are only interested in the frame, so we ignore the second element of the tuple. -[`Cr3::read`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr3.html#method.read -[`PhysFrame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/frame/struct.PhysFrame.html -[`Cr3Flags`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr3Flags.html +[`Cr3::read`]: https://docs.rs/x86_64/0.8.1/x86_64/registers/control/struct.Cr3.html#method.read +[`PhysFrame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/frame/struct.PhysFrame.html +[`Cr3Flags`]: https://docs.rs/x86_64/0.8.1/x86_64/registers/control/struct.Cr3Flags.html When we run it, we see the following output: @@ -391,7 +391,7 @@ Level 4 page table at: PhysAddr(0x1000) So the currently active level 4 page table is stored at address `0x1000` in _physical_ memory, as indicated by the [`PhysAddr`] wrapper type. The question now is: how can we access this table from our kernel? -[`PhysAddr`]: https://docs.rs/x86_64/0.7.5/x86_64/struct.PhysAddr.html +[`PhysAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.PhysAddr.html Accessing physical memory directly is not possible when paging is active, since programs could easily circumvent memory protection and access memory of other programs otherwise. So the only way to access the table is through some virtual page that is mapped to the physical frame at address `0x1000`. This problem of creating mappings for page table frames is a general problem, since the kernel needs to access the page tables regularly, for example when allocating a stack for a new thread. diff --git a/blog/content/second-edition/posts/09-paging-implementation/index.md b/blog/content/second-edition/posts/09-paging-implementation/index.md index 97bef938..ce74724f 100644 --- a/blog/content/second-edition/posts/09-paging-implementation/index.md +++ b/blog/content/second-edition/posts/09-paging-implementation/index.md @@ -31,11 +31,11 @@ To implement the approach, we will need support from the bootloader, so we'll co ### Dependency Updates -This post requires version 0.7.5 or later of the `x86_64` dependency. You can update the dependency in your `Cargo.toml`: +This post requires version 0.8.1 or later of the `x86_64` dependency. You can update the dependency in your `Cargo.toml`: ```toml [dependencies] -x86_64 = "0.7.5" +x86_64 = "0.8.1" ``` For an overview of the changes in recent versions, check out the [`x86_64` changelog]. @@ -229,7 +229,7 @@ The above code assumes that the last level 4 entry with index `0o777` (511) is r Alternatively to performing the bitwise operations by hand, you can use the [`RecursivePageTable`] type of the `x86_64` crate, which provides safe abstractions for various page table operations. For example, the code below shows how to translate a virtual address to its mapped physical address: -[`RecursivePageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.RecursivePageTable.html +[`RecursivePageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.RecursivePageTable.html ```rust // in src/memory.rs @@ -447,7 +447,7 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! { First, we convert the `physical_memory_offset` of the `BootInfo` struct to a [`VirtAddr`] and pass it to the `active_level_4_table` function. We then use the `iter` function to iterate over the page table entries and the [`enumerate`] combinator to additionally add an index `i` to each element. We only print non-empty entries because all 512 entries wouldn't fit on the screen. -[`VirtAddr`]: https://docs.rs/x86_64/0.7.5/x86_64/struct.VirtAddr.html +[`VirtAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.VirtAddr.html [`enumerate`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.enumerate When we run it, we see the following output: @@ -560,7 +560,7 @@ The `VirtAddr` struct already provides methods to compute the indexes into the p Inside the loop, we again use the `physical_memory_offset` to convert the frame into a page table reference. We then read the entry of the current page table and use the [`PageTableEntry::frame`] function to retrieve the mapped frame. If the entry is not mapped to a frame we return `None`. If the entry maps a huge 2MiB or 1GiB page we panic for now. -[`PageTableEntry::frame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page_table/struct.PageTableEntry.html#method.frame +[`PageTableEntry::frame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page_table/struct.PageTableEntry.html#method.frame Let's test our translation function by translating some addresses: @@ -616,18 +616,18 @@ The base of the abstraction are two traits that define various page table mappin - The [`Mapper`] trait is generic over the page size and provides functions that operate on pages. Examples are [`translate_page`], which translates a given page to a frame of the same size, and [`map_to`], which creates a new mapping in the page table. - The [`MapperAllSizes`] trait implies that the implementor implements `Mapper` for all pages sizes. In addition, it provides functions that work with multiple page sizes such as [`translate_addr`] or the general [`translate`]. -[`Mapper`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html -[`translate_page`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.translate_page -[`map_to`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to -[`MapperAllSizes`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.MapperAllSizes.html -[`translate_addr`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#method.translate_addr -[`translate`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#tymethod.translate +[`Mapper`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html +[`translate_page`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.translate_page +[`map_to`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to +[`MapperAllSizes`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html +[`translate_addr`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#method.translate_addr +[`translate`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#tymethod.translate The traits only define the interface, they don't provide any implementation. The `x86_64` crate currently provides three types that implement the traits with different requirements. The [`OffsetPageTable`] type assumes that the complete physical memory is mapped to the virtual address space at some offset. The [`MappedPageTable`] is a bit more flexible: It only requires that each page table frame is mapped to the virtual address space at a calculable address. Finally, the [`RecursivePageTable`] type can be used to access page table frames through [recursive page tables](#recursive-page-tables). -[`OffsetPageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.OffsetPageTable.html -[`MappedPageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MappedPageTable.html -[`RecursivePageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.RecursivePageTable.html +[`OffsetPageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.OffsetPageTable.html +[`MappedPageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MappedPageTable.html +[`RecursivePageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.RecursivePageTable.html In our case, the bootloader maps the complete physical memory at an virtual address specfied by the `physical_memory_offset` variable, so we can use the `OffsetPageTable` type. To initialize it, we create a new `init` function in our `memory` module: @@ -653,7 +653,7 @@ unsafe fn active_level_4_table(physical_memory_offset: VirtAddr) The function takes the `physical_memory_offset` as an argument and returns a new `OffsetPageTable` instance with a `'static` lifetime. This means that the instance stays valid for the complete runtime of our kernel. In the function body, we first call the `active_level_4_table` function to retrieve a mutable reference to the level 4 page table. We then invoke the [`OffsetPageTable::new`] function with this reference. As the second parameter, the `new` function expects the virtual address at which the mapping of the physical memory starts, which is given in the `physical_memory_offset` variable. -[`OffsetPageTable::new`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.OffsetPageTable.html#method.new +[`OffsetPageTable::new`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.OffsetPageTable.html#method.new The `active_level_4_table` function should be only called from the `init` function from now on because it can easily lead to aliased mutable references when called multiple times, which can cause undefined behavior. For this reason, we make the function private by removing the `pub` specifier. @@ -704,8 +704,8 @@ Until now we only looked at the page tables without modifying anything. Let's ch We will use the [`map_to`] function of the [`Mapper`] trait for our implementation, so let's take a look at that function first. The documentation tells us that it takes four arguments: the page that we want to map, the frame that the page should be mapped to, a set of flags for the page table entry, and a `frame_allocator`. The frame allocator is needed because mapping the given page might require creating additional page tables, which need unused frames as backing storage. -[`map_to`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.Mapper.html#tymethod.map_to -[`Mapper`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.Mapper.html +[`map_to`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.Mapper.html#tymethod.map_to +[`Mapper`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.Mapper.html #### A `create_example_mapping` Function @@ -718,7 +718,7 @@ The `create_example_mapping` function looks like this: use x86_64::{ PhysAddr, - structures::paging::{Page, PhysFrame, Mapper, Size4KiB, FrameAllocator} + structures::paging::{Page, PhysFrame, Mapper, Size4KiB, FrameAllocator, UnusedPhysFrame} }; /// Creates an example mapping for the given page to frame `0xb8000`. @@ -730,11 +730,11 @@ pub fn create_example_mapping( use x86_64::structures::paging::PageTableFlags as Flags; let frame = PhysFrame::containing_address(PhysAddr::new(0xb8000)); + // FIXME: ONLY FOR TEMPORARY TESTING + let unused_frame = unsafe { UnusedPhysFrame::new(frame) }; let flags = Flags::PRESENT | Flags::WRITABLE; - let map_to_result = unsafe { - mapper.map_to(page, frame, flags, frame_allocator) - }; + let map_to_result = mapper.map_to(page, unused_frame, flags, frame_allocator); map_to_result.expect("map_to failed").flush(); } ``` @@ -743,10 +743,15 @@ In addition to the `page` that should be mapped, the function expects a mutable [impl-trait-arg]: https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters [generic]: https://doc.rust-lang.org/book/ch10-00-generics.html -[`FrameAllocator`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html -[`PageSize`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/trait.PageSize.html +[`FrameAllocator`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.FrameAllocator.html +[`PageSize`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/trait.PageSize.html -For the mapping, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. Calling [`map_to`] is unsafe because it's possible to break memory safety with invalid arguments, so we need to use an `unsafe` block. For a list of all possible flags, see the [_Page Table Format_] section of the previous post. +Instead of a normal `PhysFrame`, the [`map_to`] method requires an [`UnusedPhysFrame`] wrapper type to ensure that the frame is not already in use. The reason for this is that mapping the same frame twice could result in undefined behavior, for example when two different `&mut` references point to the same physical memory location. In our case, we reuse the VGA text buffer frame, which is already mapped, so we break the required condition when calling the unsafe [`UnusedPhysFrame::new`] function. However, the `create_example_mapping` function is only a temporary testing function and will be removed after this post, so it is ok. To remind us of the unsafety, we put a `FIXME` comment on the line. + +[`UnusedPhysFrame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/struct.UnusedPhysFrame.html +[`UnusedPhysFrame::new`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/struct.UnusedPhysFrame.html#method.new + +In addition to the `page` and the `unused_frame`, the `map_to` method takes a set of flags for the mapping and a reference to the `frame_allocator`, which will be explained in a moment. For the flags, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. For a list of all possible flags, see the [_Page Table Format_] section of the previous post. [_Page Table Format_]: @/second-edition/posts/08-paging-introduction/index.md#page-table-format @@ -754,8 +759,8 @@ The [`map_to`] function can fail, so it returns a [`Result`]. Since this is just [`Result`]: https://doc.rust-lang.org/core/result/enum.Result.html [`expect`]: https://doc.rust-lang.org/core/result/enum.Result.html#method.expect -[`MapperFlush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html -[`flush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush +[`MapperFlush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html +[`flush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush [must_use]: https://doc.rust-lang.org/std/result/#results-must-be-used #### A dummy `FrameAllocator` @@ -771,7 +776,7 @@ Let's start with the simple case and assume that we don't need to create new pag pub struct EmptyFrameAllocator; unsafe impl FrameAllocator for EmptyFrameAllocator { - fn allocate_frame(&mut self) -> Option { + fn allocate_frame(&mut self) -> Option { None } } @@ -906,7 +911,7 @@ use bootloader::bootinfo::MemoryRegionType; impl BootInfoFrameAllocator { /// Returns an iterator over the usable frames specified in the memory map. - fn usable_frames(&self) -> impl Iterator { + fn usable_frames(&self) -> impl Iterator { // get usable regions from memory map let regions = self.memory_map.iter(); let usable_regions = regions @@ -917,8 +922,9 @@ impl BootInfoFrameAllocator { // transform to an iterator of frame start addresses let frame_addresses = addr_ranges.flat_map(|r| r.step_by(4096)); // create `PhysFrame` types from the start addresses - frame_addresses - .map(|addr|PhysFrame::containing_address(PhysAddr::new(addr))) + let frames = frame_addresses.map(|addr| PhysFrame::containing_address(PhysAddr::new(addr))); + // we know that the frames are really unused + frames.map(|f| unsafe { UnusedPhysFrame::new(f) }) } } ``` @@ -929,7 +935,8 @@ This function uses iterator combinator methods to transform the initial `MemoryM - Then we use the [`filter`] method to skip any reserved or otherwise unavailable regions. The bootloader updates the memory map for all the mappings it creates, so frames that are used by our kernel (code, data or stack) or to store the boot information are already marked as `InUse` or similar. Thus we can be sure that `Usable` frames are not used somewhere else. - Afterwards, we use the [`map`] combinator and Rust's [range syntax] to transform our iterator of memory regions to an iterator of address ranges. - The next step is the most complicated: We convert each range to an iterator through the `into_iter` method and then choose every 4096th address using [`step_by`]. Since 4096 bytes (= 4 KiB) is the page size, we get the start address of each frame. The bootloader page aligns all usable memory areas so that we don't need any alignment or rounding code here. By using [`flat_map`] instead of `map`, we get an `Iterator` instead of an `Iterator>`. -- Finally, we convert the start addresses to `PhysFrame` types to construct the desired `Iterator`. We then use this iterator to create and return a new `BootInfoFrameAllocator`. +- Then we convert the start addresses to `PhysFrame` types to construct the an `Iterator`. +- In the last step, we use the [`map`] combinator again to wrap each frame into the [`UnusedPhysFrame`] wrapper type. This is safe because we trust the boot information. [`MemoryRegion`]: https://docs.rs/bootloader/0.6.4/bootloader/bootinfo/struct.MemoryRegion.html [`filter`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.filter @@ -938,7 +945,7 @@ This function uses iterator combinator methods to transform the initial `MemoryM [`step_by`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.step_by [`flat_map`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.flat_map -The return type of the function uses the [`impl Trait`] feature. This way, we can specify that we return some type that implements the [`Iterator`] trait with item type `PhysFrame`, but don't need to name the concrete return type. This is important here because we _can't_ name the conrete type since it depends on unnamable closure types. +The return type of the function uses the [`impl Trait`] feature. This way, we can specify that we return some type that implements the [`Iterator`] trait with item type `UnusedPhysFrame`, but don't need to name the concrete return type. This is important here because we _can't_ name the concrete type since it depends on unnamable closure types. [`impl Trait`]: https://doc.rust-lang.org/book/ch10-02-traits.html#returning-types-that-implement-traits [`Iterator`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html @@ -951,7 +958,7 @@ Now we can implement the `FrameAllocator` trait: // in src/memory.rs unsafe impl FrameAllocator for BootInfoFrameAllocator { - fn allocate_frame(&mut self) -> Option { + fn allocate_frame(&mut self) -> Option { let frame = self.usable_frames().nth(self.next); self.next += 1; frame @@ -994,6 +1001,8 @@ With the boot info frame allocator, the mapping succeeds and we see the black-on While our `create_example_mapping` function is just some example code, we are now able to create new mappings for arbitrary pages. This will be essential for allocating memory or implementing multithreading in future posts. +At this point, we should delete the `create_example_mapping` function again to avoid accidentally invoking undefined behavior, as explained [above](#a-create-example-mapping-function). + ## Summary In this post we learned about different techniques to access the physical frames of page tables, including identity mapping, mapping of the complete physical memory, temporary mapping, and recursive page tables. We chose to map the complete physical memory since it's simple, portable, and powerful. diff --git a/blog/content/second-edition/posts/10-heap-allocation/index.md b/blog/content/second-edition/posts/10-heap-allocation/index.md index dff0d0ca..51fa3932 100644 --- a/blog/content/second-edition/posts/10-heap-allocation/index.md +++ b/blog/content/second-edition/posts/10-heap-allocation/index.md @@ -422,7 +422,7 @@ pub fn init_heap( .allocate_frame() .ok_or(MapToError::FrameAllocationFailed)?; let flags = PageTableFlags::PRESENT | PageTableFlags::WRITABLE; - unsafe { mapper.map_to(page, frame, flags, frame_allocator)?.flush() }; + mapper.map_to(page, frame, flags, frame_allocator)?.flush(); } Ok(()) @@ -431,12 +431,12 @@ pub fn init_heap( The function takes mutable references to a [`Mapper`] and a [`FrameAllocator`] instance, both limited to 4KiB pages by using [`Size4KiB`] as generic parameter. The return value of the function is a [`Result`] with the unit type `()` as success variant and a [`MapToError`] as error variant, which is the error type returned by the [`Mapper::map_to`] method. Reusing the error type makes sense here because the `map_to` method is the main source of errors in this function. -[`Mapper`]:https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html -[`FrameAllocator`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html -[`Size4KiB`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/enum.Size4KiB.html +[`Mapper`]:https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html +[`FrameAllocator`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.FrameAllocator.html +[`Size4KiB`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/enum.Size4KiB.html [`Result`]: https://doc.rust-lang.org/core/result/enum.Result.html -[`MapToError`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/enum.MapToError.html -[`Mapper::map_to`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to +[`MapToError`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/enum.MapToError.html +[`Mapper::map_to`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to The implementation can be broken down into two parts: @@ -448,20 +448,20 @@ The implementation can be broken down into two parts: - We set the required `PRESENT` flag and the `WRITABLE` flag for the page. With these flags both read and write accesses are allowed, which makes sense for heap memory. - - We use the unsafe [`Mapper::map_to`] method for creating the mapping in the active page table. The method can fail, therefore we use the [question mark operator] again to forward the error to the caller. On success, the method returns a [`MapperFlush`] instance that we can use to update the [_translation lookaside buffer_] using the [`flush`] method. + - We use the [`Mapper::map_to`] method for creating the mapping in the active page table. The method can fail, therefore we use the [question mark operator] again to forward the error to the caller. On success, the method returns a [`MapperFlush`] instance that we can use to update the [_translation lookaside buffer_] using the [`flush`] method. -[`VirtAddr`]: https://docs.rs/x86_64/0.7.5/x86_64/struct.VirtAddr.html -[`Page`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/struct.Page.html -[`containing_address`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/struct.Page.html#method.containing_address -[`Page::range_inclusive`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/struct.Page.html#method.range_inclusive -[`FrameAllocator::allocate_frame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html#tymethod.allocate_frame +[`VirtAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.VirtAddr.html +[`Page`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html +[`containing_address`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html#method.containing_address +[`Page::range_inclusive`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html#method.range_inclusive +[`FrameAllocator::allocate_frame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.FrameAllocator.html#tymethod.allocate_frame [`None`]: https://doc.rust-lang.org/core/option/enum.Option.html#variant.None -[`MapToError::FrameAllocationFailed`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/enum.MapToError.html#variant.FrameAllocationFailed +[`MapToError::FrameAllocationFailed`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/enum.MapToError.html#variant.FrameAllocationFailed [`Option::ok_or`]: https://doc.rust-lang.org/core/option/enum.Option.html#method.ok_or [question mark operator]: https://doc.rust-lang.org/edition-guide/rust-2018/error-handling-and-panics/the-question-mark-operator-for-easier-error-handling.html -[`MapperFlush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html +[`MapperFlush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html [_translation lookaside buffer_]: @/second-edition/posts/08-paging-introduction/index.md#the-translation-lookaside-buffer -[`flush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush +[`flush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush The final step is to call this function from our `kernel_main`: