From 3d25c27b111deaa2cdf91d0f6cbcecf183e2500a Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Mon, 2 Dec 2019 15:08:17 +0100 Subject: [PATCH 01/98] Add message if recent update list is empty Instead of showing an empty section, print 'No notable updates recently'. --- blog/before_build.py | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/blog/before_build.py b/blog/before_build.py index b779889e..2f6e60e5 100644 --- a/blog/before_build.py +++ b/blog/before_build.py @@ -21,18 +21,22 @@ def format_number(number): with io.open("templates/auto/recent-updates.html", 'w', encoding='utf8') as recent_updates: recent_updates.truncate() - recent_updates.write(u"") repo = g.get_repo("phil-opp/blog_os") From 78ce5f3e6e3c9b17fc9cf2a6be67da8302b0345b Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Mon, 2 Dec 2019 15:13:53 +0100 Subject: [PATCH 02/98] Updates in October and November 2019 (#695) --- blog/content/status-update/2019-12-02.md | 34 ++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 blog/content/status-update/2019-12-02.md diff --git a/blog/content/status-update/2019-12-02.md b/blog/content/status-update/2019-12-02.md new file mode 100644 index 00000000..b366c0a9 --- /dev/null +++ b/blog/content/status-update/2019-12-02.md @@ -0,0 +1,34 @@ ++++ +title = "Updates in October and November 2019" +date = 2019-12-02 ++++ + +This post gives an overview of the recent updates to the _Writing an OS in Rust_ blog and the used libraries and tools. + +I moved to a new apartment mid-October and had lots of work to do there, so I didn't have the time for creating the October status update post. Therefore, this post lists the changes from both October and November. I'm slowly picking up speed again, but I still have a lot of mails in my backlog. Sorry if you haven't received an answer yet! + +## `blog_os` + +The blog itself received only a minor update: [Use panic! instead of println! + loop in double fault handler](https://github.com/phil-opp/blog_os/pull/687). This fixes an issue where a double fault during `cargo xtest` leads to an endless loop without any output on the serial port. + +We also have other news: We plan to add [Experimental Support for Community Translations](https://github.com/phil-opp/blog_os/pull/692) to the blog. While this imposes additional challenges, it makes the content accessible to people who don't speak English, so it's definitely worth trying in my opinion. The first additional language will be [Chinese](https://github.com/phil-opp/blog_os/pull/694), based on an [existing translation](https://github.com/rustcc/writing-an-os-in-rust) by [@luojia65](https://github.com/luojia65). Many thanks also to [@TheBegining](https://github.com/TheBegining) and [@Rustin-Liu](https://github.com/Rustin-Liu) for helping with the translation! + +## `bootloader` + +- [Change the way the kernel entry point is called to honor alignement ABI](https://github.com/rust-osdev/bootloader/pull/81) by [@GuillaumeDIDIER](https://github.com/GuillaumeDIDIER) (published as version 0.8.2) +- [Add support for Github Actions](https://github.com/rust-osdev/bootloader/pull/82) +- [Remove unnecessary `extern C` on panic handler to fix not-ffi-safe warning](https://github.com/rust-osdev/bootloader/pull/85) by [@cmsd2](https://github.com/cmsd2) (published as version 0.8.3) + +## `bootimage` + +- [Don't exit with expected exit code when failed to read QEMU exit code](https://github.com/rust-osdev/bootimage/pull/47) + +## `x86_64` + +- [Switch to GitHub Actions for CI](https://github.com/rust-osdev/x86_64/pull/93) +- [Use `repr C` to suppress not-ffi-safe when used with extern handler functions](https://github.com/rust-osdev/x86_64/pull/94) by [@cmsd2](https://github.com/cmsd2) (published as version 0.7.6) +- [Add `slice` and `slice_mut` methods to IDT](https://github.com/rust-osdev/x86_64/pull/95) by [@foxcob](https://github.com/foxcob) (published as version 0.7.7) + +## `cargo-xbuild` + +- [Add support for publishing and installing cross compiled crates](https://github.com/rust-osdev/cargo-xbuild/pull/47) by [@ALSchwalm](https://github.com/ALSchwalm) (published as version 0.5.18) From f02e3a23ee55608f9b13036e99719d2b2ed7baf4 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 11 Dec 2019 16:36:27 +0100 Subject: [PATCH 03/98] Update x86_64 version in Testing post --- blog/content/second-edition/posts/04-testing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 6e66feab..585155e8 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -172,7 +172,7 @@ Instead of manually invoking the `in` and `out` assembly instructions, we use th # in Cargo.toml [dependencies] -x86_64 = "0.7.5" +x86_64 = "0.8.1" ``` Now we can use the [`Port`] type provided by the crate to create an `exit_qemu` function: From 130fc3e075cea4b36ba6b8123774d4fcf52e04ca Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Tue, 10 Dec 2019 17:02:17 +0100 Subject: [PATCH 04/98] Make double fault handler diverging --- .../second-edition/posts/06-double-faults/index.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/blog/content/second-edition/posts/06-double-faults/index.md b/blog/content/second-edition/posts/06-double-faults/index.md index a0f587d9..ce2a8f91 100644 --- a/blog/content/second-edition/posts/06-double-faults/index.md +++ b/blog/content/second-edition/posts/06-double-faults/index.md @@ -79,13 +79,15 @@ lazy_static! { // new extern "x86-interrupt" fn double_fault_handler( - stack_frame: &mut InterruptStackFrame, _error_code: u64) + stack_frame: &mut InterruptStackFrame, _error_code: u64) -> ! { panic!("EXCEPTION: DOUBLE FAULT\n{:#?}", stack_frame); } ``` -Our handler prints a short error message and dumps the exception stack frame. The error code of the double fault handler is always zero, so there's no reason to print it. +Our handler prints a short error message and dumps the exception stack frame. The error code of the double fault handler is always zero, so there's no reason to print it. One difference to the breakpoint handler is that the double fault handler is [_diverging_]. The reason is that the `x86_64` architecture does not permit returning from a double fault exception. + +[_diverging_]: https://doc.rust-lang.org/stable/rust-by-example/fn/diverging.html When we start our kernel now, we should see that the double fault handler is invoked: @@ -517,7 +519,7 @@ use x86_64::structures::idt::InterruptStackFrame; extern "x86-interrupt" fn test_double_fault_handler( _stack_frame: &mut InterruptStackFrame, _error_code: u64, -) { +) -> ! { serial_println!("[ok]"); exit_qemu(QemuExitCode::Success); loop {} From 5ad8df595c52b8a419687d8846679c310e41b82c Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 11 Dec 2019 16:33:39 +0100 Subject: [PATCH 05/98] Update Paging Implementation post for new UnusedPhysFrame type --- .../posts/09-paging-implementation/index.md | 37 ++++++++++++------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/blog/content/second-edition/posts/09-paging-implementation/index.md b/blog/content/second-edition/posts/09-paging-implementation/index.md index 97bef938..51292ca9 100644 --- a/blog/content/second-edition/posts/09-paging-implementation/index.md +++ b/blog/content/second-edition/posts/09-paging-implementation/index.md @@ -31,11 +31,11 @@ To implement the approach, we will need support from the bootloader, so we'll co ### Dependency Updates -This post requires version 0.7.5 or later of the `x86_64` dependency. You can update the dependency in your `Cargo.toml`: +This post requires version 0.8.1 or later of the `x86_64` dependency. You can update the dependency in your `Cargo.toml`: ```toml [dependencies] -x86_64 = "0.7.5" +x86_64 = "0.8.1" ``` For an overview of the changes in recent versions, check out the [`x86_64` changelog]. @@ -718,7 +718,7 @@ The `create_example_mapping` function looks like this: use x86_64::{ PhysAddr, - structures::paging::{Page, PhysFrame, Mapper, Size4KiB, FrameAllocator} + structures::paging::{Page, PhysFrame, Mapper, Size4KiB, FrameAllocator, UnusedPhysFrame} }; /// Creates an example mapping for the given page to frame `0xb8000`. @@ -730,11 +730,11 @@ pub fn create_example_mapping( use x86_64::structures::paging::PageTableFlags as Flags; let frame = PhysFrame::containing_address(PhysAddr::new(0xb8000)); + // FIXME: ONLY FOR TEMPORARY TESTING + let unused_frame = unsafe { UnusedPhysFrame::new(frame) }; let flags = Flags::PRESENT | Flags::WRITABLE; - let map_to_result = unsafe { - mapper.map_to(page, frame, flags, frame_allocator) - }; + let map_to_result = mapper.map_to(page, unused_frame, flags, frame_allocator); map_to_result.expect("map_to failed").flush(); } ``` @@ -746,7 +746,12 @@ In addition to the `page` that should be mapped, the function expects a mutable [`FrameAllocator`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html [`PageSize`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/trait.PageSize.html -For the mapping, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. Calling [`map_to`] is unsafe because it's possible to break memory safety with invalid arguments, so we need to use an `unsafe` block. For a list of all possible flags, see the [_Page Table Format_] section of the previous post. +Instead of a normal `PhysFrame`, the [`map_to`] method requires an [`UnusedPhysFrame`] wrapper type to ensure that the frame is not already in use. The reason for this is that mapping the same frame twice could result in undefined behavior, for example when two different `&mut` references point to the same physical memory location. In our case, we reuse the VGA text buffer frame, which is already mapped, so we break the required condition when calling the unsafe [`UnusedPhysFrame::new`] function. However, the `create_example_mapping` function is only a temporary testing function and will be removed after this post, so it is ok. To remind us of the unsafety, we put a `FIXME` comment on the line. + +[`UnusedPhysFrame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/struct.UnusedPhysFrame.html +[`UnusedPhysFrame::new`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/struct.UnusedPhysFrame.html#method.new + +In addition to the `page` and the `unused_frame`, the `map_to` method takes a set of flags for the mapping and a reference to the `frame_allocator`, which will be explained in a moment. For the flags, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. For a list of all possible flags, see the [_Page Table Format_] section of the previous post. [_Page Table Format_]: @/second-edition/posts/08-paging-introduction/index.md#page-table-format @@ -771,7 +776,7 @@ Let's start with the simple case and assume that we don't need to create new pag pub struct EmptyFrameAllocator; unsafe impl FrameAllocator for EmptyFrameAllocator { - fn allocate_frame(&mut self) -> Option { + fn allocate_frame(&mut self) -> Option { None } } @@ -906,7 +911,7 @@ use bootloader::bootinfo::MemoryRegionType; impl BootInfoFrameAllocator { /// Returns an iterator over the usable frames specified in the memory map. - fn usable_frames(&self) -> impl Iterator { + fn usable_frames(&self) -> impl Iterator { // get usable regions from memory map let regions = self.memory_map.iter(); let usable_regions = regions @@ -917,8 +922,9 @@ impl BootInfoFrameAllocator { // transform to an iterator of frame start addresses let frame_addresses = addr_ranges.flat_map(|r| r.step_by(4096)); // create `PhysFrame` types from the start addresses - frame_addresses - .map(|addr|PhysFrame::containing_address(PhysAddr::new(addr))) + let frames = frame_addresses.map(|addr| PhysFrame::containing_address(PhysAddr::new(addr))); + // we know that the frames are really unused + frames.map(|f| unsafe { UnusedPhysFrame::new(f) }) } } ``` @@ -929,7 +935,8 @@ This function uses iterator combinator methods to transform the initial `MemoryM - Then we use the [`filter`] method to skip any reserved or otherwise unavailable regions. The bootloader updates the memory map for all the mappings it creates, so frames that are used by our kernel (code, data or stack) or to store the boot information are already marked as `InUse` or similar. Thus we can be sure that `Usable` frames are not used somewhere else. - Afterwards, we use the [`map`] combinator and Rust's [range syntax] to transform our iterator of memory regions to an iterator of address ranges. - The next step is the most complicated: We convert each range to an iterator through the `into_iter` method and then choose every 4096th address using [`step_by`]. Since 4096 bytes (= 4 KiB) is the page size, we get the start address of each frame. The bootloader page aligns all usable memory areas so that we don't need any alignment or rounding code here. By using [`flat_map`] instead of `map`, we get an `Iterator` instead of an `Iterator>`. -- Finally, we convert the start addresses to `PhysFrame` types to construct the desired `Iterator`. We then use this iterator to create and return a new `BootInfoFrameAllocator`. +- Then we convert the start addresses to `PhysFrame` types to construct the an `Iterator`. +- In the last step, we use the [`map`] combinator again to wrap each frame into the [`UnusedPhysFrame`] wrapper type. This is safe because we trust the boot information. [`MemoryRegion`]: https://docs.rs/bootloader/0.6.4/bootloader/bootinfo/struct.MemoryRegion.html [`filter`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.filter @@ -938,7 +945,7 @@ This function uses iterator combinator methods to transform the initial `MemoryM [`step_by`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.step_by [`flat_map`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.flat_map -The return type of the function uses the [`impl Trait`] feature. This way, we can specify that we return some type that implements the [`Iterator`] trait with item type `PhysFrame`, but don't need to name the concrete return type. This is important here because we _can't_ name the conrete type since it depends on unnamable closure types. +The return type of the function uses the [`impl Trait`] feature. This way, we can specify that we return some type that implements the [`Iterator`] trait with item type `UnusedPhysFrame`, but don't need to name the concrete return type. This is important here because we _can't_ name the concrete type since it depends on unnamable closure types. [`impl Trait`]: https://doc.rust-lang.org/book/ch10-02-traits.html#returning-types-that-implement-traits [`Iterator`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html @@ -951,7 +958,7 @@ Now we can implement the `FrameAllocator` trait: // in src/memory.rs unsafe impl FrameAllocator for BootInfoFrameAllocator { - fn allocate_frame(&mut self) -> Option { + fn allocate_frame(&mut self) -> Option { let frame = self.usable_frames().nth(self.next); self.next += 1; frame @@ -994,6 +1001,8 @@ With the boot info frame allocator, the mapping succeeds and we see the black-on While our `create_example_mapping` function is just some example code, we are now able to create new mappings for arbitrary pages. This will be essential for allocating memory or implementing multithreading in future posts. +At this point, we should delete the `create_example_mapping` function again to avoid accidentally invoking undefined behavior, as explained [above](#a-create-example-mapping-function). + ## Summary In this post we learned about different techniques to access the physical frames of page tables, including identity mapping, mapping of the complete physical memory, temporary mapping, and recursive page tables. We chose to map the complete physical memory since it's simple, portable, and powerful. From b65825e99a2965f20405d7f10b1a6cef2ec2eed2 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 11 Dec 2019 16:35:57 +0100 Subject: [PATCH 06/98] Update docs.rs links to point to x86_64 0.8.1 --- .../second-edition/posts/04-testing/index.md | 4 +- .../posts/05-cpu-exceptions/index.md | 16 ++++---- .../posts/06-double-faults/index.md | 6 +-- .../posts/07-hardware-interrupts/index.md | 6 +-- .../posts/08-paging-introduction/index.md | 22 +++++------ .../posts/09-paging-implementation/index.md | 38 +++++++++---------- .../posts/10-heap-allocation/index.md | 26 ++++++------- 7 files changed, 59 insertions(+), 59 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 585155e8..28c855dc 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -166,7 +166,7 @@ The functionality of the `isa-debug-exit` device is very simple. When a `value` Instead of manually invoking the `in` and `out` assembly instructions, we use the abstractions provided by the [`x86_64`] crate. To add a dependency on that crate, we add it to the `dependencies` section in our `Cargo.toml`: -[`x86_64`]: https://docs.rs/x86_64/0.7.5/x86_64/ +[`x86_64`]: https://docs.rs/x86_64/0.8.1/x86_64/ ```toml # in Cargo.toml @@ -177,7 +177,7 @@ x86_64 = "0.8.1" Now we can use the [`Port`] type provided by the crate to create an `exit_qemu` function: -[`Port`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/port/struct.Port.html +[`Port`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/port/struct.Port.html ```rust // in src/main.rs diff --git a/blog/content/second-edition/posts/05-cpu-exceptions/index.md b/blog/content/second-edition/posts/05-cpu-exceptions/index.md index 0fa6862b..840117a6 100644 --- a/blog/content/second-edition/posts/05-cpu-exceptions/index.md +++ b/blog/content/second-edition/posts/05-cpu-exceptions/index.md @@ -82,7 +82,7 @@ Don't worry about steps 4 and 5 for now, we will learn about the global descript ## An IDT Type Instead of creating our own IDT type, we will use the [`InterruptDescriptorTable` struct] of the `x86_64` crate, which looks like this: -[`InterruptDescriptorTable` struct]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html +[`InterruptDescriptorTable` struct]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html ``` rust #[repr(C)] @@ -113,10 +113,10 @@ pub struct InterruptDescriptorTable { The fields have the type [`idt::Entry`], which is a struct that represents the fields of an IDT entry (see the table above). The type parameter `F` defines the expected handler function type. We see that some entries require a [`HandlerFunc`] and some entries require a [`HandlerFuncWithErrCode`]. The page fault even has its own special type: [`PageFaultHandlerFunc`]. -[`idt::Entry`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.Entry.html -[`HandlerFunc`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/type.HandlerFunc.html -[`HandlerFuncWithErrCode`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/type.HandlerFuncWithErrCode.html -[`PageFaultHandlerFunc`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/type.PageFaultHandlerFunc.html +[`idt::Entry`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.Entry.html +[`HandlerFunc`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/type.HandlerFunc.html +[`HandlerFuncWithErrCode`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/type.HandlerFuncWithErrCode.html +[`PageFaultHandlerFunc`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/type.PageFaultHandlerFunc.html Let's look at the `HandlerFunc` type first: @@ -191,7 +191,7 @@ So the _interrupt stack frame_ looks like this: In the `x86_64` crate, the interrupt stack frame is represented by the [`InterruptStackFrame`] struct. It is passed to interrupt handlers as `&mut` and can be used to retrieve additional information about the exception's cause. The struct contains no error code field, since only some few exceptions push an error code. These exceptions use the separate [`HandlerFuncWithErrCode`] function type, which has an additional `error_code` argument. -[`InterruptStackFrame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptStackFrame.html +[`InterruptStackFrame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptStackFrame.html Note that there is currently [a bug in LLVM] that leads to wrong error code arguments. The cause of the issue is already known and a solution is [being worked on]. @@ -279,7 +279,7 @@ This error occurs because the `x86-interrupt` calling convention is still unstab In order that the CPU uses our new interrupt descriptor table, we need to load it using the [`lidt`] instruction. The `InterruptDescriptorTable` struct of the `x86_64` provides a [`load`][InterruptDescriptorTable::load] method function for that. Let's try to use it: [`lidt`]: https://www.felixcloutier.com/x86/lgdt:lidt -[InterruptDescriptorTable::load]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html#method.load +[InterruptDescriptorTable::load]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html#method.load ```rust // in src/interrupts.rs @@ -462,7 +462,7 @@ You can try this new test by running `cargo xtest` (all tests) or `cargo xtest - The `x86-interrupt` calling convention and the [`InterruptDescriptorTable`] type made the exception handling process relatively straightforward and painless. If this was too much magic for you and you like to learn all the gory details of exception handling, we got you covered: Our [“Handling Exceptions with Naked Functions”] series shows how to handle exceptions without the `x86-interrupt` calling convention and also creates its own IDT type. Historically, these posts were the main exception handling posts before the `x86-interrupt` calling convention and the `x86_64` crate existed. Note that these posts are based on the [first edition] of this blog and might be out of date. [“Handling Exceptions with Naked Functions”]: @/first-edition/extra/naked-exceptions/_index.md -[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html +[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html [first edition]: @/first-edition/_index.md ## What's next? diff --git a/blog/content/second-edition/posts/06-double-faults/index.md b/blog/content/second-edition/posts/06-double-faults/index.md index ce2a8f91..42c0acf4 100644 --- a/blog/content/second-edition/posts/06-double-faults/index.md +++ b/blog/content/second-edition/posts/06-double-faults/index.md @@ -227,7 +227,7 @@ The _Privilege Stack Table_ is used by the CPU when the privilege level changes. ### Creating a TSS Let's create a new TSS that contains a separate double fault stack in its interrupt stack table. For that we need a TSS struct. Fortunately, the `x86_64` crate already contains a [`TaskStateSegment` struct] that we can use. -[`TaskStateSegment` struct]: https://docs.rs/x86_64/0.7.5/x86_64/structures/tss/struct.TaskStateSegment.html +[`TaskStateSegment` struct]: https://docs.rs/x86_64/0.8.1/x86_64/structures/tss/struct.TaskStateSegment.html We create the TSS in a new `gdt` module (the name will make sense later): @@ -373,8 +373,8 @@ pub fn init() { We reload the code segment register using [`set_cs`] and to load the TSS using [`load_tss`]. The functions are marked as `unsafe`, so we need an `unsafe` block to invoke them. The reason is that it might be possible to break memory safety by loading invalid selectors. -[`set_cs`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/segmentation/fn.set_cs.html -[`load_tss`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/tables/fn.load_tss.html +[`set_cs`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/segmentation/fn.set_cs.html +[`load_tss`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/tables/fn.load_tss.html Now that we loaded a valid TSS and interrupt stack table, we can set the stack index for our double fault handler in the IDT: diff --git a/blog/content/second-edition/posts/07-hardware-interrupts/index.md b/blog/content/second-edition/posts/07-hardware-interrupts/index.md index 886c38b8..9f1cd8f1 100644 --- a/blog/content/second-edition/posts/07-hardware-interrupts/index.md +++ b/blog/content/second-edition/posts/07-hardware-interrupts/index.md @@ -206,7 +206,7 @@ extern "x86-interrupt" fn timer_interrupt_handler( Our `timer_interrupt_handler` has the same signature as our exception handlers, because the CPU reacts identically to exceptions and external interrupts (the only difference is that some exceptions push an error code). The [`InterruptDescriptorTable`] struct implements the [`IndexMut`] trait, so we can access individual entries through array indexing syntax. -[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.InterruptDescriptorTable.html +[`InterruptDescriptorTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.InterruptDescriptorTable.html [`IndexMut`]: https://doc.rust-lang.org/core/ops/trait.IndexMut.html In our timer interrupt handler, we print a dot to the screen. As the timer interrupt happens periodically, we would expect to see a dot appearing on each timer tick. However, when we run it we see that only a single dot is printed: @@ -331,7 +331,7 @@ pub fn _print(args: fmt::Arguments) { The [`without_interrupts`] function takes a [closure] and executes it in an interrupt-free environment. We use it to ensure that no interrupt can occur as long as the `Mutex` is locked. When we run our kernel now we see that it keeps running without hanging. (We still don't notice any dots, but this is because they're scrolling by too fast. Try to slow down the printing, e.g. by putting a `for _ in 0..10000 {}` inside the loop.) -[`without_interrupts`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/interrupts/fn.without_interrupts.html +[`without_interrupts`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/interrupts/fn.without_interrupts.html [closure]: https://doc.rust-lang.org/book/second-edition/ch13-01-closures.html We can apply the same change to our serial printing function to ensure that no deadlocks occur with it either: @@ -586,7 +586,7 @@ extern "x86-interrupt" fn keyboard_interrupt_handler( We use the [`Port`] type of the `x86_64` crate to read a byte from the keyboard's data port. This byte is called the [_scancode_] and is a number that represents the key press/release. We don't do anything with the scancode yet, we just print it to the screen: -[`Port`]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/port/struct.Port.html +[`Port`]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/port/struct.Port.html [_scancode_]: https://en.wikipedia.org/wiki/Scancode ![QEMU printing scancodes to the screen when keys are pressed](qemu-printing-scancodes.gif) diff --git a/blog/content/second-edition/posts/08-paging-introduction/index.md b/blog/content/second-edition/posts/08-paging-introduction/index.md index 4ed55aff..29f8784c 100644 --- a/blog/content/second-edition/posts/08-paging-introduction/index.md +++ b/blog/content/second-edition/posts/08-paging-introduction/index.md @@ -229,8 +229,8 @@ Let's take a closer look at the available flags: The `x86_64` crate provides types for [page tables] and their [entries], so we don't need to create these structures ourselves. -[page tables]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page_table/struct.PageTable.html -[entries]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page_table/struct.PageTableEntry.html +[page tables]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page_table/struct.PageTable.html +[entries]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page_table/struct.PageTableEntry.html ### The Translation Lookaside Buffer @@ -239,7 +239,7 @@ A 4-level page table makes the translation of virtual addresses expensive, becau Unlike the other CPU caches, the TLB is not fully transparent and does not update or remove translations when the contents of page tables change. This means that the kernel must manually update the TLB whenever it modifies a page table. To do this, there is a special CPU instruction called [`invlpg`] (“invalidate page”) that removes the translation for the specified page from the TLB, so that it is loaded again from the page table on the next access. The TLB can also be flushed completely by reloading the `CR3` register, which simulates an address space switch. The `x86_64` crate provides Rust functions for both variants in the [`tlb` module]. [`invlpg`]: https://www.felixcloutier.com/x86/INVLPG.html -[`tlb` module]: https://docs.rs/x86_64/0.7.5/x86_64/instructions/tlb/index.html +[`tlb` module]: https://docs.rs/x86_64/0.8.1/x86_64/instructions/tlb/index.html It is important to remember flushing the TLB on each page table modification because otherwise the CPU might keep using the old translation, which can lead to non-deterministic bugs that are very hard to debug. @@ -294,8 +294,8 @@ extern "x86-interrupt" fn page_fault_handler( The [`CR2`] register is automatically set by the CPU on a page fault and contains the accessed virtual address that caused the page fault. We use the [`Cr2::read`] function of the `x86_64` crate to read and print it. The [`PageFaultErrorCode`] type provides more information about the type of memory access that caused the page fault, for example whether it was caused by a read or write operation. For this reason we print it too. We can't continue execution without resolving the page fault, so we enter a [`hlt_loop`] at the end. [`CR2`]: https://en.wikipedia.org/wiki/Control_register#CR2 -[`Cr2::read`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr2.html#method.read -[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html +[`Cr2::read`]: https://docs.rs/x86_64/0.8.1/x86_64/registers/control/struct.Cr2.html#method.read +[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.PageFaultErrorCode.html [LLVM bug]: https://github.com/rust-lang/rust/issues/57270 [`hlt_loop`]: @/second-edition/posts/07-hardware-interrupts/index.md#the-hlt-instruction @@ -329,7 +329,7 @@ When we run it, we see that our page fault handler is called: The `CR2` register indeed contains `0xdeadbeaf`, the address that we tried to access. The error code tells us through the [`CAUSED_BY_WRITE`] that the fault occurred while trying to perform a write operation. It tells us even more through the [bits that are _not_ set][`PageFaultErrorCode`]. For example, the fact that the `PROTECTION_VIOLATION` flag is not set means that the page fault occurred because the target page wasn't present. -[`CAUSED_BY_WRITE`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.CAUSED_BY_WRITE +[`CAUSED_BY_WRITE`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.CAUSED_BY_WRITE We see that the current instruction pointer is `0x2031b2`, so we know that this address points to a code page. Code pages are mapped read-only by the bootloader, so reading from this address works but writing causes a page fault. You can try this by changing the `0xdeadbeaf` pointer to `0x2031b2`: @@ -353,7 +353,7 @@ By commenting out the last line, we see that the read access works, but the writ We see that the _"read worked"_ message is printed, which indicates that the read operation did not cause any errors. However, instead of the _"write worked"_ message a page fault occurs. This time the [`PROTECTION_VIOLATION`] flag is set in addition to the [`CAUSED_BY_WRITE`] flag, which indicates that the page was present, but the operation was not allowed on it. In this case, writes to the page are not allowed since code pages are mapped as read-only. -[`PROTECTION_VIOLATION`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.PROTECTION_VIOLATION +[`PROTECTION_VIOLATION`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/idt/struct.PageFaultErrorCode.html#associatedconstant.PROTECTION_VIOLATION ### Accessing the Page Tables @@ -379,9 +379,9 @@ pub extern "C" fn _start() -> ! { The [`Cr3::read`] function of the `x86_64` returns the currently active level 4 page table from the `CR3` register. It returns a tuple of a [`PhysFrame`] and a [`Cr3Flags`] type. We are only interested in the frame, so we ignore the second element of the tuple. -[`Cr3::read`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr3.html#method.read -[`PhysFrame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/frame/struct.PhysFrame.html -[`Cr3Flags`]: https://docs.rs/x86_64/0.7.5/x86_64/registers/control/struct.Cr3Flags.html +[`Cr3::read`]: https://docs.rs/x86_64/0.8.1/x86_64/registers/control/struct.Cr3.html#method.read +[`PhysFrame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/frame/struct.PhysFrame.html +[`Cr3Flags`]: https://docs.rs/x86_64/0.8.1/x86_64/registers/control/struct.Cr3Flags.html When we run it, we see the following output: @@ -391,7 +391,7 @@ Level 4 page table at: PhysAddr(0x1000) So the currently active level 4 page table is stored at address `0x1000` in _physical_ memory, as indicated by the [`PhysAddr`] wrapper type. The question now is: how can we access this table from our kernel? -[`PhysAddr`]: https://docs.rs/x86_64/0.7.5/x86_64/struct.PhysAddr.html +[`PhysAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.PhysAddr.html Accessing physical memory directly is not possible when paging is active, since programs could easily circumvent memory protection and access memory of other programs otherwise. So the only way to access the table is through some virtual page that is mapped to the physical frame at address `0x1000`. This problem of creating mappings for page table frames is a general problem, since the kernel needs to access the page tables regularly, for example when allocating a stack for a new thread. diff --git a/blog/content/second-edition/posts/09-paging-implementation/index.md b/blog/content/second-edition/posts/09-paging-implementation/index.md index 51292ca9..ce74724f 100644 --- a/blog/content/second-edition/posts/09-paging-implementation/index.md +++ b/blog/content/second-edition/posts/09-paging-implementation/index.md @@ -229,7 +229,7 @@ The above code assumes that the last level 4 entry with index `0o777` (511) is r Alternatively to performing the bitwise operations by hand, you can use the [`RecursivePageTable`] type of the `x86_64` crate, which provides safe abstractions for various page table operations. For example, the code below shows how to translate a virtual address to its mapped physical address: -[`RecursivePageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.RecursivePageTable.html +[`RecursivePageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.RecursivePageTable.html ```rust // in src/memory.rs @@ -447,7 +447,7 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! { First, we convert the `physical_memory_offset` of the `BootInfo` struct to a [`VirtAddr`] and pass it to the `active_level_4_table` function. We then use the `iter` function to iterate over the page table entries and the [`enumerate`] combinator to additionally add an index `i` to each element. We only print non-empty entries because all 512 entries wouldn't fit on the screen. -[`VirtAddr`]: https://docs.rs/x86_64/0.7.5/x86_64/struct.VirtAddr.html +[`VirtAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.VirtAddr.html [`enumerate`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.enumerate When we run it, we see the following output: @@ -560,7 +560,7 @@ The `VirtAddr` struct already provides methods to compute the indexes into the p Inside the loop, we again use the `physical_memory_offset` to convert the frame into a page table reference. We then read the entry of the current page table and use the [`PageTableEntry::frame`] function to retrieve the mapped frame. If the entry is not mapped to a frame we return `None`. If the entry maps a huge 2MiB or 1GiB page we panic for now. -[`PageTableEntry::frame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page_table/struct.PageTableEntry.html#method.frame +[`PageTableEntry::frame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page_table/struct.PageTableEntry.html#method.frame Let's test our translation function by translating some addresses: @@ -616,18 +616,18 @@ The base of the abstraction are two traits that define various page table mappin - The [`Mapper`] trait is generic over the page size and provides functions that operate on pages. Examples are [`translate_page`], which translates a given page to a frame of the same size, and [`map_to`], which creates a new mapping in the page table. - The [`MapperAllSizes`] trait implies that the implementor implements `Mapper` for all pages sizes. In addition, it provides functions that work with multiple page sizes such as [`translate_addr`] or the general [`translate`]. -[`Mapper`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html -[`translate_page`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.translate_page -[`map_to`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to -[`MapperAllSizes`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.MapperAllSizes.html -[`translate_addr`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#method.translate_addr -[`translate`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#tymethod.translate +[`Mapper`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html +[`translate_page`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.translate_page +[`map_to`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to +[`MapperAllSizes`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html +[`translate_addr`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#method.translate_addr +[`translate`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#tymethod.translate The traits only define the interface, they don't provide any implementation. The `x86_64` crate currently provides three types that implement the traits with different requirements. The [`OffsetPageTable`] type assumes that the complete physical memory is mapped to the virtual address space at some offset. The [`MappedPageTable`] is a bit more flexible: It only requires that each page table frame is mapped to the virtual address space at a calculable address. Finally, the [`RecursivePageTable`] type can be used to access page table frames through [recursive page tables](#recursive-page-tables). -[`OffsetPageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.OffsetPageTable.html -[`MappedPageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MappedPageTable.html -[`RecursivePageTable`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.RecursivePageTable.html +[`OffsetPageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.OffsetPageTable.html +[`MappedPageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MappedPageTable.html +[`RecursivePageTable`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.RecursivePageTable.html In our case, the bootloader maps the complete physical memory at an virtual address specfied by the `physical_memory_offset` variable, so we can use the `OffsetPageTable` type. To initialize it, we create a new `init` function in our `memory` module: @@ -653,7 +653,7 @@ unsafe fn active_level_4_table(physical_memory_offset: VirtAddr) The function takes the `physical_memory_offset` as an argument and returns a new `OffsetPageTable` instance with a `'static` lifetime. This means that the instance stays valid for the complete runtime of our kernel. In the function body, we first call the `active_level_4_table` function to retrieve a mutable reference to the level 4 page table. We then invoke the [`OffsetPageTable::new`] function with this reference. As the second parameter, the `new` function expects the virtual address at which the mapping of the physical memory starts, which is given in the `physical_memory_offset` variable. -[`OffsetPageTable::new`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.OffsetPageTable.html#method.new +[`OffsetPageTable::new`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.OffsetPageTable.html#method.new The `active_level_4_table` function should be only called from the `init` function from now on because it can easily lead to aliased mutable references when called multiple times, which can cause undefined behavior. For this reason, we make the function private by removing the `pub` specifier. @@ -704,8 +704,8 @@ Until now we only looked at the page tables without modifying anything. Let's ch We will use the [`map_to`] function of the [`Mapper`] trait for our implementation, so let's take a look at that function first. The documentation tells us that it takes four arguments: the page that we want to map, the frame that the page should be mapped to, a set of flags for the page table entry, and a `frame_allocator`. The frame allocator is needed because mapping the given page might require creating additional page tables, which need unused frames as backing storage. -[`map_to`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.Mapper.html#tymethod.map_to -[`Mapper`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.Mapper.html +[`map_to`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.Mapper.html#tymethod.map_to +[`Mapper`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.Mapper.html #### A `create_example_mapping` Function @@ -743,8 +743,8 @@ In addition to the `page` that should be mapped, the function expects a mutable [impl-trait-arg]: https://doc.rust-lang.org/book/ch10-02-traits.html#traits-as-parameters [generic]: https://doc.rust-lang.org/book/ch10-00-generics.html -[`FrameAllocator`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html -[`PageSize`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/trait.PageSize.html +[`FrameAllocator`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.FrameAllocator.html +[`PageSize`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/trait.PageSize.html Instead of a normal `PhysFrame`, the [`map_to`] method requires an [`UnusedPhysFrame`] wrapper type to ensure that the frame is not already in use. The reason for this is that mapping the same frame twice could result in undefined behavior, for example when two different `&mut` references point to the same physical memory location. In our case, we reuse the VGA text buffer frame, which is already mapped, so we break the required condition when calling the unsafe [`UnusedPhysFrame::new`] function. However, the `create_example_mapping` function is only a temporary testing function and will be removed after this post, so it is ok. To remind us of the unsafety, we put a `FIXME` comment on the line. @@ -759,8 +759,8 @@ The [`map_to`] function can fail, so it returns a [`Result`]. Since this is just [`Result`]: https://doc.rust-lang.org/core/result/enum.Result.html [`expect`]: https://doc.rust-lang.org/core/result/enum.Result.html#method.expect -[`MapperFlush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html -[`flush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush +[`MapperFlush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html +[`flush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush [must_use]: https://doc.rust-lang.org/std/result/#results-must-be-used #### A dummy `FrameAllocator` diff --git a/blog/content/second-edition/posts/10-heap-allocation/index.md b/blog/content/second-edition/posts/10-heap-allocation/index.md index dff0d0ca..e9631eb7 100644 --- a/blog/content/second-edition/posts/10-heap-allocation/index.md +++ b/blog/content/second-edition/posts/10-heap-allocation/index.md @@ -431,12 +431,12 @@ pub fn init_heap( The function takes mutable references to a [`Mapper`] and a [`FrameAllocator`] instance, both limited to 4KiB pages by using [`Size4KiB`] as generic parameter. The return value of the function is a [`Result`] with the unit type `()` as success variant and a [`MapToError`] as error variant, which is the error type returned by the [`Mapper::map_to`] method. Reusing the error type makes sense here because the `map_to` method is the main source of errors in this function. -[`Mapper`]:https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html -[`FrameAllocator`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html -[`Size4KiB`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/enum.Size4KiB.html +[`Mapper`]:https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html +[`FrameAllocator`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.FrameAllocator.html +[`Size4KiB`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/enum.Size4KiB.html [`Result`]: https://doc.rust-lang.org/core/result/enum.Result.html -[`MapToError`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/enum.MapToError.html -[`Mapper::map_to`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to +[`MapToError`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/enum.MapToError.html +[`Mapper::map_to`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to The implementation can be broken down into two parts: @@ -450,18 +450,18 @@ The implementation can be broken down into two parts: - We use the unsafe [`Mapper::map_to`] method for creating the mapping in the active page table. The method can fail, therefore we use the [question mark operator] again to forward the error to the caller. On success, the method returns a [`MapperFlush`] instance that we can use to update the [_translation lookaside buffer_] using the [`flush`] method. -[`VirtAddr`]: https://docs.rs/x86_64/0.7.5/x86_64/struct.VirtAddr.html -[`Page`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/struct.Page.html -[`containing_address`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/struct.Page.html#method.containing_address -[`Page::range_inclusive`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/page/struct.Page.html#method.range_inclusive -[`FrameAllocator::allocate_frame`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/trait.FrameAllocator.html#tymethod.allocate_frame +[`VirtAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.VirtAddr.html +[`Page`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html +[`containing_address`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html#method.containing_address +[`Page::range_inclusive`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html#method.range_inclusive +[`FrameAllocator::allocate_frame`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/trait.FrameAllocator.html#tymethod.allocate_frame [`None`]: https://doc.rust-lang.org/core/option/enum.Option.html#variant.None -[`MapToError::FrameAllocationFailed`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/enum.MapToError.html#variant.FrameAllocationFailed +[`MapToError::FrameAllocationFailed`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/enum.MapToError.html#variant.FrameAllocationFailed [`Option::ok_or`]: https://doc.rust-lang.org/core/option/enum.Option.html#method.ok_or [question mark operator]: https://doc.rust-lang.org/edition-guide/rust-2018/error-handling-and-panics/the-question-mark-operator-for-easier-error-handling.html -[`MapperFlush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html +[`MapperFlush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html [_translation lookaside buffer_]: @/second-edition/posts/08-paging-introduction/index.md#the-translation-lookaside-buffer -[`flush`]: https://docs.rs/x86_64/0.7.5/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush +[`flush`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/mapper/struct.MapperFlush.html#method.flush The final step is to call this function from our `kernel_main`: From b0317d0e4a35459bd9a03a75c1fef036f4aa7d26 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 11 Dec 2019 16:50:13 +0100 Subject: [PATCH 07/98] Remove now unneeded unsafe block in Heap Allocation post The `map_to` method is safe since x86_64 0.8.1. --- blog/content/second-edition/posts/10-heap-allocation/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blog/content/second-edition/posts/10-heap-allocation/index.md b/blog/content/second-edition/posts/10-heap-allocation/index.md index e9631eb7..51fa3932 100644 --- a/blog/content/second-edition/posts/10-heap-allocation/index.md +++ b/blog/content/second-edition/posts/10-heap-allocation/index.md @@ -422,7 +422,7 @@ pub fn init_heap( .allocate_frame() .ok_or(MapToError::FrameAllocationFailed)?; let flags = PageTableFlags::PRESENT | PageTableFlags::WRITABLE; - unsafe { mapper.map_to(page, frame, flags, frame_allocator)?.flush() }; + mapper.map_to(page, frame, flags, frame_allocator)?.flush(); } Ok(()) @@ -448,7 +448,7 @@ The implementation can be broken down into two parts: - We set the required `PRESENT` flag and the `WRITABLE` flag for the page. With these flags both read and write accesses are allowed, which makes sense for heap memory. - - We use the unsafe [`Mapper::map_to`] method for creating the mapping in the active page table. The method can fail, therefore we use the [question mark operator] again to forward the error to the caller. On success, the method returns a [`MapperFlush`] instance that we can use to update the [_translation lookaside buffer_] using the [`flush`] method. + - We use the [`Mapper::map_to`] method for creating the mapping in the active page table. The method can fail, therefore we use the [question mark operator] again to forward the error to the caller. On success, the method returns a [`MapperFlush`] instance that we can use to update the [_translation lookaside buffer_] using the [`flush`] method. [`VirtAddr`]: https://docs.rs/x86_64/0.8.1/x86_64/struct.VirtAddr.html [`Page`]: https://docs.rs/x86_64/0.8.1/x86_64/structures/paging/page/struct.Page.html From ef1885d63e35d13dd5b95d02f9e102e12b42e8b1 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 09:54:30 +0100 Subject: [PATCH 08/98] Update copyright year in MIT license --- LICENSE-MIT | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/LICENSE-MIT b/LICENSE-MIT index 2286d30b..de62280d 100644 --- a/LICENSE-MIT +++ b/LICENSE-MIT @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2015 Philipp Oppermann +Copyright (c) 2019 Philipp Oppermann Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: From 9406eb32f65bf05b6b7eb25a58e9190e40eb5142 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 09:55:03 +0100 Subject: [PATCH 09/98] License blog content under CC BY-NC --- blog/content/LICENSE-CC-BY-NC | 96 +++++++++++++++++++++++++++++++++++ blog/content/README.md | 15 ++++++ 2 files changed, 111 insertions(+) create mode 100644 blog/content/LICENSE-CC-BY-NC create mode 100644 blog/content/README.md diff --git a/blog/content/LICENSE-CC-BY-NC b/blog/content/LICENSE-CC-BY-NC new file mode 100644 index 00000000..165e4763 --- /dev/null +++ b/blog/content/LICENSE-CC-BY-NC @@ -0,0 +1,96 @@ +Creative Commons Attribution-NonCommercial 4.0 International Public License + +By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions. + +Section 1 – Definitions. + + Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image. + Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License. + Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights. + Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements. + Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material. + Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License. + Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license. + Licensor means the individual(s) or entity(ies) granting rights under this Public License. + NonCommercial means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange. + Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them. + Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world. + You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning. + +Section 2 – Scope. + + License grant. + Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to: + reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and + produce, reproduce, and Share Adapted Material for NonCommercial purposes only. + Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions. + Term. The term of this Public License is specified in Section 6(a). + Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material. + Downstream recipients. + Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License. + No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material. + No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i). + + Other rights. + Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise. + Patent and trademark rights are not licensed under this Public License. + To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes. + +Section 3 – License Conditions. + +Your exercise of the Licensed Rights is expressly made subject to the following conditions. + + Attribution. + + If You Share the Licensed Material (including in modified form), You must: + retain the following if it is supplied by the Licensor with the Licensed Material: + identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated); + a copyright notice; + a notice that refers to this Public License; + a notice that refers to the disclaimer of warranties; + a URI or hyperlink to the Licensed Material to the extent reasonably practicable; + indicate if You modified the Licensed Material and retain an indication of any previous modifications; and + indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License. + You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information. + If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable. + If You Share Adapted Material You produce, the Adapter's License You apply must not prevent recipients of the Adapted Material from complying with this Public License. + +Section 4 – Sui Generis Database Rights. + +Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material: + + for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only; + if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and + You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database. + +For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights. + +Section 5 – Disclaimer of Warranties and Limitation of Liability. + + Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You. + To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You. + + The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability. + +Section 6 – Term and Termination. + + This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically. + + Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates: + automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or + upon express reinstatement by the Licensor. + For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License. + For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License. + Sections 1, 5, 6, 7, and 8 survive termination of this Public License. + +Section 7 – Other Terms and Conditions. + + The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed. + Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License. + +Section 8 – Interpretation. + + For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License. + To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions. + No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor. + Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority. diff --git a/blog/content/README.md b/blog/content/README.md new file mode 100644 index 00000000..99da59dc --- /dev/null +++ b/blog/content/README.md @@ -0,0 +1,15 @@ +# Blog Content + +This folder contains the content for the _"Writing an OS in Rust"_ blog. + +## License + +This folder is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, available in [LICENSE-CC-BY-NC](LICENSE-CC-BY-NC) or under . + +All _code examples_ between markdown code blocks denoted by three backticks (`\``) are licensed under either of + +- Apache License, Version 2.0 ([LICENSE-APACHE](../../LICENSE-APACHE) or + http://www.apache.org/licenses/LICENSE-2.0) +- MIT license ([LICENSE-MIT](../../LICENSE-MIT) or http://opensource.org/licenses/MIT) + +at your option. \ No newline at end of file From a7ae6519c7152181b38da5b183910c0868733b2b Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 09:55:23 +0100 Subject: [PATCH 10/98] Clarify licensing in root README --- README.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 08c6199a..4b8e6e36 100644 --- a/README.md +++ b/README.md @@ -104,4 +104,17 @@ The current version of the blog is already the second edition. The first edition ([source code](https://github.com/phil-opp/blog_os/tree/returning_from_exceptions)) ## License -The source code is dual-licensed under MIT or the Apache License (Version 2.0). This excludes the `blog` directory. + +This project, with exception of the `blog/content` folder, is licensed under either of + +- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or + http://www.apache.org/licenses/LICENSE-2.0) +- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT) + +at your option. + +For licensing of the `blog/content` folder, see the [`blog/content/README.md`](blog/content/README.md). + +### Contribution + +Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions. From 5b7129acb78aa8b7479f3c66bbf893aed2f39e98 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 10:06:39 +0100 Subject: [PATCH 11/98] Ignore README.md in zola processing --- blog/config.toml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/blog/config.toml b/blog/config.toml index e7f374f9..7fc83ae9 100644 --- a/blog/config.toml +++ b/blog/config.toml @@ -5,6 +5,8 @@ highlight_code = true highlight_theme = "visual-studio-dark" generate_rss = true +ignored_content = ["*/README.md"] + [extra] subtitle = "Philipp Oppermann's blog" author = { name = "Philipp Oppermann" } From 7205b5f9424bbed78eac91373af8b2e687c1e1e7 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 10:08:17 +0100 Subject: [PATCH 12/98] Remove azure pipelines CI script --- .appveyor.yml | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 .appveyor.yml diff --git a/.appveyor.yml b/.appveyor.yml deleted file mode 100644 index 23538d63..00000000 --- a/.appveyor.yml +++ /dev/null @@ -1,10 +0,0 @@ -branches: - only: - - staging - - trying - - /post-.*/ - -build: false - -test_script: - - echo "Nothing to do for master branch" From f53f09cea517e2ccce61afb95cc1b0288777e4b2 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 10:10:07 +0100 Subject: [PATCH 13/98] Clarify licensing of contributions to blog/content --- blog/content/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/blog/content/README.md b/blog/content/README.md index 99da59dc..ef274ea8 100644 --- a/blog/content/README.md +++ b/blog/content/README.md @@ -12,4 +12,8 @@ All _code examples_ between markdown code blocks denoted by three backticks (`\` http://www.apache.org/licenses/LICENSE-2.0) - MIT license ([LICENSE-MIT](../../LICENSE-MIT) or http://opensource.org/licenses/MIT) -at your option. \ No newline at end of file +at your option. + +### Contribution + +Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you shall be licensed as above, without any additional terms or conditions. From 44cefeec2857bd9ae774f6c85774d4ccd7c16bd1 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 10:51:15 +0100 Subject: [PATCH 14/98] Improve rendering of "```" --- blog/content/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/content/README.md b/blog/content/README.md index ef274ea8..0740a220 100644 --- a/blog/content/README.md +++ b/blog/content/README.md @@ -6,7 +6,7 @@ This folder contains the content for the _"Writing an OS in Rust"_ blog. This folder is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, available in [LICENSE-CC-BY-NC](LICENSE-CC-BY-NC) or under . -All _code examples_ between markdown code blocks denoted by three backticks (`\``) are licensed under either of +All _code examples_ between markdown code blocks denoted by three backticks (\`\`\`) are licensed under either of - Apache License, Version 2.0 ([LICENSE-APACHE](../../LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0) From f8a50cb752454173aff0fe0e254a2469f96bd4ec Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 12 Dec 2019 10:51:57 +0100 Subject: [PATCH 15/98] Code examples are _additionally_ licensed under MIT/Apache2 --- blog/content/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/content/README.md b/blog/content/README.md index 0740a220..df08330f 100644 --- a/blog/content/README.md +++ b/blog/content/README.md @@ -6,7 +6,7 @@ This folder contains the content for the _"Writing an OS in Rust"_ blog. This folder is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, available in [LICENSE-CC-BY-NC](LICENSE-CC-BY-NC) or under . -All _code examples_ between markdown code blocks denoted by three backticks (\`\`\`) are licensed under either of +All _code examples_ between markdown code blocks denoted by three backticks (\`\`\`) are additionally licensed under either of - Apache License, Version 2.0 ([LICENSE-APACHE](../../LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0) From bf5e2cf4dcbab607c8126c9be3027c04d3f2e6ec Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Fri, 13 Dec 2019 12:36:35 +0100 Subject: [PATCH 16/98] Use

instead of

for 'Recent Updates'/'Repository' --- blog/static/css/main.css | 2 +- blog/templates/second-edition/index.html | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/blog/static/css/main.css b/blog/static/css/main.css index 69a4a432..fd587608 100644 --- a/blog/static/css/main.css +++ b/blog/static/css/main.css @@ -243,7 +243,7 @@ aside#all-posts-link { margin-bottom: 1.5rem; } - aside#recent-updates h2 { + aside#recent-updates h1 { font-size: 110%; margin-bottom: .2rem; } diff --git a/blog/templates/second-edition/index.html b/blog/templates/second-edition/index.html index 5f27c1a7..76e4b5d0 100644 --- a/blog/templates/second-edition/index.html +++ b/blog/templates/second-edition/index.html @@ -93,12 +93,12 @@