mirror of
https://github.com/phil-opp/blog_os.git
synced 2025-12-16 22:37:49 +00:00
Merge pull request #584 from phil-opp/rewrite-test-post
New testing post
This commit is contained in:
Philipp Oppermann
GitHub
23
README.md
23
README.md
@@ -10,16 +10,16 @@ The code for each post lives in a separate git branch. This makes it possible to
|
||||
|
||||
**The code for the latest post is available [here][latest-post].**
|
||||
|
||||
[latest-post]: https://github.com/phil-opp/blog_os/tree/post-10
|
||||
[latest-post]: https://github.com/phil-opp/blog_os/tree/post-09
|
||||
|
||||
You can find the branch for each post by following the `(source code)` link in the [post list](#posts) below. The branches are named `post-XX` where `XX` is the post number, for example `post-03` for the _VGA Text Mode_ post or `post-08` for the _Hardware Interrupts_ post. For build instructions, see the Readme of the respective branch.
|
||||
You can find the branch for each post by following the `(source code)` link in the [post list](#posts) below. The branches are named `post-XX` where `XX` is the post number, for example `post-03` for the _VGA Text Mode_ post or `post-07` for the _Hardware Interrupts_ post. For build instructions, see the Readme of the respective branch.
|
||||
|
||||
You can check out a branch in a subdirectory using [git worktree]:
|
||||
|
||||
[git worktree]: https://git-scm.com/docs/git-worktree
|
||||
|
||||
```
|
||||
git worktree add code post-10
|
||||
git worktree add code post-09
|
||||
```
|
||||
|
||||
The above command creates a subdirectory named `code` that contains the code for the 10th post ("Paging Implementation").
|
||||
@@ -36,29 +36,24 @@ The goal of this project is to provide step-by-step tutorials in individual blog
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-02))
|
||||
- [VGA Text Mode](https://os.phil-opp.com/vga-text-mode/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-03))
|
||||
|
||||
**Testing:**
|
||||
|
||||
- [Unit Testing](https://os.phil-opp.com/unit-testing/)
|
||||
- [Testing](https://os.phil-opp.com/testing/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-04))
|
||||
- [Integration Tests](https://os.phil-opp.com/integration-tests/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-05))
|
||||
|
||||
**Interrupts:**
|
||||
|
||||
- [CPU Exceptions](https://os.phil-opp.com/cpu-exceptions/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-06))
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-05))
|
||||
- [Double Faults](https://os.phil-opp.com/double-fault-exceptions/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-07))
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-06))
|
||||
- [Hardware Interrupts](https://os.phil-opp.com/hardware-interrupts/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-08))
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-07))
|
||||
|
||||
**Memory Management:**
|
||||
|
||||
- [Introduction to Paging](https://os.phil-opp.com/paging-introduction/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-09))
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-08))
|
||||
- [Paging Implementation](https://os.phil-opp.com/paging-implementation/)
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-10))
|
||||
([source code](https://github.com/phil-opp/blog_os/tree/post-09))
|
||||
|
||||
## First Edition Posts
|
||||
|
||||
|
||||
@@ -363,7 +363,7 @@ Instead of writing our own bootloader, which is a project on its own, we use the
|
||||
# in Cargo.toml
|
||||
|
||||
[dependencies]
|
||||
bootloader = "0.5.1"
|
||||
bootloader = "0.6.0"
|
||||
```
|
||||
|
||||
Adding the bootloader as dependency is not enough to actually create a bootable disk image. The problem is that we need to link our kernel with the bootloader after compilation, but cargo has no support for [post-build scripts].
|
||||
|
||||
990
blog/content/second-edition/posts/04-testing/index.md
Normal file
990
blog/content/second-edition/posts/04-testing/index.md
Normal file
@@ -0,0 +1,990 @@
|
||||
+++
|
||||
title = "Testing"
|
||||
weight = 4
|
||||
path = "testing"
|
||||
date = 2019-04-27
|
||||
|
||||
+++
|
||||
|
||||
This post explores unit and integration testing in `no_std` executables. We will use Rust's support for custom test frameworks to execute test functions inside our kernel. To report the results out of QEMU, we will use different features of QEMU and the `bootimage` tool.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-04`][post branch] branch.
|
||||
|
||||
[GitHub]: https://github.com/phil-opp/blog_os
|
||||
[at the bottom]: #comments
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-04
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
## Requirements
|
||||
|
||||
This post replaces the (now deprecated) [_Unit Testing_] and [_Integration Tests_] posts. It assumes that you have followed the [_A Minimal Rust Kernel_] post after 2019-04-27. Mainly, it requires that you have a `.cargo/config` file that [sets a default target] and [defines a runner executable].
|
||||
|
||||
[_Unit Testing_]: ./second-edition/posts/deprecated/04-unit-testing/index.md
|
||||
[_Integration Tests_]: ./second-edition/posts/deprecated/05-integration-tests/index.md
|
||||
[_A Minimal Rust Kernel_]: ./second-edition/posts/02-minimal-rust-kernel/index.md
|
||||
[sets a default target]: ./second-edition/posts/02-minimal-rust-kernel/index.md#set-a-default-target
|
||||
[defines a runner executable]: ./second-edition/posts/02-minimal-rust-kernel/index.md#using-cargo-run
|
||||
|
||||
## Testing in Rust
|
||||
|
||||
Rust has a [built-in test framework] that is capable of running unit tests without the need to set anything up. Just create a function that checks some results through assertions and add the `#[test]` attribute to the function header. Then `cargo test` will automatically find and execute all test functions of your crate.
|
||||
|
||||
[built-in test framework]: https://doc.rust-lang.org/book/second-edition/ch11-00-testing.html
|
||||
|
||||
Unfortunately it's a bit more complicated for `no_std` applications such as our kernel. The problem is that Rust's test framework implicitly uses the built-in [`test`] library, which depends on the standard library. This means that we can't use the default test framework for our `#[no_std]` kernel.
|
||||
|
||||
[`test`]: https://doc.rust-lang.org/test/index.html
|
||||
|
||||
We can see this when we try to run `cargo xtest` in our project:
|
||||
|
||||
```
|
||||
> cargo xtest
|
||||
Compiling blog_os v0.1.0 (/…/blog_os)
|
||||
error[E0463]: can't find crate for `test`
|
||||
```
|
||||
|
||||
Since the `test` crate depends on the standard library, it is not available for our bare metal target. While porting the `test` crate to a `#[no_std]` context [is possible][utest], it is highly unstable and requires some hacks such as redefining the `panic` macro.
|
||||
|
||||
[utest]: https://github.com/japaric/utest
|
||||
|
||||
### Custom Test Frameworks
|
||||
|
||||
Fortunately, Rust supports replacing the default test framework through the unstable [`custom_test_frameworks`] feature. This feature requires no external libraries and thus also works in `#[no_std]` environments. It works by collecting all functions annotated with a `#[test_case]` attribute and then invoking a user-specified runner function with the list of tests as argument. Thus it gives the implementation maximal control over the test process.
|
||||
|
||||
[`custom_test_frameworks`]: https://doc.rust-lang.org/unstable-book/language-features/custom-test-frameworks.html
|
||||
|
||||
The disadvantage compared to the default test framework is that many advanced features such as [`should_panic` tests] are not available. Instead, it is up to the implementation to provide such features itself if needed. This is ideal for us since we have a very special execution environment where the default implementations of such advanced features probably wouldn't work anyway. For example, the `#[should_panic]` attribute relies on stack unwinding to catch the panics, which we disabled for our kernel.
|
||||
|
||||
[`should_panic` tests]: https://doc.rust-lang.org/book/ch11-01-writing-tests.html#checking-for-panics-with-should_panic
|
||||
|
||||
To implement a custom test framework for our kernel, we add the following to our `main.rs`:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#![feature(custom_test_frameworks)]
|
||||
#![test_runner(crate::test_runner)]
|
||||
|
||||
#[cfg(test)]
|
||||
fn test_runner(tests: &[&dyn Fn()]) {
|
||||
println!("Running {} tests", tests.len());
|
||||
for test in tests {
|
||||
test();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Our runner just prints a short debug message and then calls each test function in the list. The argument type `&[&dyn Fn()]` is a [_slice_] of [_trait object_] references of the [_Fn()_] trait. It is basically a list of references to types that can be called like a function. Since the function is useless for non-test runs, we use the `#[cfg(test)]` attribute to include it only for tests.
|
||||
|
||||
[_slice_]: https://doc.rust-lang.org/std/primitive.slice.html
|
||||
[_trait object_]: https://doc.rust-lang.org/1.30.0/book/first-edition/trait-objects.html
|
||||
[_Fn()_]: https://doc.rust-lang.org/std/ops/trait.Fn.html
|
||||
|
||||
When we run `cargo xtest` now, we see that it now succeeds. However, we still see our "Hello World" instead of the message from our `test_runner`. The reason is that our `_start` function is still used as entry point. The custom test frameworks feature generates a `main` function that calls `test_runner`, but this function is ignored because we use the `#[no_main]` attribute and provide our own entry point.
|
||||
|
||||
To fix this, we first need to change the name of the generated function to something different than `main` through the `reexport_test_harness_main` attribute. Then we can call the renamed function from our `_start` function:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#![reexport_test_harness_main = "test_main"]
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
#[cfg(test)]
|
||||
test_main();
|
||||
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
We set the name of the test framework entry function to `test_main` and call it from our `_start` entry point. We use [conditional compilation] to add the call to `test_main` only in test contexts because the function is not generated on a normal run.
|
||||
|
||||
When we now execute `cargo xtest`, we see the "Running 0 tests" message from our `test_runner` on the screen. We are now ready to create our first test function:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[test_case]
|
||||
fn trivial_assertion() {
|
||||
print!("trivial assertion... ");
|
||||
assert_eq!(1, 1);
|
||||
println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
When we run `cargo xtest` now, we see the following output:
|
||||
|
||||
![QEMU printing "Hello World!", "Running 1 tests", and "trivial assertion... [ok]"](qemu-test-runner-output.png)
|
||||
|
||||
The `tests` slice passed to our `test_runner` function now contains a reference to the `trivial_assertion` function. From the `trivial assertion... [ok]` output on the screen we see that the test was called and that it succeeded.
|
||||
|
||||
After executing the tests, our `test_runner` returns to the `test_main` function, which in turn returns to our `_start` entry point function. At the end of `_start`, we enter an endless loop because the entry point function is not allowed to return. This is a problem, because we want `cargo xtest` to exit after running all tests.
|
||||
|
||||
## Exiting QEMU
|
||||
|
||||
Right now we have an endless loop at the end of our `_start` function and need to close QEMU manually on each execution of `cargo xtest`. This is unfortunate because we also want to run `cargo xtest` in scripts without user interaction. The clean solution to this would be to implement a proper way to shutdown our OS. Unfortunately this is relatively complex, because it requires implementing support for either the [APM] or [ACPI] power management standard.
|
||||
|
||||
[APM]: https://wiki.osdev.org/APM
|
||||
[ACPI]: https://wiki.osdev.org/ACPI
|
||||
|
||||
Luckily, there is an escape hatch: QEMU supports a special `isa-debug-exit` device, which provides an easy way to exit QEMU from the guest system. To enable it, we need to pass a `-device` argument to QEMU. We can do so by adding a `package.metadata.bootimage.test-args` configuration key in our `Cargo.toml`:
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[package.metadata.bootimage]
|
||||
test-args = ["-device", "isa-debug-exit,iobase=0xf4,iosize=0x04"]
|
||||
```
|
||||
|
||||
The `bootimage runner` appends the `test-args` to the default QEMU command for all test executables. For a normal `cargo xrun`, the arguments are ignored.
|
||||
|
||||
Together with the device name (`isa-debug-exit`), we pass the two parameters `iobase` and `iozize` that specify the _I/O port_ through which the device can be reached from our kernel.
|
||||
|
||||
### I/O Ports
|
||||
|
||||
There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the VGA device.
|
||||
|
||||
[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md
|
||||
|
||||
In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`).
|
||||
|
||||
The `isa-debug-exit` devices uses port-mapped I/O. The `iobase` parameter specifies on which port address the device should live (`0xf4` is a [generally unused][list of x86 I/O ports] port on the x86's IO bus) and the `iosize` specifies the port size (`0x04` means four bytes).
|
||||
|
||||
[list of x86 I/O ports]: https://wiki.osdev.org/I/O_Ports#The_list
|
||||
|
||||
### Using the Exit Device
|
||||
|
||||
The functionality of the `isa-debug-exit` device is very simple. When a `value` is written to the I/O port specified by `iobase`, it causes QEMU to exit with [exit status] `(value << 1) | 1`. So when we write `0` to the port QEMU will exit with exit status `(0 << 1) | 1 = 1` and when we write `1` to the port it will exit with exit status `(1 << 1) | 1 = 3`.
|
||||
|
||||
[exit status]: https://en.wikipedia.org/wiki/Exit_status
|
||||
|
||||
Instead of manually invoking the `in` and `out` assembly instructions, we use the abstractions provided by the [`x86_64`] crate. To add a dependency on that crate, we add it to the `dependencies` section in our `Cargo.toml`:
|
||||
|
||||
[`x86_64`]: https://docs.rs/x86_64/0.5.2/x86_64/
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[dependencies]
|
||||
x86_64 = "0.5.2"
|
||||
```
|
||||
|
||||
Now we can use the [`Port`] type provided by the crate to create an `exit_qemu` function:
|
||||
|
||||
[`Port`]: https://docs.rs/x86_64/0.5.2/x86_64/instructions/port/struct.Port.html
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[repr(u32)]
|
||||
pub enum QemuExitCode {
|
||||
Success = 0x10,
|
||||
Failed = 0x11,
|
||||
}
|
||||
|
||||
pub fn exit_qemu(exit_code: QemuExitCode) {
|
||||
use x86_64::instructions::port::Port;
|
||||
|
||||
unsafe {
|
||||
let mut port = Port::new(0xf4);
|
||||
port.write(exit_code as u32);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The function creates a new [`Port`] at `0xf4`, which is the `iobase` of the `isa-debug-exit` device. Then it writes the the passed exit code to the port. We use `u32` because we specified the `iosize` of the `isa-debug-exit` device as 4 bytes. Both operations are unsafe, because writing to an I/O port can generally result in arbitrary behavior.
|
||||
|
||||
For specifying the exit status, we create a `QemuExitCode` enum. The idea is to exit with the success exit code if all tests succeeded and with the failure exit code otherwise. The enum is marked as `#[repr(u32)]` to represent each variant by an `u32` integer. We use exit code `0x10` for success and `0x11` for failure. The actual exit codes do not matter much, as long as they don't clash with the default exit codes of QEMU. For example, using exit code `0` for success is not a good idea because it becomes `(0 << 1) | 1 = 1` after the transformation, which is the default exit code when QEMU failed to run. So we could not differentiate a QEMU error from a successful test run.
|
||||
|
||||
We can now update our `test_runner` to exit QEMU after all tests ran:
|
||||
|
||||
```rust
|
||||
fn test_runner(tests: &[&dyn Fn()]) {
|
||||
println!("Running {} tests", tests.len());
|
||||
for test in tests {
|
||||
test();
|
||||
}
|
||||
/// new
|
||||
exit_qemu(QemuExitCode::Success);
|
||||
}
|
||||
```
|
||||
|
||||
When we run `cargo xtest` now, we see that QEMU immediately closes after executing the tests. The problem is that `cargo test` interprets the test as failed even though we passed our `Success` exit code:
|
||||
|
||||
```
|
||||
> cargo xtest
|
||||
Finished dev [unoptimized + debuginfo] target(s) in 0.03s
|
||||
Running target/x86_64-blog_os/debug/deps/blog_os-5804fc7d2dd4c9be
|
||||
Building bootloader
|
||||
Compiling bootloader v0.5.3 (/home/philipp/Documents/bootloader)
|
||||
Finished release [optimized + debuginfo] target(s) in 1.07s
|
||||
Running: `qemu-system-x86_64 -drive format=raw,file=/…/target/x86_64-blog_os/debug/
|
||||
deps/bootimage-blog_os-5804fc7d2dd4c9be.bin -device isa-debug-exit,iobase=0xf4,
|
||||
iosize=0x04`
|
||||
error: test failed, to rerun pass '--bin blog_os'
|
||||
```
|
||||
|
||||
The problem is that `cargo test` considers all error codes other than `0` as failure.
|
||||
|
||||
### Success Exit Code
|
||||
|
||||
To work around this, `bootimage` provides a `test-success-exit-code` configuration key that maps a specified exit code to the exit code `0`:
|
||||
|
||||
```toml
|
||||
[package.metadata.bootimage]
|
||||
test-args = […]
|
||||
test-success-exit-code = 33 # (0x10 << 1) | 1
|
||||
```
|
||||
|
||||
With this configuration, `bootimage` maps our success exit code to exit code 0, so that `cargo xtest` correctly recognizes the success case and does no count the test as failed.
|
||||
|
||||
Our test runner now automatically closes QEMU and correctly reports the test results out. We still see the QEMU window open for a very short time, but it does not suffice to read the results. It would be nice if we could print the test results to the console instead, so that we can still see them after QEMU exited.
|
||||
|
||||
## Printing to the Console
|
||||
|
||||
To see the test output on the console, we need to send the data from our kernel to the host system somehow. There are various ways to achieve this, for example by sending the data over a TCP network interface. However, setting up a networking stack is a quite complex task, so we will choose a simpler solution instead.
|
||||
|
||||
### Serial Port
|
||||
|
||||
A simple way to send the data is to use the [serial port], an old interface standard which is no longer found in modern computers. It is easy to program and QEMU can redirect the bytes sent over serial to the host's standard output or a file.
|
||||
|
||||
[serial port]: https://en.wikipedia.org/wiki/Serial_port
|
||||
|
||||
The chips implementing a serial interface are called [UARTs]. There are [lots of UART models] on x86, but fortunately the only differences between them are some advanced features we don't need. The common UARTs today are all compatible to the [16550 UART], so we will use that model for our testing framework.
|
||||
|
||||
[UARTs]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter
|
||||
[lots of UART models]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter#UART_models
|
||||
[16550 UART]: https://en.wikipedia.org/wiki/16550_UART
|
||||
|
||||
We will use the [`uart_16550`] crate to initialize the UART and send data over the serial port. To add it as a dependency, we update our `Cargo.toml` and `main.rs`:
|
||||
|
||||
[`uart_16550`]: https://docs.rs/uart_16550
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[dependencies]
|
||||
uart_16550 = "0.2.0"
|
||||
```
|
||||
|
||||
The `uart_16550` crate contains a `SerialPort` struct that represents the UART registers, but we still need to construct an instance of it ourselves. For that we create a new `serial` module with the following content:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
mod serial;
|
||||
```
|
||||
|
||||
```rust
|
||||
// in src/serial.rs
|
||||
|
||||
use uart_16550::SerialPort;
|
||||
use spin::Mutex;
|
||||
use lazy_static::lazy_static;
|
||||
|
||||
lazy_static! {
|
||||
pub static ref SERIAL1: Mutex<SerialPort> = {
|
||||
let mut serial_port = unsafe { SerialPort::new(0x3F8) };
|
||||
serial_port.init();
|
||||
Mutex::new(serial_port)
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static` writer instance. By using `lazy_static` we can ensure that the `init` method is called exactly once on its first use.
|
||||
|
||||
Like the `isa-debug-exit` device, the UART is programmed using port I/O. Since the UART is more complex, it uses multiple I/O ports for programming different device registers. The unsafe `SerialPort::new` function expects the address of the first I/O port of the UART as argument, from which it can calculate the addresses of all needed ports. We're passing the port address `0x3F8`, which is the standard port number for the first serial interface.
|
||||
|
||||
[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics
|
||||
|
||||
To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros:
|
||||
|
||||
```rust
|
||||
#[doc(hidden)]
|
||||
pub fn _print(args: ::core::fmt::Arguments) {
|
||||
use core::fmt::Write;
|
||||
SERIAL1.lock().write_fmt(args).expect("Printing to serial failed");
|
||||
}
|
||||
|
||||
/// Prints to the host through the serial interface.
|
||||
#[macro_export]
|
||||
macro_rules! serial_print {
|
||||
($($arg:tt)*) => {
|
||||
$crate::serial::_print(format_args!($($arg)*));
|
||||
};
|
||||
}
|
||||
|
||||
/// Prints to the host through the serial interface, appending a newline.
|
||||
#[macro_export]
|
||||
macro_rules! serial_println {
|
||||
() => ($crate::serial_print!("\n"));
|
||||
($fmt:expr) => ($crate::serial_print!(concat!($fmt, "\n")));
|
||||
($fmt:expr, $($arg:tt)*) => ($crate::serial_print!(
|
||||
concat!($fmt, "\n"), $($arg)*));
|
||||
}
|
||||
```
|
||||
|
||||
The implementation is very similar to the implementation of our `print` and `println` macros. Since the `SerialPort` type already implements the [`fmt::Write`] trait, we don't need to provide our own implementation.
|
||||
|
||||
[`fmt::Write`]: https://doc.rust-lang.org/nightly/core/fmt/trait.Write.html
|
||||
|
||||
Now we can print to the serial interface instead of the VGA text buffer in our test code:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(test)]
|
||||
fn test_runner(tests: &[&dyn Fn()]) {
|
||||
serial_println!("Running {} tests", tests.len());
|
||||
[…]
|
||||
}
|
||||
|
||||
#[test_case]
|
||||
fn trivial_assertion() {
|
||||
serial_print!("trivial assertion... ");
|
||||
assert_eq!(1, 1);
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
Note that the `serial_println` macro lives directly under the root namespace because we used the `#[macro_export]` attribute, so importing it through `use crate::serial::serial_println` will not work.
|
||||
|
||||
### QEMU Arguments
|
||||
|
||||
To see the serial output from QEMU, we need use the `-serial` argument to redirect the output to stdout:
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[package.metadata.bootimage]
|
||||
test-args = [
|
||||
"-device", "isa-debug-exit,iobase=0xf4,iosize=0x04", "-serial", "mon:stdio"
|
||||
]
|
||||
```
|
||||
|
||||
When we run `cargo xtest` now, we see the test output directly in the console:
|
||||
|
||||
```
|
||||
> cargo xtest
|
||||
Finished dev [unoptimized + debuginfo] target(s) in 0.02s
|
||||
Running target/x86_64-blog_os/debug/deps/blog_os-7b7c37b4ad62551a
|
||||
Building bootloader
|
||||
Finished release [optimized + debuginfo] target(s) in 0.02s
|
||||
Running: `qemu-system-x86_64 -drive format=raw,file=/…/target/x86_64-blog_os/debug/
|
||||
deps/bootimage-blog_os-7b7c37b4ad62551a.bin -device
|
||||
isa-debug-exit,iobase=0xf4,iosize=0x04 -serial mon:stdio`
|
||||
Running 1 tests
|
||||
trivial assertion... [ok]
|
||||
```
|
||||
|
||||
However, when a test fails we still see the output inside QEMU because our panic handler still uses `println`. To simulate this, we can change the assertion in our `trivial_assertion` test to `assert_eq!(0, 1)`:
|
||||
|
||||
|
||||
|
||||
We see that the panic message is still printed to the VGA buffer, while the other test output is printed to the serial port. The panic message is quite useful, so it would be useful to see it in the console too.
|
||||
|
||||
Note that it's no longer possible to exit QEMU from the console through `Ctrl+c` when `serial mon:stdio` is passed. An alternative keyboard shortcut is `Ctrl+a` and then `x`. Or you can just close the QEMU window manually.
|
||||
|
||||
### Print a Error Message on Panic
|
||||
|
||||
To exit QEMU with an error message on a panic, we can use [conditional compilation] to use a different panic handler in testing mode:
|
||||
|
||||
[conditional compilation]: https://doc.rust-lang.org/1.30.0/book/first-edition/conditional-compilation.html
|
||||
|
||||
```rust
|
||||
// our existing panic handler
|
||||
#[cfg(not(test))] // new attribute
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
println!("{}", info);
|
||||
loop {}
|
||||
}
|
||||
|
||||
// our panic handler in test mode
|
||||
#[cfg(test)]
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
serial_println!("[failed]\n");
|
||||
serial_println!("Error: {}\n", info);
|
||||
exit_qemu(QemuExitCode::Failed);
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
For our test panic handler, we use `serial_println` instead of `println` and then exit QEMU with a failure exit code. Note that we still need an endless `loop` after the `exit_qemu` call because the compiler does not know that the `isa-debug-exit` device causes a program exit.
|
||||
|
||||
Now QEMU also exits for failed tests and prints a useful error message on the console:
|
||||
|
||||
```
|
||||
> cargo xtest
|
||||
Finished dev [unoptimized + debuginfo] target(s) in 0.02s
|
||||
Running target/x86_64-blog_os/debug/deps/blog_os-7b7c37b4ad62551a
|
||||
Building bootloader
|
||||
Finished release [optimized + debuginfo] target(s) in 0.02s
|
||||
Running: `qemu-system-x86_64 -drive format=raw,file=/…/target/x86_64-blog_os/debug/
|
||||
deps/bootimage-blog_os-7b7c37b4ad62551a.bin -device
|
||||
isa-debug-exit,iobase=0xf4,iosize=0x04 -serial mon:stdio`
|
||||
Running 1 tests
|
||||
trivial assertion... [failed]
|
||||
|
||||
Error: panicked at 'assertion failed: `(left == right)`
|
||||
left: `0`,
|
||||
right: `1`', src/main.rs:65:5
|
||||
```
|
||||
|
||||
Since we see all test output on the console now, we no longer need the QEMU window that pops up for a short time. So we can hide it completely.
|
||||
|
||||
### Hiding QEMU
|
||||
|
||||
Since we report out the complete test results using the `isa-debug-exit` device and the serial port, we don't need the QEMU window anymore. We can hide it by passing the `-display none` argument to QEMU:
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[package.metadata.bootimage]
|
||||
test-args = [
|
||||
"-device", "isa-debug-exit,iobase=0xf4,iosize=0x04", "-serial", "mon:stdio",
|
||||
"-display", "none"
|
||||
]
|
||||
```
|
||||
|
||||
Now QEMU runs completely in the background and no window is opened anymore. This is not only less annoying, but also allows our test framework to run in environments without a graphical user interface, such as CI services or [SSH] connections.
|
||||
|
||||
[SSH]: https://en.wikipedia.org/wiki/Secure_Shell
|
||||
|
||||
### Timeouts
|
||||
|
||||
Since `cargo xtest` waits until the test runner exits, a test that never returns can block the test runner forever. That's unfortunate, but not a big problem in practice since it's normally easy to avoid endless loops. In our case, however, endless loops can occur in various situations:
|
||||
|
||||
- The bootloader fails to load our kernel, which causes the system to reboot endlessly.
|
||||
- The BIOS/UEFI firmware fails to load the bootloader, which causes the same endless rebooting.
|
||||
- The CPU enters a `loop {}` statement at the end of some of our functions, for example because the QEMU exit device doesn't work properly.
|
||||
- The hardware causes a system reset, for example when a CPU exception is not caught (explained in a future post).
|
||||
|
||||
Since endless loops can occur in so many situations, the `bootimage` tool sets a timeout of 5 minutes for each test executable by default. If the test does not finish in this time, it is marked as failed and a "Timed Out" error is printed to the console. This feature ensures that tests that are stuck in an endless loop don't block `cargo xtest` forever.
|
||||
|
||||
You can try it yourself by adding a `loop {}` statement in the `trivial_assertion` test. When you run `cargo xtest`, you see that the test is marked as timed out after 5 minutes. The timeout duration is [configurable][bootimage config] through a `test-timeout` key in the Cargo.toml:
|
||||
|
||||
[bootimage config]: https://github.com/rust-osdev/bootimage#configuration
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[package.metadata.bootimage]
|
||||
test-timeout = 300 # (in seconds)
|
||||
```
|
||||
|
||||
If you don't want to wait 5 minutes for the `trivial_assertion` test to time out, you can temporarily decrease the above value.
|
||||
|
||||
After this, we no longer need the `trivial_assertion` test, so we can delete it.
|
||||
|
||||
## Testing the VGA Buffer
|
||||
|
||||
Now that we have a working test framework, we can create a few tests for our VGA buffer implementation. First, we create a very simple test to verify that `println` works without panicking:
|
||||
|
||||
```rust
|
||||
// in src/vga_buffer.rs
|
||||
|
||||
#[cfg(test)]
|
||||
use crate::{serial_print, serial_println};
|
||||
|
||||
#[test_case]
|
||||
fn test_println_simple() {
|
||||
serial_print!("test_println... ");
|
||||
println!("test_println_simple output");
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
The test just prints something to the VGA buffer. If it finishes without panicking, it means that the `println` invocation did not panic either. Since we only need the `serial_println` import in test mode, we add the `cfg(test)` attribute to avoid the unused import warning for a normal `cargo xbuild`.
|
||||
|
||||
To ensure that no panic occurs even if many lines are printed and lines are shifted off the screen, we can create another test:
|
||||
|
||||
```rust
|
||||
// in src/vga_buffer.rs
|
||||
|
||||
#[test_case]
|
||||
fn test_println_many() {
|
||||
serial_print!("test_println_many... ");
|
||||
for _ in 0..200 {
|
||||
println!("test_println_many output");
|
||||
}
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
We can also create a test function to verify that the printed lines really appear on the screen:
|
||||
|
||||
```rust
|
||||
// in src/vga_buffer.rs
|
||||
|
||||
#[test_case]
|
||||
fn test_println_output() {
|
||||
serial_print!("test_println_output... ");
|
||||
|
||||
let s = "Some test string that fits on a single line";
|
||||
println!("{}", s);
|
||||
for (i, c) in s.chars().enumerate() {
|
||||
let screen_char = WRITER.lock().buffer.chars[BUFFER_HEIGHT - 2][i].read();
|
||||
assert_eq!(char::from(screen_char.ascii_character), c);
|
||||
}
|
||||
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
The function defines a test string, prints it using `println`, and then iterates over the screen characters of the static `WRITER`, which represents the vga text buffer. Since `println` prints to the last screen line and then immediately appends a newline, the string should appear on line `BUFFER_HEIGHT - 2`.
|
||||
|
||||
By using [`enumerate`], we count the number of iterations in the variable `i`, which we then use for loading the screen character corresponding to `c`. By comparing the `ascii_character` of the screen character with `c`, we ensure that each character of the string really appears in the vga text buffer.
|
||||
|
||||
[`enumerate`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.enumerate
|
||||
|
||||
As you can imagine, we could create many more test functions, for example a function that tests that no panic occurs when printing very long lines and that they're wrapped correctly. Or a function for testing that newlines, non-printable characters, and non-unicode characters are handled correctly.
|
||||
|
||||
For the rest of this post, however, we will explain how to create _integration tests_ to test the interaction of different components together.
|
||||
|
||||
## Integration Tests
|
||||
|
||||
The convention for [integration tests] in Rust is to put them into a `tests` directory in the project root (i.e. next to the `src` directory). Both the default test framework and custom test frameworks will automatically pick up and execute all tests in that directory.
|
||||
|
||||
[integration tests]: https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests
|
||||
|
||||
All integration tests are their own executables and completely separate from our `main.rs`. This means that each test needs to define its own entry point function. Let's create an example integration test named `basic_boot` to see how it works in detail:
|
||||
|
||||
```rust
|
||||
// in tests/basic_boot.rs
|
||||
|
||||
#![no_std]
|
||||
#![no_main]
|
||||
#![feature(custom_test_frameworks)]
|
||||
#![test_runner(crate::test_runner)]
|
||||
#![reexport_test_harness_main = "test_main"]
|
||||
|
||||
use core::panic::PanicInfo;
|
||||
|
||||
#[no_mangle] // don't mangle the name of this function
|
||||
pub extern "C" fn _start() -> ! {
|
||||
test_main();
|
||||
|
||||
loop {}
|
||||
}
|
||||
|
||||
fn test_runner(tests: &[&dyn Fn()]) {
|
||||
unimplemented!();
|
||||
}
|
||||
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
Since integration tests are separate executables, we need to provide all the crate attributes (`no_std`, `no_main`, `test_runner`, etc.) again. We also need to create a new entry point function `_start`, which calls the test entry point function `test_main`. We don't need any `cfg(test)` attributes because integration test executables are never built in non-test mode.
|
||||
|
||||
We use the [`unimplemented`] macro that always panics as a placeholder for the `test_runner` function and just `loop` in the `panic` handler for now. Ideally, we want to implement these functions exactly as we did in our `main.rs` using the `serial_println` macro and the `exit_qemu` function. The problem is that we don't have access to these functions since tests are built completely separately of our `main.rs` executable.
|
||||
|
||||
[`unimplemented`]: https://doc.rust-lang.org/core/macro.unimplemented.html
|
||||
|
||||
If you run `cargo xtest` at this stage, you will get an endless loop because the panic handler loops endlessly. You need to use the `Ctrl-a` and then `x` keyboard shortcut for exiting QEMU.
|
||||
|
||||
### Create a Library
|
||||
|
||||
To make the required functions available to our integration test, we need to split off a library from our `main.rs`, which can be included by other crates and integration test executables. To do this, we create a new `src/lib.rs` file:
|
||||
|
||||
```rust
|
||||
// src/lib.rs
|
||||
|
||||
#![no_std]
|
||||
```
|
||||
|
||||
Like the `main.rs`, the `lib.rs` is a special file that is automatically recognized by cargo. The library is a separate compilation unit, so we need to specify the `#![no_std]` attribute again.
|
||||
|
||||
To make our library work with `cargo xtest`, we need to also add the test functions and attributes:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
#![cfg_attr(test, no_main)]
|
||||
#![feature(custom_test_frameworks)]
|
||||
#![test_runner(crate::test_runner)]
|
||||
#![reexport_test_harness_main = "test_main"]
|
||||
|
||||
use core::panic::PanicInfo;
|
||||
|
||||
pub fn test_runner(tests: &[&dyn Fn()]) {
|
||||
serial_println!("Running {} tests", tests.len());
|
||||
for test in tests {
|
||||
test();
|
||||
}
|
||||
exit_qemu(QemuExitCode::Success);
|
||||
}
|
||||
|
||||
pub fn test_panic_handler(info: &PanicInfo) -> ! {
|
||||
serial_println!("[failed]\n");
|
||||
serial_println!("Error: {}\n", info);
|
||||
exit_qemu(QemuExitCode::Failed);
|
||||
loop {}
|
||||
}
|
||||
|
||||
/// Entry point for `cargo xtest`
|
||||
#[cfg(test)]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
test_main();
|
||||
loop {}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
test_panic_handler(info)
|
||||
}
|
||||
```
|
||||
|
||||
To make our `test_runner` available to executables and integration tests, we don't apply the `cfg(test)` attribute to it and make it public. We also factor out the implementation of our panic handler into a public `test_panic_handler` function, so that it is available for executables too.
|
||||
|
||||
Since our `lib.rs` is tested independently of our `main.rs`, we need to add a `_start` entry point and a panic handler when the library is compiled in test mode. By using the [`cfg_attr`] crate attribute, we conditionally enable the `no_main` attribute in this case.
|
||||
|
||||
[`cfg_attr`]: https://doc.rust-lang.org/reference/conditional-compilation.html#the-cfg_attr-attribute
|
||||
|
||||
We also move over the `QemuExitCode` enum and the `exit_qemu` function and make them public:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[repr(u32)]
|
||||
pub enum QemuExitCode {
|
||||
Success = 0x10,
|
||||
Failed = 0x11,
|
||||
}
|
||||
|
||||
pub fn exit_qemu(exit_code: QemuExitCode) {
|
||||
use x86_64::instructions::port::Port;
|
||||
|
||||
unsafe {
|
||||
let mut port = Port::new(0xf4);
|
||||
port.write(exit_code as u32);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Now executables and integration tests can import these functions from the library and don't need to define their own implementations. To also make `println` and `serial_println` available, we move the module declarations too:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
pub mod serial;
|
||||
pub mod vga_buffer;
|
||||
```
|
||||
|
||||
We make the modules public to make them usable from outside of our library. This is also required for making our `println` and `serial_println` macros usable, since they use the `_print` functions of the modules.
|
||||
|
||||
Now we can update our `main.rs` to use the library:
|
||||
|
||||
```rust
|
||||
// src/main.rs
|
||||
|
||||
#![no_std]
|
||||
#![no_main]
|
||||
#![feature(custom_test_frameworks)]
|
||||
#![test_runner(blog_os::test_runner)]
|
||||
#![reexport_test_harness_main = "test_main"]
|
||||
|
||||
use core::panic::PanicInfo;
|
||||
use blog_os::println;
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
#[cfg(test)]
|
||||
test_main();
|
||||
|
||||
loop {}
|
||||
}
|
||||
|
||||
/// This function is called on panic.
|
||||
#[cfg(not(test))]
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
println!("{}", info);
|
||||
loop {}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
blog_os::test_panic_handler(info)
|
||||
}
|
||||
```
|
||||
|
||||
The library is usable like a normal external crate. It is called like our crate, which is `blog_os` in our case. The above code uses the `blog_os::test_runner` function in the `test_runner` attribute and the `blog_os::test_panic_handler` function in our `cfg(test)` panic handler. It also imports the `println` macro to make it available to our `_start` and `panic` functions.
|
||||
|
||||
At this point, `cargo xrun` and `cargo xtest` should work again. Of course, `cargo xtest` still loops endlessly (remember the `ctrl+a` and then `x` shortcut to exit). Let's fix this by using the required library functions in our integration test.
|
||||
|
||||
### Completing the Integration Test
|
||||
|
||||
Like our `src/main.rs`, our `tests/basic_boot.rs` executable can import types from our new library. This allows us to import the missing components to complete our test.
|
||||
|
||||
```rust
|
||||
// in tests/basic_boot.rs
|
||||
|
||||
#![test_runner(blog_os::test_runner)]
|
||||
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
blog_os::test_panic_handler(info)
|
||||
}
|
||||
```
|
||||
|
||||
Instead of reimplementing the test runner, we use the `test_runner` function from our library. For our `panic` handler, we call the `blog_os::test_panic_handler` function like we did in our `main.rs`.
|
||||
|
||||
Now `cargo xtest` exits normally again. When you run it, you see that it builds and runs the tests for our `lib.rs`, `main.rs`, and `basic_boot.rs` separately after each other. For the `main.rs` and the `basic_boot` integration test, it reports "Running 0 tests" since these files don't have any functions annotated with `#[test_case]`.
|
||||
|
||||
We can now add tests to our `basic_boot.rs`. For example, we can test that `println` works without panicking, like we did in the vga buffer tests:
|
||||
|
||||
```rust
|
||||
// in tests/basic_boot.rs
|
||||
|
||||
use blog_os::{println, serial_print, serial_println};
|
||||
|
||||
#[test_case]
|
||||
fn test_println() {
|
||||
serial_print!("test_println... ");
|
||||
println!("test_println output");
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
When we run `cargo xtest` now, we see that it finds and executes the test function.
|
||||
|
||||
The test might seem a bit useless right now since it's almost identical to one of the VGA buffer tests. However, in the future the `_start` functions of our `main.rs` and `lib.rs` might grow and call various initialization routines before running the `test_main` function, so that the two tests are executed in very different environments.
|
||||
|
||||
By testing `println` in a `basic_boot` environment without calling any initialization routines in `_start`, we can ensure that `println` works right after booting. This is important because we rely on it e.g. for printing panic messages.
|
||||
|
||||
### Future Tests
|
||||
|
||||
The power of integration tests is that they're treated as completely separate executables. This gives them complete control over the environment, which makes it possible to test that the code interacts correctly with the CPU or hardware devices.
|
||||
|
||||
Our `basic_boot` test is a very simple example for an integration test. In the future, our kernel will become much more featureful and interact with the hardware in various ways. By adding integration tests, we can ensure that these interactions work (and keep working) as expected. Some ideas for possible future tests are:
|
||||
|
||||
- **CPU Exceptions**: When the code performs invalid operations (e.g. divides by zero), the CPU throws an exception. The kernel can register handler functions for such exceptions. An integration test could verify that the correct exception handler is called when a CPU exception occurs or that the execution continues correctly after resolvable exceptions.
|
||||
- **Page Tables**: Page tables define which memory regions are valid and accessible. By modifying the page tables, it is possible to allocate new memory regions, for example when launching programs. An integration test could perform some modifications of the page tables in the `_start` function and then verify that the modifications have the desired effects in `#[test_case]` functions.
|
||||
- **Userspace Programs**: Userspace programs are programs with limited access to the system's resources. For example, they don't have access to kernel data structures or to the memory of other programs. An integration test could launch userspace programs that perform forbidden operations and verify that the kernel prevents them all.
|
||||
|
||||
As you can imagine, many more tests are possible. By adding such tests, we can ensure that we don't break them accidentally when we add new features to our kernel or refactor our code. This is especially important when our kernel becomes larger and more complex.
|
||||
|
||||
## Testing Our Panic Handler
|
||||
|
||||
Another thing that we can test with an integration test is that our panic handler is called correctly. The idea is to deliberately cause a panic in the test function and exit with a success exit code in the panic handler.
|
||||
|
||||
Since we exit from our panic handler, the panicking test never returns to the test runner. For this reason, it does not make sense to add more than one test because subsequent tests are never executed. For cases like this, where only a single test function exists, we can disable the test runner completely and run our test directly in the `_start` function.
|
||||
|
||||
### No Harness
|
||||
|
||||
The `harness` flag defines whether a test runner is used for an integration test. When it's set to `false`, both the default test runner and the custom test runner feature are disabled, so that the test is treated like a normal executable.
|
||||
|
||||
Let's create a panic handler test with a disabled `harness` flag. First, we create a skeleton for the test at `tests/panic_handler.rs`:
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
#![no_std]
|
||||
#![no_main]
|
||||
|
||||
use core::panic::PanicInfo;
|
||||
use blog_os::{QemuExitCode, exit_qemu};
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
exit_qemu(QemuExitCode::Failed);
|
||||
loop {}
|
||||
}
|
||||
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
exit_qemu(QemuExitCode::Failed);
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
The code is similar to the `basic_boot` test with the difference that no test attributes are needed and no runner function is called. We immediately exit with an error from the `_start` entry point and the panic handler for now and first try to get it to compile.
|
||||
|
||||
If you run `cargo xtest` now, you will get an error that the `test` crate is missing. This error occurs because we didn't set a custom test framework, so that the compiler tries to use the default test framework, which is unavailable for our panic. By setting the `harness` flag to `false` for the test in our `Cargo.toml`, we can fix this error:
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[[test]]
|
||||
name = "panic_handler"
|
||||
harness = false
|
||||
```
|
||||
|
||||
Now the test compiles fine, but fails of course since we always exit with an error exit code.
|
||||
|
||||
### Implementing the Test
|
||||
|
||||
Let's complete the implementation of our panic handler test:
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
use blog_os::{serial_print, serial_println, QemuExitCode, exit_qemu};
|
||||
|
||||
const MESSAGE: &str = "Example panic message from panic_handler test";
|
||||
const PANIC_LINE: u32 = 14; // adjust this when moving the `panic!` call
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
serial_print!("panic_handler... ");
|
||||
panic!(MESSAGE); // must be in line `PANIC_LINE`
|
||||
}
|
||||
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
serial_println!("[ok]");
|
||||
exit_qemu(QemuExitCode::Success);
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
We immediately `panic` in our `_start` function with a `MESSAGE`. In the panic handler, we exit with a success exit code. We don't need a `qemu_exit` call at the end of our `_start` function, since the Rust compiler knows for sure that the code after the `panic` is unreachable. If we run the test with `cargo xtest --test panic_handler` now, we see that it succeeds as expected.
|
||||
|
||||
We will need the `MESSAGE` and `PANIC_LINE` constants in the next section. The `PANIC_LINE` constant specifies the line number that contains the `panic!` invocation, which is `14` in our case (but it might be different for you).
|
||||
|
||||
### Checking the `PanicInfo`
|
||||
|
||||
To ensure that the given `PanicInfo` is correct, we can extend the `panic` function to check that the reported message and file/line information are correct:
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
check_message(info);
|
||||
check_location(info);
|
||||
|
||||
// same as before
|
||||
serial_println!("[ok]");
|
||||
exit_qemu(QemuExitCode::Success);
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
We will show the implementation of `check_message` and `check_location` in a moment. Before that, we create a `fail` helper function that can be used to print an error message and exit QEMU with an failure exit code:
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
fn fail(error: &str) -> ! {
|
||||
serial_println!("[failed]");
|
||||
serial_println!("{}", error);
|
||||
exit_qemu(QemuExitCode::Failed);
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
Now we can implement our `check_location` function:
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
fn check_location(info: &PanicInfo) {
|
||||
let location = info.location().unwrap_or_else(|| fail("no location"));
|
||||
if location.file() != file!() {
|
||||
fail("file name wrong");
|
||||
}
|
||||
if location.line() != PANIC_LINE {
|
||||
fail("file line wrong");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The function takes queries the location information from the `PanicInfo` and fails if it does not exist. It then checks that the reported file name is correct by comparing it with the output of the compiler-provided [`file!`] macro. To check the reported line number, it compares it with the `PANIC_LINE` constant that we manually defined above.
|
||||
|
||||
[`file!`]: https://doc.rust-lang.org/core/macro.file.html
|
||||
|
||||
#### Checking the Panic Message
|
||||
|
||||
Checking the reported panic message is a bit more complicated. The reason is that the [`PanicInfo::message`] function returns a [`fmt::Arguments`] instance that can't be compared with our `MESSAGE` string directly. To work around this, we need to create a `CompareMessage` struct:
|
||||
|
||||
[`PanicInfo::message`]: https://doc.rust-lang.org/core/macro.file.html
|
||||
[`fmt::Arguments`]: https://doc.rust-lang.org/core/fmt/struct.Arguments.html
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
/// Compares a `fmt::Arguments` instance with the `MESSAGE` string.
|
||||
///
|
||||
/// To use this type, write the `fmt::Arguments` instance to it using the
|
||||
/// `write` macro. If a message component matches `MESSAGE`, the equals
|
||||
/// field is set to true.
|
||||
struct CompareMessage {
|
||||
equals: bool,
|
||||
}
|
||||
|
||||
impl fmt::Write for CompareMessage {
|
||||
fn write_str(&mut self, s: &str) -> fmt::Result {
|
||||
if s == MESSAGE {
|
||||
self.equals = true;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The trick is to implement the [`fmt::Write`] trait, which is called by the [`write`] macro with `&str` arguments. This makes it possible to compare the panic arguments with `MESSAGE`. By the way, this is the same trait that we implemented for our VGA buffer writer in order to print to the screen.
|
||||
|
||||
[`fmt::Write`]: https://doc.rust-lang.org/core/fmt/trait.Write.html
|
||||
[`write`]: https://doc.rust-lang.org/core/macro.write.html
|
||||
|
||||
The above code only works for messages with a single component. This means that it works for `panic!("some message")`, but not for `panic!("some {}", message)`. This isn't ideal, but good enough for our test.
|
||||
|
||||
With the `CompareMessage` type, we can finally implement our `check_message` function:
|
||||
|
||||
```rust
|
||||
// in tests/panic_handler.rs
|
||||
|
||||
#![feature(panic_info_message)] // at the top of the file
|
||||
|
||||
fn check_message(info: &PanicInfo) {
|
||||
let message = info.message().unwrap_or_else(|| fail("no message"));
|
||||
let mut compare_message = CompareMessage { equals: false };
|
||||
write!(&mut compare_message, "{}", message)
|
||||
.unwrap_or_else(|_| fail("write failed"));
|
||||
if !compare_message.equals {
|
||||
fail("message not equal to expected message");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The function uses the [`PanicInfo::message`] function to get the panic message. If no message is reported, it calls `fail` to fail the test. Since the function is unstable, we need to add the `#![feature(panic_info_message)]` attribute at the top of our test file.
|
||||
|
||||
[`PanicInfo::message`]: https://doc.rust-lang.org/core/panic/struct.PanicInfo.html#method.message
|
||||
|
||||
After querying the message, the function constructs a `CompareMessage` instance and writes the message to it using the `write!` macro. Afterwards it reads the `equals` field and fails the test if the panic message does not equal `MESSAGE`.
|
||||
|
||||
Now we can run the test using `cargo xtest --test panic_handler`. We see that it passes, which means that the reported panic info is correct. If we use a wrong line number in `PANIC_LINE` or panic with an additional character through `panic!("{}x", MESSAGE)`, we see that the test indeed fails.
|
||||
|
||||
## Summary
|
||||
|
||||
Testing is a very useful technique to ensure that certain components have a desired behavior. Even if they cannot show the absence of bugs, they're still an useful tool for finding them and especially for avoiding regressions.
|
||||
|
||||
This post explained how to set up a test framework for our Rust kernel. We used the custom test frameworks feature of Rust to implement support for a simple `#[test_case]` attribute in our bare-metal environment. By using the `isa-debug-exit` device of QEMU, our test runner can exit QEMU after running the tests and report the test status out. To print error messages to the console instead of the VGA buffer, we created a basic driver for the serial port.
|
||||
|
||||
After creating some tests for our `println` macro, we explored integration tests in the second half of the post. We learned that they live in the `tests` directory and are treated as completely separate executables. To give them access to the `exit_qemu` function and the `serial_println` macro, we moved most of our code into a library that can be imported by all executables and integration tests. Since integration tests run in their own separate environment, they make it possible to test the interactions with the hardware or Rust's panic system.
|
||||
|
||||
We now have a test framework that runs in a realistic environment inside QEMU. By creating more tests in future posts, we can keep our kernel maintainable when it becomes more complex.
|
||||
|
||||
## What's next?
|
||||
|
||||
In the next post, we will explore _CPU exceptions_. These exceptions are thrown by the CPU when something illegal happens, such as a division by zero or an access to an unmapped memory page (a so-called “page fault”). Being able to catch and examine these exceptions is very important for debugging future errors. Exception handling is also very similar to the handling of hardware interrupts, which is required for keyboard support.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 8.9 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.2 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 8.3 KiB After Width: | Height: | Size: 8.3 KiB |
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "CPU Exceptions"
|
||||
weight = 6
|
||||
weight = 5
|
||||
path = "cpu-exceptions"
|
||||
date = 2018-06-17
|
||||
|
||||
@@ -12,11 +12,11 @@ CPU exceptions occur in various erroneous situations, for example when accessing
|
||||
|
||||
<!-- more -->
|
||||
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-06`][post branch] branch.
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-05`][post branch] branch.
|
||||
|
||||
[GitHub]: https://github.com/phil-opp/blog_os
|
||||
[at the bottom]: #comments
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-06
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-05
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
@@ -371,21 +371,37 @@ pub fn init_idt() {
|
||||
|
||||
Note how this solution requires no `unsafe` blocks. The `lazy_static!` macro does use `unsafe` behind the scenes, but it is abstracted away in a safe interface.
|
||||
|
||||
### Testing it
|
||||
Now we should be able to handle breakpoint exceptions! Let's try it in our `_start` function:
|
||||
### Running it
|
||||
|
||||
The last step for making exceptions work in our kernel it to call the `init_idt` function from our `main.rs`. Instead of calling it directly, we introduce a general `init` function in our `lib.rs`:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
pub fn init() {
|
||||
interrupts::init_idt();
|
||||
}
|
||||
```
|
||||
|
||||
With this function we now have a central place for initialization routines that can be shared between the different `_start` functions in our `main.rs`, `lib.rs`, and integration tests.
|
||||
|
||||
Now we can update the `_start` function of our `main.rs` to call `init` and then trigger a breakpoint exception:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::interrupts::init_idt();
|
||||
blog_os::init(); // new
|
||||
|
||||
// invoke a breakpoint exception
|
||||
x86_64::instructions::interrupts::int3();
|
||||
x86_64::instructions::interrupts::int3(); // new
|
||||
|
||||
// as before
|
||||
#[cfg(test)]
|
||||
test_main();
|
||||
|
||||
println!("It did not crash!");
|
||||
loop {}
|
||||
@@ -402,77 +418,45 @@ We see that the interrupt stack frame tells us the instruction and stack pointer
|
||||
|
||||
### Adding a Test
|
||||
|
||||
Let's create an integration test that ensures that the above continues to work. For that we create a file named `test-exception-breakpoint.rs`:
|
||||
|
||||
```rust
|
||||
// in src/bin/test-exception-breakpoint.rs
|
||||
|
||||
#![no_std]
|
||||
#![cfg_attr(not(test), no_main)]
|
||||
#![cfg_attr(test, allow(dead_code, unused_macros, unused_imports))]
|
||||
|
||||
use core::panic::PanicInfo;
|
||||
use blog_os::{exit_qemu, serial_println};
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
blog_os::interrupts::init_idt();
|
||||
|
||||
x86_64::instructions::interrupts::int3();
|
||||
|
||||
serial_println!("ok");
|
||||
|
||||
unsafe { exit_qemu(); }
|
||||
loop {}
|
||||
}
|
||||
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
serial_println!("failed");
|
||||
|
||||
serial_println!("{}", info);
|
||||
|
||||
unsafe { exit_qemu(); }
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
It is similar to our `main.rs`, but instead of printing "It did not crash!" to the VGA buffer, it prints "ok" to the serial output and calls `exit_qemu`. This allows the `bootimage` tool to detect that our code successfully continued after invoking the `int3` instruction. If our `panic_handler` is called, we instead print `failed` to signalize the failure to `bootimage`.
|
||||
|
||||
You can try this new test by running `bootimage test`.
|
||||
|
||||
### Fixing `cargo test` on Windows
|
||||
|
||||
The `x86-interrupt` calling convention has an annoying problem: There is a bug in LLVM that leads to a "LLVM ERROR: offset is not a multiple of 16" when compiling a function with the `x86-interrupt` calling convention _for a Windows target_. Normally this is no problem, since we only compile for our custom `x86_64-blog_os.json` target. But `cargo test` compiles our crate for the host system, so the error occurs if the host system is Windows.
|
||||
|
||||
To fix this problem, we add a conditional compilation attribute, so that the `x86-interrupt` functions are not compiled on Windows systems. We don't have any unit tests that rely on the `interrupts` module, so we can simply skip compilation of the whole module:
|
||||
|
||||
```rust
|
||||
// in src/interrupts.rs
|
||||
|
||||
// LLVM throws an error if a function with the
|
||||
// x86-interrupt calling convention is compiled
|
||||
// for a Windows system.
|
||||
#![cfg(not(windows))]
|
||||
```
|
||||
|
||||
The bang ("!") after the hash ("#") indicates that this is an [inner attribute] and applies to the module we're in. Without the bang it would only apply to the next item in the file. Note that inner attributes must be right at the beginning of a module.
|
||||
|
||||
[inner attribute]: https://doc.rust-lang.org/reference/attributes.html
|
||||
|
||||
We could achieve the same effect by using an _outer_ attribute (without a bang) in our `lib.rs`:
|
||||
Let's create a test that ensures that the above continues to work. First, we update the `_start` function to also call `init`:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
#[cfg(not(windows))] // no bang ("!") after the hash ("#")
|
||||
pub mod interrupts;
|
||||
/// Entry point for `cargo xtest`
|
||||
#[cfg(test)]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
init(); // new
|
||||
test_main();
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
Both variants have the exact same effects, so it comes down to personal preference which one to use. I prefer the inner attribute in this case because it does not clutter our `lib.rs` with a workaround for a LLVM bug, but either way is fine.
|
||||
Remember, this `_start` function is used when running `cargo xtest --lib`, since Rust's tests the `lib.rs` completely independent of the `main.rs`. We need to call `init` here to set up an IDT before running the tests.
|
||||
|
||||
Now we can create a `test_breakpoint_exception` test:
|
||||
|
||||
```rust
|
||||
// in src/interrupts.rs
|
||||
|
||||
#[cfg(test)]
|
||||
use crate::{serial_print, serial_println};
|
||||
|
||||
#[test_case]
|
||||
fn test_breakpoint_exception() {
|
||||
serial_print!("test_breakpoint_exception...");
|
||||
// invoke a breakpoint exception
|
||||
x86_64::instructions::interrupts::int3();
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
Apart from printing status messages through the [serial port], the test invokes the `int3` function to trigger a breakpoint exception. By checking that the execution continues afterwards, we verify that our breakpoint handler is working correctly.
|
||||
|
||||
[serial port]: ./second-edition/posts/04-testing/index.md#serial-port
|
||||
|
||||
You can try this new test by running `cargo xtest` (all tests) or `cargo xtest --lib` (only tests of `lib.rs` and its modules). You should see `test_breakpoint_exception...[ok]` in the output.
|
||||
|
||||
## Too much Magic?
|
||||
The `x86-interrupt` calling convention and the [`InterruptDescriptorTable`] type made the exception handling process relatively straightforward and painless. If this was too much magic for you and you like to learn all the gory details of exception handling, we got you covered: Our [“Handling Exceptions with Naked Functions”] series shows how to handle exceptions without the `x86-interrupt` calling convention and also creates its own IDT type. Historically, these posts were the main exception handling posts before the `x86-interrupt` calling convention and the `x86_64` crate existed. Note that these posts are based on the [first edition] of this blog and might be out of date.
|
||||
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "Double Faults"
|
||||
weight = 7
|
||||
weight = 6
|
||||
path = "double-fault-exceptions"
|
||||
date = 2018-06-18
|
||||
|
||||
@@ -10,18 +10,18 @@ This post explores the double fault exception in detail, which occurs when the C
|
||||
|
||||
<!-- more -->
|
||||
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-07`][post branch] branch.
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-06`][post branch] branch.
|
||||
|
||||
[GitHub]: https://github.com/phil-opp/blog_os
|
||||
[at the bottom]: #comments
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-07
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-06
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
## What is a Double Fault?
|
||||
In simplified terms, a double fault is a special exception that occurs when the CPU fails to invoke an exception handler. For example, it occurs when a page fault is triggered but there is no page fault handler registered in the [Interrupt Descriptor Table][IDT] (IDT). So it's kind of similar to catch-all blocks in programming languages with exceptions, e.g. `catch(...)` in C++ or `catch(Exception e)` in Java or C#.
|
||||
|
||||
[IDT]: ./second-edition/posts/06-cpu-exceptions/index.md#the-interrupt-descriptor-table
|
||||
[IDT]: ./second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
|
||||
|
||||
A double fault behaves like a normal exception. It has the vector number `8` and we can define a normal handler function for it in the IDT. It is really important to provide a double fault handler, because if a double fault is unhandled a fatal _triple fault_ occurs. Triple faults can't be caught and most hardware reacts with a system reset.
|
||||
|
||||
@@ -31,18 +31,21 @@ Let's provoke a double fault by triggering an exception for that we didn't defin
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::interrupts::init_idt();
|
||||
blog_os::init();
|
||||
|
||||
// trigger a page fault
|
||||
unsafe {
|
||||
*(0xdeadbeef as *mut u64) = 42;
|
||||
};
|
||||
|
||||
// as before
|
||||
#[cfg(test)]
|
||||
test_main();
|
||||
|
||||
println!("It did not crash!");
|
||||
loop {}
|
||||
}
|
||||
@@ -78,8 +81,7 @@ lazy_static! {
|
||||
extern "x86-interrupt" fn double_fault_handler(
|
||||
stack_frame: &mut InterruptStackFrame, _error_code: u64)
|
||||
{
|
||||
println!("EXCEPTION: DOUBLE FAULT\n{:#?}", stack_frame);
|
||||
loop {}
|
||||
panic!("EXCEPTION: DOUBLE FAULT\n{:#?}", stack_frame);
|
||||
}
|
||||
```
|
||||
|
||||
@@ -160,12 +162,11 @@ Let's try it ourselves! We can easily provoke a kernel stack overflow by calling
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle] // don't mangle the name of this function
|
||||
pub extern "C" fn _start() -> ! {
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::interrupts::init_idt();
|
||||
blog_os::init();
|
||||
|
||||
fn stack_overflow() {
|
||||
stack_overflow(); // for each recursion, the return address is pushed
|
||||
@@ -174,8 +175,7 @@ pub extern "C" fn _start() -> ! {
|
||||
// trigger a stack overflow
|
||||
stack_overflow();
|
||||
|
||||
println!("It did not crash!");
|
||||
loop {}
|
||||
[…] // test_main(), println(…), and loop {}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -196,7 +196,7 @@ struct InterruptStackTable {
|
||||
|
||||
For each exception handler, we can choose a stack from the IST through the `options` field in the corresponding [IDT entry]. For example, we could use the first stack in the IST for our double fault handler. Then the CPU would automatically switch to this stack whenever a double fault occurs. This switch would happen before anything is pushed, so it would prevent the triple fault.
|
||||
|
||||
[IDT entry]: ./second-edition/posts/06-cpu-exceptions/index.md#the-interrupt-descriptor-table
|
||||
[IDT entry]: ./second-edition/posts/05-cpu-exceptions/index.md#the-interrupt-descriptor-table
|
||||
|
||||
### The IST and TSS
|
||||
The Interrupt Stack Table (IST) is part of an old legacy structure called _[Task State Segment]_ \(TSS). The TSS used to hold various information (e.g. processor register state) about a task in 32-bit mode and was for example used for [hardware context switching]. However, hardware context switching is no longer supported in 64-bit mode and the format of the TSS changed completely.
|
||||
@@ -301,7 +301,7 @@ We use `lazy_static` again, because Rust's const evaluator is not powerful enoug
|
||||
|
||||
#### Loading the GDT
|
||||
|
||||
To load our GDT we create a new `gdt::init` function, that we call from our `_start` function:
|
||||
To load our GDT we create a new `gdt::init` function, that we call from our `init` function:
|
||||
|
||||
```rust
|
||||
// in src/gdt.rs
|
||||
@@ -310,22 +310,15 @@ pub fn init() {
|
||||
GDT.load();
|
||||
}
|
||||
|
||||
// in src/main.rs
|
||||
// in src/lib.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::gdt::init();
|
||||
blog_os::interrupts::init_idt();
|
||||
|
||||
[…]
|
||||
pub fn init() {
|
||||
gdt::init();
|
||||
interrupts::init_idt();
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
Now our GDT is loaded, but we still see the boot loop on stack overflow.
|
||||
Now our GDT is loaded (since the `_start` function calls `init`), but we still see the boot loop on stack overflow.
|
||||
|
||||
### The final Steps
|
||||
|
||||
@@ -408,11 +401,132 @@ That's it! Now the CPU should switch to the double fault stack whenever a double
|
||||
|
||||
|
||||
|
||||
From now on we should never see a triple fault again!
|
||||
From now on we should never see a triple fault again! To ensure that we don't accidentally break the above, we should add a test for this.
|
||||
|
||||
To ensure that we don't accidentally break the above, we should add a integration test for this. We don't show the code here for space reasons, but you can find it [on Github][stack overflow test]. The idea is to do a `serial_println!("ok");` from the double fault handler to ensure that it is called. The rest of the file is is very similar to our `main.rs`.
|
||||
## A Stack Overflow Test
|
||||
|
||||
[stack overflow test]: https://github.com/phil-opp/blog_os/blob/post-07/src/bin/test-exception-double-fault-stack-overflow.rs
|
||||
To test our new `gdt` module and ensure that the double fault handler is correctly called on a stack overflow, we can add an integration test. The idea is to do provoke a double fault in the test function and verify that the double fault handler is called.
|
||||
|
||||
Let's start with a minimal skeleton:
|
||||
|
||||
```rust
|
||||
// in tests/stack_overflow.rs
|
||||
|
||||
#![no_std]
|
||||
#![no_main]
|
||||
|
||||
use core::panic::PanicInfo;
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
unimplemented!();
|
||||
}
|
||||
|
||||
#[panic_handler]
|
||||
fn panic(info: &PanicInfo) -> ! {
|
||||
blog_os::test_panic_handler(info)
|
||||
}
|
||||
```
|
||||
|
||||
Like our `panic_handler` test, the test will run [without a test harness]. The reason is that we can't continue execution after a double fault, so more than one test doesn't make sense. To disable, the test harness for the test, we add the following to our `Cargo.toml`:
|
||||
|
||||
```toml
|
||||
# in Cargo.toml
|
||||
|
||||
[[test]]
|
||||
name = "stack_overflow"
|
||||
harness = false
|
||||
```
|
||||
|
||||
[without a test harness]: ./second-edition/posts/04-testing/index.md#no-harness
|
||||
|
||||
Now `cargo xtest --test stack_overflow` should compile successfully. The test fails of course, since the `unimplemented` macro panics.
|
||||
|
||||
### Implementing `_start`
|
||||
|
||||
The implementation of the `_start` function looks like this:
|
||||
|
||||
```rust
|
||||
// in tests/stack_overflow.rs
|
||||
|
||||
use blog_os::serial_print;
|
||||
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
serial_print!("stack_overflow... ");
|
||||
|
||||
blog_os::gdt::init();
|
||||
init_test_idt();
|
||||
|
||||
// trigger a stack overflow
|
||||
stack_overflow();
|
||||
|
||||
panic!("Execution continued after stack overflow");
|
||||
}
|
||||
|
||||
#[allow(unconditional_recursion)]
|
||||
fn stack_overflow() {
|
||||
stack_overflow(); // for each recursion, the return address is pushed
|
||||
}
|
||||
```
|
||||
|
||||
We call our `gdt::init` function to initialize a new GDT. Instead of calling our `interrupts::init_idt` function, we call a `init_test_idt` function that will be explained in a moment. The reason is that we want to register a custom double fault handler that does a `exit_qemu(QemuExitCode::Success)` instead of panicking.
|
||||
|
||||
The `stack_overflow` function is identical to the function in our `main.rs`. We additionally added the `allow(unconditional_recursion)` attribute to silence the warning that the function recurses endlessly.
|
||||
|
||||
### The Test IDT
|
||||
|
||||
As noted above, the test needs its own IDT with a custom double fault handler. The implementation looks like this:
|
||||
|
||||
```rust
|
||||
// in tests/stack_overflow.rs
|
||||
|
||||
use lazy_static::lazy_static;
|
||||
use x86_64::structures::idt::InterruptDescriptorTable;
|
||||
|
||||
lazy_static! {
|
||||
static ref TEST_IDT: InterruptDescriptorTable = {
|
||||
let mut idt = InterruptDescriptorTable::new();
|
||||
unsafe {
|
||||
idt.double_fault
|
||||
.set_handler_fn(test_double_fault_handler)
|
||||
.set_stack_index(blog_os::gdt::DOUBLE_FAULT_IST_INDEX);
|
||||
}
|
||||
|
||||
idt
|
||||
};
|
||||
}
|
||||
|
||||
pub fn init_test_idt() {
|
||||
TEST_IDT.load();
|
||||
}
|
||||
```
|
||||
|
||||
The implementation is very similar to our normal IDT in `interrupts.rs`. Like in the normal IDT, we set a stack index into the IST for the double fault handler in order to switch to a separate stack. The `init_test_idt` function loads the IDT on the CPU through the `load` method.
|
||||
|
||||
### The Double Fault Handler
|
||||
|
||||
The only missing piece is our double fault handler. It looks like this:
|
||||
|
||||
```rust
|
||||
// in tests/stack_overflow.rs
|
||||
|
||||
use blog_os::{exit_qemu, QemuExitCode, serial_println};
|
||||
use x86_64::structures::idt::InterruptStackFrame;
|
||||
|
||||
extern "x86-interrupt" fn test_double_fault_handler(
|
||||
_stack_frame: &mut InterruptStackFrame,
|
||||
_error_code: u64,
|
||||
) {
|
||||
serial_println!("[ok]");
|
||||
exit_qemu(QemuExitCode::Success);
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
When the double fault handler is called, we exit QEMU with a success exit code, which marks the test as passed. Since integration tests are completely separate executables, we need to set `#![feature(abi_x86_interrupt)]` attribute again at the top of our test file.
|
||||
|
||||
Now we can run our test through `cargo xtest --test stack_overflow` (or `cargo xtest` to run all tests). As expected, we see the `stack_overflow... [ok]` output in the console. TRy to comment out the `set_stack_index` line: it should cause the test to fail.
|
||||
|
||||
## Summary
|
||||
In this post we learned what a double fault is and under which conditions it occurs. We added a basic double fault handler that prints an error message and added an integration test for it.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 11 KiB |
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "Hardware Interrupts"
|
||||
weight = 8
|
||||
weight = 7
|
||||
path = "hardware-interrupts"
|
||||
date = 2018-10-22
|
||||
|
||||
@@ -10,11 +10,11 @@ In this post we set up the programmable interrupt controller to correctly forwar
|
||||
|
||||
<!-- more -->
|
||||
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-08`][post branch] branch.
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-07`][post branch] branch.
|
||||
|
||||
[GitHub]: https://github.com/phil-opp/blog_os
|
||||
[at the bottom]: #comments
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-08
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-07
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
@@ -66,7 +66,7 @@ This graphic shows the typical assignment of interrupt lines. We see that most o
|
||||
|
||||
Each controller can be configured through two [I/O ports], one “command” port and one “data” port. For the primary controller these ports are `0x20` (command) and `0x21` (data). For the secondary controller they are `0xa0` (command) and `0xa1` (data). For more information on how the PICs can be configured see the [article on osdev.org].
|
||||
|
||||
[I/O ports]: ./second-edition/posts/05-integration-tests/index.md#port-i-o
|
||||
[I/O ports]: ./second-edition/posts/04-testing/index.md#i-o-ports
|
||||
[article on osdev.org]: https://wiki.osdev.org/8259_PIC
|
||||
|
||||
### Implementation
|
||||
@@ -109,24 +109,15 @@ We're setting the offsets for the pics to the range 32–47 as we noted above. B
|
||||
|
||||
[spin mutex lock]: https://docs.rs/spin/0.4.8/spin/struct.Mutex.html#method.lock
|
||||
|
||||
We can now initialize the 8259 PIC from our `_start` function:
|
||||
We can now initialize the 8259 PIC in our `init` function:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
// in src/lib.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
use blog_os::interrupts::PICS; // new
|
||||
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::gdt::init();
|
||||
blog_os::interrupts::init_idt();
|
||||
unsafe { PICS.lock().initialize() }; // new
|
||||
|
||||
println!("It did not crash!");
|
||||
loop {}
|
||||
pub fn init() {
|
||||
gdt::init();
|
||||
interrupts::init_idt();
|
||||
unsafe { interrupts::PICS.lock().initialize() }; // new
|
||||
}
|
||||
```
|
||||
|
||||
@@ -141,22 +132,13 @@ If all goes well we should continue to see the "It did not crash" message when e
|
||||
Until now nothing happened because interrupts are still disabled in the CPU configuration. This means that the CPU does not listen to the interrupt controller at all, so no interrupts can reach the CPU. Let's change that:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
// in src/lib.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
use blog_os::interrupts::PICS;
|
||||
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::gdt::init();
|
||||
blog_os::interrupts::init_idt();
|
||||
unsafe { PICS.lock().initialize() };
|
||||
pub fn init() {
|
||||
gdt::init();
|
||||
interrupts::init_idt();
|
||||
unsafe { interrupts::PICS.lock().initialize() };
|
||||
x86_64::instructions::interrupts::enable(); // new
|
||||
|
||||
println!("It did not crash!");
|
||||
loop {}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -309,7 +291,6 @@ We can easily provoke such a deadlock in our kernel by printing something in the
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
[…]
|
||||
@@ -374,6 +355,86 @@ pub fn _print(args: ::core::fmt::Arguments) {
|
||||
|
||||
Note that disabling interrupts shouldn't be a general solution. The problem is that it increases the worst case interrupt latency, i.e. the time until the system reacts to an interrupt. Therefore interrupts should be only disabled for a very short time.
|
||||
|
||||
## Fixing a Race Condition
|
||||
|
||||
If you run `cargo xtest` you might see the `test_println_output` test failing:
|
||||
|
||||
```
|
||||
> cargo xtest --lib
|
||||
[…]
|
||||
Running 4 tests
|
||||
test_breakpoint_exception...[ok]
|
||||
test_println... [ok]
|
||||
test_println_many... [ok]
|
||||
test_println_output... [failed]
|
||||
|
||||
Error: panicked at 'assertion failed: `(left == right)`
|
||||
left: `'.'`,
|
||||
right: `'S'`', src/vga_buffer.rs:205:9
|
||||
```
|
||||
|
||||
The reason is a _race condition_ between the test and our timer handler. Remember, the test looks like this:
|
||||
|
||||
```rust
|
||||
// in src/vga_buffer.rs
|
||||
|
||||
#[test_case]
|
||||
fn test_println_output() {
|
||||
serial_print!("test_println_output... ");
|
||||
|
||||
let s = "Some test string that fits on a single line";
|
||||
println!("{}", s);
|
||||
for (i, c) in s.chars().enumerate() {
|
||||
let screen_char = WRITER.lock().buffer.chars[BUFFER_HEIGHT - 2][i].read();
|
||||
assert_eq!(char::from(screen_char.ascii_character), c);
|
||||
}
|
||||
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
The test prints a string to the VGA buffer and then checks the output by manually iterating over the `buffer_chars` array. The race condition occurs because the timer interrupt handler might run between the `println` and the reading of the screen characters. Note that this isn't a dangerous _data race_, which Rust completely prevents at compile time. See the [_Rustonomicon_][nomicon-races] for details.
|
||||
|
||||
[nomicon-races]: https://doc.rust-lang.org/nomicon/races.html
|
||||
|
||||
To fix this, we need to keep the `WRITER` locked for the complete duration of the test, so that the timer handler can't write a `.` to the screen in between. The fixed test looks like this:
|
||||
|
||||
```rust
|
||||
// in src/vga_buffer.rs
|
||||
|
||||
#[test_case]
|
||||
fn test_println_output() {
|
||||
use core::fmt::Write;
|
||||
use x86_64::instructions::interrupts;
|
||||
|
||||
serial_print!("test_println_output... ");
|
||||
|
||||
let s = "Some test string that fits on a single line";
|
||||
interrupts::without_interrupts(|| {
|
||||
let mut writer = WRITER.lock();
|
||||
writeln!(writer, "\n{}", s).expect("writeln failed");
|
||||
for (i, c) in s.chars().enumerate() {
|
||||
let screen_char = writer.buffer.chars[BUFFER_HEIGHT - 2][i].read();
|
||||
assert_eq!(char::from(screen_char.ascii_character), c);
|
||||
}
|
||||
});
|
||||
|
||||
serial_println!("[ok]");
|
||||
}
|
||||
```
|
||||
|
||||
We performed the following changes:
|
||||
|
||||
- We keep the writer locked for the complete test by using the `lock()` method explicitly. Instead of `println`, we use the [`writeln`] marco that allows printing to an already locked writer.
|
||||
- To avoid another deadlock, we disable interrupts for the tests duration. Otherwise the test might get interrupted while the writer is still locked.
|
||||
- Since the timer interrupt handler can still run before the test, we print an additional newline `\n` before printing the string `s`. This way, we avoid test failure when the timer handler already printed some `.` characters to the current line.
|
||||
|
||||
[`writeln`]: https://doc.rust-lang.org/core/macro.writeln.html
|
||||
|
||||
With the above changes, `cargo xtest` now deterministically succeeds again.
|
||||
|
||||
This was a very harmless race condition that only caused a test failure. As you can imagine, other race conditions can be much more difficult to debug due to their non-deterministic nature. Luckily, Rust prevents us from data races, which are the most serious class of race conditions since they can cause all kinds of undefined behavior, including system crashes and silent memory corruptions.
|
||||
|
||||
## The `hlt` Instruction
|
||||
|
||||
Until now we used a simple empty loop statement at the end of our `_start` and `panic` functions. This causes the CPU to spin endlessly and thus works as expected. But it is also very inefficient, because the CPU continues to run at full speed even though there's no work to do. You can see this problem in your task manager when you run your kernel: The QEMU process needs close to 100% CPU the whole time.
|
||||
@@ -401,7 +462,6 @@ We can now use this `hlt_loop` instead of the endless loops in our `_start` and
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
[…]
|
||||
@@ -420,7 +480,29 @@ fn panic(info: &PanicInfo) -> ! {
|
||||
|
||||
```
|
||||
|
||||
We can also use `hlt_loop` in our double fault exception handler as well:
|
||||
Let's update our `lib.rs` as well:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
/// Entry point for `cargo xtest`
|
||||
#[cfg(test)]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
init();
|
||||
test_main();
|
||||
hlt_loop(); // new
|
||||
}
|
||||
|
||||
pub fn test_panic_handler(info: &PanicInfo) -> ! {
|
||||
serial_println!("[failed]\n");
|
||||
serial_println!("Error: {}\n", info);
|
||||
exit_qemu(QemuExitCode::Failed);
|
||||
hlt_loop(); // new
|
||||
}
|
||||
```
|
||||
|
||||
We can also use `hlt_loop` in our double fault exception handler:
|
||||
|
||||
```rust
|
||||
// in src/interrupts.rs
|
||||
@@ -497,7 +579,7 @@ We now see that a `k` appears on the screen when we press a key. However, this o
|
||||
|
||||
To find out _which_ key was pressed, we need to query the keyboard controller. We do this by reading from the data port of the PS/2 controller, which is the [I/O port] with number `0x60`:
|
||||
|
||||
[I/O port]: ./second-edition/posts/05-integration-tests/index.md#port-i-o
|
||||
[I/O port]: ./second-edition/posts/04-testing/index.md#i-o-ports
|
||||
|
||||
```rust
|
||||
// in src/interrupts.rs
|
||||
|
Before Width: | Height: | Size: 8.5 KiB After Width: | Height: | Size: 8.5 KiB |
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 10 KiB |
|
Before Width: | Height: | Size: 5.8 KiB After Width: | Height: | Size: 5.8 KiB |
|
Before Width: | Height: | Size: 4.9 KiB After Width: | Height: | Size: 4.9 KiB |
|
Before Width: | Height: | Size: 4.6 KiB After Width: | Height: | Size: 4.6 KiB |
|
Before Width: | Height: | Size: 2.9 KiB After Width: | Height: | Size: 2.9 KiB |
|
Before Width: | Height: | Size: 8.1 KiB After Width: | Height: | Size: 8.1 KiB |
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "Introduction to Paging"
|
||||
weight = 9
|
||||
weight = 8
|
||||
path = "paging-introduction"
|
||||
date = 2019-01-14
|
||||
|
||||
@@ -10,11 +10,11 @@ This post introduces _paging_, a very common memory management scheme that we wi
|
||||
|
||||
<!-- more -->
|
||||
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-09`][post branch] branch.
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-08`][post branch] branch.
|
||||
|
||||
[GitHub]: https://github.com/phil-opp/blog_os
|
||||
[at the bottom]: #comments
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-09
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-08
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
@@ -257,7 +257,7 @@ Paging makes our kernel already relatively safe, since every memory access that
|
||||
|
||||
Let's try to cause a page fault by accessing some memory outside of our kernel. First, we create a page fault handler and register it in our IDT, so that we see a page fault exception instead of a generic [double fault] :
|
||||
|
||||
[double fault]: ./second-edition/posts/07-double-faults/index.md
|
||||
[double fault]: ./second-edition/posts/06-double-faults/index.md
|
||||
|
||||
```rust
|
||||
// in src/interrupts.rs
|
||||
@@ -280,7 +280,6 @@ extern "x86-interrupt" fn page_fault_handler(
|
||||
stack_frame: &mut InterruptStackFrame,
|
||||
_error_code: PageFaultErrorCode,
|
||||
) {
|
||||
use crate::hlt_loop;
|
||||
use x86_64::registers::control::Cr2;
|
||||
|
||||
println!("EXCEPTION: PAGE FAULT");
|
||||
@@ -296,31 +295,27 @@ The [`CR2`] register is automatically set by the CPU on a page fault and contain
|
||||
[`Cr2::read`]: https://docs.rs/x86_64/0.5.2/x86_64/registers/control/struct.Cr2.html#method.read
|
||||
[`PageFaultErrorCode`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/idt/struct.PageFaultErrorCode.html
|
||||
[LLVM bug]: https://github.com/rust-lang/rust/issues/57270
|
||||
[`hlt_loop`]: ./second-edition/posts/08-hardware-interrupts/index.md#the
|
||||
[`hlt_loop`]: ./second-edition/posts/07-hardware-interrupts/index.md#the
|
||||
|
||||
Now we can try to access some memory outside our kernel:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
use blog_os::interrupts::PICS;
|
||||
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
// set up the IDT first, otherwise we would enter a boot loop instead of
|
||||
// invoking our page fault handler
|
||||
blog_os::gdt::init();
|
||||
blog_os::interrupts::init_idt();
|
||||
unsafe { PICS.lock().initialize() };
|
||||
x86_64::instructions::interrupts::enable();
|
||||
blog_os::init();
|
||||
|
||||
// new
|
||||
let ptr = 0xdeadbeaf as *mut u32;
|
||||
unsafe { *ptr = 42; }
|
||||
|
||||
// as before
|
||||
#[cfg(test)]
|
||||
test_main();
|
||||
|
||||
println!("It did not crash!");
|
||||
blog_os::hlt_loop();
|
||||
}
|
||||
@@ -353,18 +348,18 @@ Let's try to take a look at the page tables that our kernel runs on:
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start() -> ! {
|
||||
[…] // initialize GDT, IDT, PICS
|
||||
println!("Hello World{}", "!");
|
||||
|
||||
blog_os::init();
|
||||
|
||||
use x86_64::registers::control::Cr3;
|
||||
|
||||
let (level_4_page_table, _) = Cr3::read();
|
||||
println!("Level 4 page table at: {:?}", level_4_page_table.start_address());
|
||||
|
||||
println!("It did not crash!");
|
||||
blog_os::hlt_loop();
|
||||
[…] // test_main(), println(…), and hlt_loop()
|
||||
}
|
||||
```
|
||||
|
||||
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 29 KiB |
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
|
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 15 KiB After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 30 KiB |
|
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 24 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "Paging Implementation"
|
||||
weight = 10
|
||||
weight = 9
|
||||
path = "paging-implementation"
|
||||
date = 2019-03-14
|
||||
+++
|
||||
@@ -9,11 +9,11 @@ This post shows how to implement paging support in our kernel. It first explores
|
||||
|
||||
<!-- more -->
|
||||
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-10`][post branch] branch.
|
||||
This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-09`][post branch] branch.
|
||||
|
||||
[GitHub]: https://github.com/phil-opp/blog_os
|
||||
[at the bottom]: #comments
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-10
|
||||
[post branch]: https://github.com/phil-opp/blog_os/tree/post-09
|
||||
|
||||
|
||||
<aside class="post_aside">
|
||||
@@ -37,21 +37,21 @@ I hope that you enjoy this new post!
|
||||
|
||||
The [previous post] gave an introduction to the concept of paging. It motivated paging by comparing it with segmentation, explained how paging and page tables work, and then introduced the 4-level page table design of `x86_64`. We found out that the bootloader already set up a page table hierarchy for our kernel, which means that our kernel already runs on virtual addresses. This improves safety since illegal memory accesses cause page fault exceptions instead of modifying arbitrary physical memory.
|
||||
|
||||
[previous post]: ./second-edition/posts/09-paging-introduction/index.md
|
||||
[previous post]: ./second-edition/posts/08-paging-introduction/index.md
|
||||
|
||||
The post ended with the problem that we [can't access the page tables from our kernel][end of previous post] because they are stored in physical memory and our kernel already runs on virtual addresses. This post continues at this point and explores different approaches of making the page table frames accessible to our kernel. We will discuss the advantages and drawbacks of each approach and then decide for an approach for our kernel.
|
||||
|
||||
[end of previous post]: ./second-edition/posts/09-paging-introduction/index.md#accessing-the-page-tables
|
||||
[end of previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
|
||||
|
||||
To implement the approach, we will need support from the bootloader, so we'll configure it first. Afterward, we will implement a function that traverses the page table hierarchy in order to translate virtual to physical addresses. Finally, we learn how to create new mappings in the page tables and how to find unused memory frames for creating new page tables.
|
||||
|
||||
### Dependency Updates
|
||||
|
||||
This post requires version 0.5.1 or later of the `bootloader` dependency and version 0.5.2 or later of the `x86_64` dependency. You can update the dependencies in your `Cargo.toml`:
|
||||
This post requires version 0.6.0 or later of the `bootloader` dependency and version 0.5.2 or later of the `x86_64` dependency. You can update the dependencies in your `Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
bootloader = "0.5.1"
|
||||
bootloader = "0.6.0"
|
||||
x86_64 = "0.5.2"
|
||||
```
|
||||
|
||||
@@ -83,7 +83,7 @@ In this example, we see various identity-mapped page table frames. This way the
|
||||
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000 KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `28 KiB` because it would collide with the already mapped page at `1004 MiB`. So we have to look further until we find a large enough unmapped area, for example at `1008 KiB`. This is a similar fragmentation problem as with [segmentation].
|
||||
|
||||
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
|
||||
[segmentation]: ./second-edition/posts/09-paging-introduction/index.md#fragmentation
|
||||
[segmentation]: ./second-edition/posts/08-paging-introduction/index.md#fragmentation
|
||||
|
||||
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000 KiB memory region starting at `1008 KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000 KiB` and `2008 KiB` anymore, because we can't identity map it.
|
||||
|
||||
@@ -210,7 +210,7 @@ Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 i
|
||||
|
||||
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
|
||||
|
||||
[sign extension]: ./second-edition/posts/09-paging-introduction/index.md#paging-on-x86
|
||||
[sign extension]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86
|
||||
|
||||
We use [octal] numbers for representing the addresses since each octal character represents three bits, which allows us to clearly separate the 9-bit indexes of the different page table levels. This isn't possible with the hexadecimal system where each character represents four bits.
|
||||
|
||||
@@ -305,7 +305,7 @@ We choose the first approach for our kernel since it is simple, platform-indepen
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
bootloader = { version = "0.5.1", features = ["map_physical_memory"]}
|
||||
bootloader = { version = "0.6.0", features = ["map_physical_memory"]}
|
||||
```
|
||||
|
||||
With this feature enabled, the bootloader maps the complete physical memory to some unused virtual address range. To communicate the virtual address range to our kernel, the bootloader passes a _boot information_ structure.
|
||||
@@ -327,7 +327,6 @@ The bootloader passes the `BootInfo` struct to our kernel in the form of a `&'st
|
||||
|
||||
use bootloader::BootInfo;
|
||||
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
pub extern "C" fn _start(boot_info: &'static BootInfo) -> ! { // new argument
|
||||
[…]
|
||||
@@ -342,7 +341,7 @@ Since our `_start` function is called externally from the bootloader, no checkin
|
||||
|
||||
To make sure that the entry point function has always the correct signature that the bootloader expects, the `bootloader` crate provides an [`entry_point`] macro that provides a type-checked way to define a Rust function as the entry point. Let's rewrite our entry point function to use this macro:
|
||||
|
||||
[`entry_point`]: https://docs.rs/bootloader/0.5.1/bootloader/macro.entry_point.html
|
||||
[`entry_point`]: https://docs.rs/bootloader/0.6.0/bootloader/macro.entry_point.html
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
@@ -351,7 +350,6 @@ use bootloader::{BootInfo, entry_point};
|
||||
|
||||
entry_point!(kernel_main);
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…]
|
||||
}
|
||||
@@ -359,9 +357,32 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
|
||||
We no longer need to use `extern "C"` or `no_mangle` for our entry point, as the macro defines the real lower level `_start` entry point for us. The `kernel_main` function is now a completely normal Rust function, so we can choose an arbitrary name for it. The important thing is that it is type-checked so that a compilation error occurs when we use a wrong function signature, for example by adding an argument or changing the argument type.
|
||||
|
||||
Let's perform the same change in our `lib.rs`:
|
||||
|
||||
```rust
|
||||
// in src/lib.rs
|
||||
|
||||
#[cfg(test)]
|
||||
use bootloader::{entry_point, BootInfo};
|
||||
|
||||
#[cfg(test)]
|
||||
entry_point!(test_kernel_main);
|
||||
|
||||
/// Entry point for `cargo xtest`
|
||||
#[cfg(test)]
|
||||
fn test_kernel_main(_boot_info: &'static BootInfo) -> ! {
|
||||
// like before
|
||||
init();
|
||||
test_main();
|
||||
hlt_loop();
|
||||
}
|
||||
```
|
||||
|
||||
Since the entry point is only used in test mode, we add the `#[cfg(test)]` attribute to all items. We give our test entry point the distinct name `test_kernel_main` to avoid confusion with the `kernel_main` of our `main.rs`. We don't use the `BootInfo` parameter for now, so we prefix the parameter name with a `_` to silence the unused variable warning.
|
||||
|
||||
## Implementation
|
||||
|
||||
Now that we have access to physical memory, we can finally start our implementation. First, we will take a look at the currently active page tables that our kernel runs on. In the second step, we will create a translation function that returns the physical address that a given virtual address is mapped to. As the last step, we will try to modify the page tables in order to create a new mapping.
|
||||
Now that we have access to physical memory, we can finally start to implement our page table code. First, we will take a look at the currently active page tables that our kernel runs on. In the second step, we will create a translation function that returns the physical address that a given virtual address is mapped to. As the last step, we will try to modify the page tables in order to create a new mapping.
|
||||
|
||||
Before we begin, we create a new `memory` module for our code:
|
||||
|
||||
@@ -377,7 +398,7 @@ For the module we create an empty `src/memory.rs` file.
|
||||
|
||||
At the [end of the previous post], we tried to take a look at the page tables our kernel runs on, but failed since we couldn't access the physical frame that the `CR3` register points to. We're now able to continue from there by creating an `active_level_4_table` function that returns a reference to the active level 4 page table:
|
||||
|
||||
[end of the previous post]: ./second-edition/posts/09-paging-introduction/index.md#accessing-the-page-tables
|
||||
[end of the previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
|
||||
|
||||
```rust
|
||||
// in src/memory.rs
|
||||
@@ -414,12 +435,12 @@ We can now use this function to print the entries of the level 4 table:
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…] // initialize GDT, IDT, PICS
|
||||
|
||||
use blog_os::memory::active_level_4_table;
|
||||
|
||||
println!("Hello World{}", "!");
|
||||
blog_os::init();
|
||||
|
||||
let l4_table = unsafe {
|
||||
active_level_4_table(boot_info.physical_memory_offset)
|
||||
};
|
||||
@@ -429,6 +450,10 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
}
|
||||
}
|
||||
|
||||
// as before
|
||||
#[cfg(test)]
|
||||
test_main();
|
||||
|
||||
println!("It did not crash!");
|
||||
blog_os::hlt_loop();
|
||||
}
|
||||
@@ -548,20 +573,19 @@ The `VirtAddr` struct already provides methods to compute the indexes into the p
|
||||
|
||||
Inside the loop, we again use the `physical_memory_offset` to convert the frame into a page table reference. We then read the entry of the current page table and use the [`PageTableEntry::frame`] function to retrieve the mapped frame. If the entry is not mapped to a frame we return `None`. If the entry maps a huge 2MiB or 1GiB page we panic for now.
|
||||
|
||||
[`PageTableEntry::frame`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/page_table/struct.PageTableEntry.html#method.frame
|
||||
[`PageTableEntry::frame`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/page_table/struct.PageTableEntry.html#method.frame
|
||||
|
||||
Let's test our translation function by translating some addresses:
|
||||
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…] // initialize GDT, IDT, PICS
|
||||
|
||||
use blog_os::memory::translate_addr;
|
||||
use x86_64::VirtAddr;
|
||||
|
||||
[…] // hello world and blog_os::init
|
||||
|
||||
let addresses = [
|
||||
// the identity-mapped vga buffer page
|
||||
0xb8000,
|
||||
@@ -581,8 +605,7 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
println!("{:?} -> {:?}", virt, phys);
|
||||
}
|
||||
|
||||
println!("It did not crash!");
|
||||
blog_os::hlt_loop();
|
||||
[…] // test_main(), "it did not crash" printing, and hlt_loop()
|
||||
}
|
||||
```
|
||||
|
||||
@@ -601,17 +624,17 @@ The base of the abstraction are two traits that define various page table mappin
|
||||
- The [`Mapper`] trait is generic over the page size and provides functions that operate on pages. Examples are [`translate_page`], which translates a given page to a frame of the same size, and [`map_to`], which creates a new mapping in the page table.
|
||||
- The [`MapperAllSizes`] trait implies that the implementor implements `Mapper` for all pages sizes. In addition, it provides functions that work with multiple page sizes such as [`translate_addr`] or the general [`translate`].
|
||||
|
||||
[`Mapper`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/mapper/trait.Mapper.html
|
||||
[`translate_page`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.translate_page
|
||||
[`map_to`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to
|
||||
[`MapperAllSizes`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html
|
||||
[`translate_addr`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#method.translate_addr
|
||||
[`translate`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#tymethod.translate
|
||||
[`Mapper`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/mapper/trait.Mapper.html
|
||||
[`translate_page`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.translate_page
|
||||
[`map_to`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/mapper/trait.Mapper.html#tymethod.map_to
|
||||
[`MapperAllSizes`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/mapper/trait.MapperAllSizes.html
|
||||
[`translate_addr`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#method.translate_addr
|
||||
[`translate`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/mapper/trait.MapperAllSizes.html#tymethod.translate
|
||||
|
||||
The traits only define the interface, they don't provide any implementation. The `x86_64` crate currently provides two types that implement the traits: [`MappedPageTable`] and [`RecursivePageTable`]. The former type requires that each page table frame is mapped somewhere (e.g. at an offset). The latter type can be used when the level 4 table is [mapped recursively](#recursive-page-tables).
|
||||
|
||||
[`MappedPageTable`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/struct.MappedPageTable.html
|
||||
[`RecursivePageTable`]: https://docs.rs/x86_64/0.5.1/x86_64/structures/paging/struct.RecursivePageTable.html
|
||||
[`MappedPageTable`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/struct.MappedPageTable.html
|
||||
[`RecursivePageTable`]: https://docs.rs/x86_64/0.5.2/x86_64/structures/paging/struct.RecursivePageTable.html
|
||||
|
||||
We have the complete physical memory mapped at `physical_memory_offset`, so we can use the `MappedPageTable` type. To initialize it, we create a new `init` function in our `memory` module:
|
||||
|
||||
@@ -656,14 +679,13 @@ To use the `MapperAllSizes::translate_addr` method instead of our own `memory::t
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…] // initialize GDT, IDT, PICS
|
||||
|
||||
// new: different imports
|
||||
use blog_os::memory;
|
||||
use x86_64::{structures::paging::MapperAllSizes, VirtAddr};
|
||||
|
||||
[…] // hello world and blog_os::init
|
||||
|
||||
// new: initialize a mapper
|
||||
let mapper = unsafe { memory::init(boot_info.physical_memory_offset) };
|
||||
|
||||
@@ -676,8 +698,7 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
println!("{:?} -> {:?}", virt, phys);
|
||||
}
|
||||
|
||||
println!("It did not crash!");
|
||||
blog_os::hlt_loop();
|
||||
[…] // test_main(), "it did not crash" printing, and hlt_loop()
|
||||
}
|
||||
```
|
||||
|
||||
@@ -732,7 +753,7 @@ In addition to the `page` that should be mapped, the function expects a `mapper`
|
||||
|
||||
For the mapping, we set the `PRESENT` flag because it is required for all valid entries and the `WRITABLE` flag to make the mapped page writable. Calling `map_to` is unsafe because it's possible to break memory safety with invalid arguments, so we need to use an `unsafe` block. For a list of all possible flags, see the [_Page Table Format_] section of the previous post.
|
||||
|
||||
[_Page Table Format_]: ./second-edition/posts/09-paging-introduction/index.md#page-table-format
|
||||
[_Page Table Format_]: ./second-edition/posts/08-paging-introduction/index.md#page-table-format
|
||||
|
||||
The `map_to` function can fail, so it returns a [`Result`]. Since this is just some example code that does not need to be robust, we just use [`expect`] to panic when an error occurs. On success, the function returns a [`MapperFlush`] type that provides an easy way to flush the newly mapped page from the translation lookaside buffer (TLB) with its [`flush`] method. Like `Result`, the type uses the [`#[must_use]`] attribute to emit a warning when we accidentally forget to use it.
|
||||
|
||||
@@ -768,13 +789,12 @@ To test our mapping function, we first map page `0x1000` and then try to write t
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…] // initialize GDT, IDT, PICS
|
||||
|
||||
use blog_os::memory;
|
||||
use x86_64::{structures::paging::Page, VirtAddr};
|
||||
|
||||
[…] // hello world and blog_os::init
|
||||
|
||||
let mut mapper = unsafe { memory::init(boot_info.physical_memory_offset) };
|
||||
let mut frame_allocator = memory::EmptyFrameAllocator;
|
||||
|
||||
@@ -786,8 +806,7 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
let page_ptr: *mut u64 = page.start_address().as_mut_ptr();
|
||||
unsafe { page_ptr.offset(400).write_volatile(0x_f021_f077_f065_f04e)};
|
||||
|
||||
println!("It did not crash!");
|
||||
blog_os::hlt_loop();
|
||||
[…] // test_main(), "it did not crash" printing, and hlt_loop()
|
||||
}
|
||||
```
|
||||
|
||||
@@ -809,7 +828,6 @@ Creating that mapping only worked because there was already a level 1 table for
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…]
|
||||
let page = Page::containing_address(VirtAddr::new(0xdeadbeaf000));
|
||||
@@ -889,7 +907,7 @@ This function uses iterator combinator methods to transform the initial `MemoryM
|
||||
- The third step is the most complicated: We convert each range to an iterator through the `into_iter` method and then choose every 4096th address using [`step_by`]. Since 4096 bytes (= 4 KiB) is the page size, we get the start address of each frame. The bootloader page aligns all usable memory areas so that we don't need any alignment or rounding code here. By using [`flat_map`] instead of `map`, we get an `Iterator<Item = u64>` instead of an `Iterator<Item = Iterator<Item = u64>>`.
|
||||
- In the final step, we convert the start addresses to `PhysFrame` types to construct the desired `Iterator<Item = PhysFrame>`. We then use this iterator to create and return a new `BootInfoFrameAllocator`.
|
||||
|
||||
[`MemoryRegion`]: https://docs.rs/bootloader/0.5.1/bootloader/bootinfo/struct.MemoryRegion.html
|
||||
[`MemoryRegion`]: https://docs.rs/bootloader/0.6.0/bootloader/bootinfo/struct.MemoryRegion.html
|
||||
[`filter`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.filter
|
||||
[`map`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.map
|
||||
[range syntax]: https://doc.rust-lang.org/core/ops/struct.Range.html
|
||||
@@ -901,7 +919,6 @@ We can now modify our `kernel_main` function to pass a `BootInfoFrameAllocator`
|
||||
```rust
|
||||
// in src/main.rs
|
||||
|
||||
#[cfg(not(test))]
|
||||
fn kernel_main(boot_info: &'static BootInfo) -> ! {
|
||||
[…]
|
||||
let mut frame_allocator = memory::init_frame_allocator(&boot_info.memory_map);
|
||||
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 25 KiB |
|
Before Width: | Height: | Size: 23 KiB After Width: | Height: | Size: 23 KiB |
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
|
Before Width: | Height: | Size: 7.9 KiB After Width: | Height: | Size: 7.9 KiB |
|
Before Width: | Height: | Size: 15 KiB After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
|
Before Width: | Height: | Size: 51 KiB After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 51 KiB After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 52 KiB After Width: | Height: | Size: 52 KiB |
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 36 KiB |
@@ -4,6 +4,9 @@ weight = 4
|
||||
path = "unit-testing"
|
||||
date = 2018-04-29
|
||||
|
||||
[extra]
|
||||
warning_short = "Deprecated: "
|
||||
warning = "This post is deprecated in favor of the [_Testing_](/testing) post and will no longer receive updates."
|
||||
+++
|
||||
|
||||
This post explores unit testing in `no_std` executables using Rust's built-in test framework. We will adjust our code so that `cargo test` works and add some basic unit tests to our VGA buffer module.
|
||||
@@ -18,6 +21,16 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
## Requirements
|
||||
|
||||
In this post we explore how to execute `cargo test` on the host system (as a normal Linux/Windows/macOS executable). This only works if you don't have a `.cargo/config` file that sets a default target. If you followed the [_Minimal Rust Kernel_] post before 2019-04-27, you should be fine. If you followed it after that date, you need to remove the `build.target` key from your `.cargo/config` file and explicitly pass a target argument to `cargo xbuild`.
|
||||
|
||||
[_Minimal Rust Kernel_]: ./second-edition/posts/02-minimal-rust-kernel/index.md
|
||||
|
||||
Alternatively, consider reading the new [_Testing_] post instead. It sets up a similar functionality as this post, but instead of running the tests on your host system, they are run in a realistic environment inside QEMU.
|
||||
|
||||
[_Testing_]: ./second-edition/posts/04-testing/index.md
|
||||
|
||||
## Unit Tests for `no_std` Binaries
|
||||
Rust has a [built-in test framework] that is capable of running unit tests without the need to set anything up. Just create a function that checks some results through assertions and add the `#[test]` attribute to the function header. Then `cargo test` will automatically find and execute all test functions of your crate.
|
||||
|
||||
@@ -4,6 +4,9 @@ weight = 5
|
||||
path = "integration-tests"
|
||||
date = 2018-06-15
|
||||
|
||||
[extra]
|
||||
warning_short = "Deprecated: "
|
||||
warning = "This post is deprecated in favor of the [_Testing_](/testing) post and will no longer receive updates."
|
||||
+++
|
||||
|
||||
To complete the testing picture we implement a basic integration test framework, which allows us to run tests on the target system. The idea is to run tests inside QEMU and report the results back to the host through the serial port.
|
||||
@@ -18,6 +21,13 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
|
||||
|
||||
<!-- toc -->
|
||||
|
||||
## Requirements
|
||||
|
||||
This post builds upon the [_Unit Testing_] post, so you need to follow it first. Alternatively, consider reading the new [_Testing_] post instead, which replaces both _Unit Testing_ and this post. The new posts implements similar functionality, but integrates it directly in `cargo xtest`, so that both unit and integration tests run in a realistic environment inside QEMU.
|
||||
|
||||
[_Unit Testing_]: ./second-edition/posts/deprecated/04-unit-testing/index.md
|
||||
[_Testing_]: ./second-edition/posts/04-testing/index.md
|
||||
|
||||
## Overview
|
||||
|
||||
In the previous post we added support for unit tests. The goal of unit tests is to test small components in isolation to ensure that each of them works as intended. The tests are run on the host machine and thus shouldn't rely on architecture specific functionality.
|
||||
@@ -23,11 +23,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions
|
||||
|
||||
In the [previous post] we learned about the principles of paging and how the 4-level page tables on x86_64 work. We also found out that the bootloader already set up a page table hierarchy for our kernel, which means that our kernel already runs on virtual addresses. This improves safety since illegal memory accesses cause page fault exceptions instead of modifying arbitrary physical memory.
|
||||
|
||||
[previous post]: ./second-edition/posts/09-paging-introduction/index.md
|
||||
[previous post]: ./second-edition/posts/08-paging-introduction/index.md
|
||||
|
||||
However, it also causes a problem when we try to access the page tables from our kernel because we can't directly access the physical addresses that are stored in page table entries or the `CR3` register. We experienced that problem already [at the end of the previous post] when we tried to inspect the active page tables.
|
||||
|
||||
[at the end of the previous post]: ./second-edition/posts/09-paging-introduction/index.md#accessing-the-page-tables
|
||||
[at the end of the previous post]: ./second-edition/posts/08-paging-introduction/index.md#accessing-the-page-tables
|
||||
|
||||
The next section discusses the problem in detail and provides different approaches to a solution. Afterward, we implement a function that traverses the page table hierarchy in order to translate virtual to physical addresses. Finally, we learn how to create new mappings in the page tables and how to find unused memory frames for creating new page tables.
|
||||
|
||||
@@ -63,7 +63,7 @@ So in order to access page table frames, we need to map some virtual pages to th
|
||||
However, it clutters the virtual address space and makes it more difficult to find continuous memory regions of larger sizes. For example, imagine that we want to create a virtual memory region of size 1000 KiB in the above graphic, e.g. for [memory-mapping a file]. We can't start the region at `28 KiB` because it would collide with the already mapped page at `1004 MiB`. So we have to look further until we find a large enough unmapped area, for example at `1008 KiB`. This is a similar fragmentation problem as with [segmentation].
|
||||
|
||||
[memory-mapping a file]: https://en.wikipedia.org/wiki/Memory-mapped_file
|
||||
[segmentation]: ./second-edition/posts/09-paging-introduction/index.md#fragmentation
|
||||
[segmentation]: ./second-edition/posts/08-paging-introduction/index.md#fragmentation
|
||||
|
||||
Equally, it makes it much more difficult to create new page tables, because we need to find physical frames whose corresponding pages aren't already in use. For example, let's assume that we reserved the _virtual_ 1000 KiB memory region starting at `1008 KiB` for our memory-mapped file. Now we can't use any frame with a _physical_ address between `1000 KiB` and `2008 KiB` anymore, because we can't identity map it.
|
||||
|
||||
@@ -156,7 +156,7 @@ Whereas `AAA` is the level 4 index, `BBB` the level 3 index, `CCC` the level 2 i
|
||||
|
||||
`SSSSSS` are sign extension bits, which means that they are all copies of bit 47. This is a special requirement for valid addresses on the x86_64 architecture. We explained it in the [previous post][sign extension].
|
||||
|
||||
[sign extension]: ./second-edition/posts/09-paging-introduction/index.md#paging-on-x86
|
||||
[sign extension]: ./second-edition/posts/08-paging-introduction/index.md#paging-on-x86
|
||||
|
||||
We use [octal] numbers for representing the addresses since each octal character represents three bits, which allows us to clearly separate the 9-bit indexes of the different page table levels. This isn't possible with the hexadecimal system where each character represents four bits.
|
||||
|
||||
|
||||
@@ -67,10 +67,6 @@ main img {
|
||||
border: 2px solid #66f;
|
||||
}
|
||||
|
||||
.posts.testing {
|
||||
border: 2px solid #0a0;
|
||||
}
|
||||
|
||||
.posts.memory-management {
|
||||
border: 2px solid #fc0
|
||||
}
|
||||
@@ -106,10 +102,6 @@ main img {
|
||||
color: #55d;
|
||||
}
|
||||
|
||||
.post-category.testing {
|
||||
color: #090;
|
||||
}
|
||||
|
||||
.post-category.memory-management {
|
||||
color: #990;
|
||||
}
|
||||
|
||||
@@ -26,25 +26,20 @@
|
||||
{{ macros::post_link(page=posts.0) }}
|
||||
{{ macros::post_link(page=posts.1) }}
|
||||
{{ macros::post_link(page=posts.2) }}
|
||||
</div>
|
||||
|
||||
<div id="testing" class="post-category testing">Testing</div>
|
||||
<div class="posts testing">
|
||||
{{ macros::post_link(page=posts.3) }}
|
||||
{{ macros::post_link(page=posts.4) }}
|
||||
</div>
|
||||
|
||||
<div id="exceptions" class="post-category exceptions">Exceptions</div>
|
||||
<div class="posts exceptions">
|
||||
{{ macros::post_link(page=posts.4) }}
|
||||
{{ macros::post_link(page=posts.5) }}
|
||||
{{ macros::post_link(page=posts.6) }}
|
||||
{{ macros::post_link(page=posts.7) }}
|
||||
</div>
|
||||
|
||||
<div id="memory-management" class="post-category memory-management">Memory Management</div>
|
||||
<div class="posts memory-management">
|
||||
{{ macros::post_link(page=posts.7) }}
|
||||
{{ macros::post_link(page=posts.8) }}
|
||||
{{ macros::post_link(page=posts.9) }}
|
||||
</div>
|
||||
|
||||
<div class="posts subscribe">
|
||||
|
||||
Reference in New Issue
Block a user