From 005dd7d95119153be74cf1d701452d0129ebd302 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 17 Apr 2019 11:56:26 +0200 Subject: [PATCH 01/42] Begin new testing post --- .../second-edition/posts/04-testing/index.md | 1240 +++++++++++++++++ 1 file changed, 1240 insertions(+) create mode 100644 blog/content/second-edition/posts/04-testing/index.md diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md new file mode 100644 index 00000000..c6d646e9 --- /dev/null +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -0,0 +1,1240 @@ ++++ +title = "Testing" +weight = 4 +path = "testing" +date = 0000-01-01 + ++++ + +This post explores unit and integration testing in `no_std` executables. We will use Rust's support for custom test frameworks to execute test functions inside our kernel. To report the results out of QEMU, we will use different features of QEMU and the `bootimage` tool. + + + +This blog is openly developed on [GitHub]. If you have any problems or questions, please open an issue there. You can also leave comments [at the bottom]. The complete source code for this post can be found in the [`post-04`][post branch] branch. + +[GitHub]: https://github.com/phil-opp/blog_os +[at the bottom]: #comments +[post branch]: https://github.com/phil-opp/blog_os/tree/post-04 + + + +## Requirements + +This post assumes that you have a `.cargo/config` file with a default target... TODO + +Earlier posts since XX, bootimage runner + +## Testing in Rust + +Rust has a [built-in test framework] that is capable of running unit tests without the need to set anything up. Just create a function that checks some results through assertions and add the `#[test]` attribute to the function header. Then `cargo test` will automatically find and execute all test functions of your crate. + +[built-in test framework]: https://doc.rust-lang.org/book/second-edition/ch11-00-testing.html + +Unfortunately it's a bit more complicated for `no_std` applications such as our kernel. The problem is that Rust's test framework implicitly uses the built-in [`test`] library, which depends on the standard library. This means that we can't use the default test framework for our `#[no_std]` kernel. + +[`test`]: https://doc.rust-lang.org/test/index.html + +We can see this when we try to run `cargo xtest` in our project: + +``` +> cargo xtest + Compiling blog_os v0.1.0 (/home/philipp/Documents/blog_os/code) +error[E0463]: can't find crate for `test` +``` + +Since the `test` crate depends on the standard library, it is not available for our bare metal target. While porting the `test` crate to a `#[no_std]` context [is possible][utest], it is highly unstable and requires some hacks such as redefining the `panic` macro. + +[utest]: https://github.com/japaric/utest + +### Custom Test Frameworks + +Fortunately, Rust supports replacing the default test framework through the unstable [`custom_test_frameworks`] feature. This feature requires no external libraries and thus also works in `#[no_std]` environments. It works by collecting all functions annotated with a `#[test_case]` attribute and then invoking a specified runner function with the list of tests as argument. Thus it gives the implementation maximal control over the test process. + +[`custom_test_frameworks`]: https://doc.rust-lang.org/unstable-book/language-features/custom-test-frameworks.html + +The disadvantage compared to the default test framework is that many advanced features such as [`should_panic` tests] are not available. Instead, it is up to the implementation to provide such features itself if needed. This is ideal for us since we have a very special execution environment where the default implementations of such advanced features probably wouldn't work anyway. For example, the `#[should_panic]` attribute relies on stack unwinding to catch the panics, which we disabled for our kernel. + +[`should_panic` tests]: https://doc.rust-lang.org/book/ch11-01-writing-tests.html#checking-for-panics-with-should_panic + +To implement a custom test framework for our kernel, we add the following to our `main.rs`: + +```rust +// in src/main.rs + +#![feature(custom_test_frameworks)] +#![test_runner(crate::test_runner)] + +fn test_runner(tests: &[&dyn Fn()]) { + println!("Running {} tests", tests.len()); + for test in tests { + test(); + } +} +``` + +Our runner just prints a short debug message and then calls each test function in the list. The argument type `&[&dyn Fn()]` is a [_slice_] of [_trait object_] references of the [_Fn()_] trait. It is basically a list of references to types that can be called like a function. + +[_slice_]: https://doc.rust-lang.org/std/primitive.slice.html +[_trait object_]: https://doc.rust-lang.org/1.30.0/book/first-edition/trait-objects.html +[_Fn()_]: https://doc.rust-lang.org/std/ops/trait.Fn.html + +When we run `cargo xtest` now, we see that it now succeeds. However, we still see our "Hello World" instead of the message from our `test_runner`: + +TODO image + +The reason is that our `_start` function is still used as entry point. The custom test frameworks feature generates a `main` function that calls `test_runner`, but this function is ignored because we use the `#[no_main]` attribute and provide our own entry point. + +To fix this, we first need to change the name of the generated function to something different than `main` through the `reexport_test_harness_main` attribute. Then we can call the renamed function from our `_start` function: + +```rust +// in src/main.rs + +#![reexport_test_harness_main = "test_main"] + +#[no_mangle] +pub extern "C" fn _start() -> ! { + println!("Hello World{}", "!"); + + #[cfg(test)] + test_main(); + + loop {} +} +``` + +We set the name of the test framework entry function to `test_main` and call it from our `_start` entry point. We use [conditional compilation] to add the call to `test_main` only in test contexts because the function is not generated on a normal run. + +When we now execute `cargo xtest`, we see the message from our `test_runner` on the screen: + +TODO image + +We are now ready to create our first test function: + +```rust +// in src/main.rs + +#[test_case] +fn trivial_assertion() { + print!("trivial assertion... "); + assert_eq!(1, 1); + println("[ok]"); +} +``` + +Of course the test succeeds and we see the `trivial assertion... [ok]` output on the screen. The problem is that QEMU never exits so that `cargo xtest` runs forever. + +## Exiting QEMU + +Right now we have an endless loop at the end of our `_start` function and need to close QEMU manually. The clean solution to this would be to implement a proper way to shutdown our OS. Unfortunatly this is relatively complex, because it requires implementing support for either the [APM] or [ACPI] power management standard. + +[APM]: https://wiki.osdev.org/APM +[ACPI]: https://wiki.osdev.org/ACPI + +Luckily, there is an escape hatch: QEMU supports a special `isa-debug-exit` device, which provides an easy way to exit QEMU from the guest system. To enable it, we need to pass a `-device` argument to QEMU. We can do so by adding a `package.metadata.bootimage.test-args` configuration key in our `Cargo.toml`: + +```toml +# in Cargo.toml + +[package.metadata.bootimage] +test-args = ["-device", "isa-debug-exit,iobase=0xf4,iosize=0x04"] +``` + +The `bootimage runner` appends the `test-args` to the default QEMU command for all test executables. For a normal `cargo xrun`, the arguments are ignored. + +Together with the device name (`isa-debug-exit`), we pass the two parameters `iobase` and `iozize` that specify the _I/O port_ through which the device can be reached from our kernel. + +### I/O Ports + +There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the VGA device. + +[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md + +In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`). + +The `isa-debug-exit` devices uses port-mapped I/O. The `iobase` parameter specifies on which port address the device should live (`0xf4` is a [generally unused][list of x86 I/O ports] port on the x86's IO bus) and the `iosize` specifies the port size (`0x04` means four bytes). + +[list of x86 I/O ports]: https://wiki.osdev.org/I/O_Ports#The_list + +### Using the Exit Device + +The functionality of the `isa-debug-exit` device is very simple. When a `value` is written to the I/O port specified by `iobase`, it causes QEMU to exit with [exit status] `(value << 1) | 1`. So when we write `0` to the port QEMU will exit with exit status `(0 << 1) | 1 = 1` and when we write `1` to the port it will exit with exit status `(1 << 1) | 1 = 3`. + +[exit status]: https://en.wikipedia.org/wiki/Exit_status + +Instead of manually invoking the `in` and `out` assembly instructions, we use the abstractions provided by the [`x86_64`] crate. To add a dependency on that crate, we add it to the `dependencies` section in our `Cargo.toml`: + +[`x86_64`]: https://docs.rs/x86_64/0.5.2/x86_64/ + +```toml +# in Cargo.toml + +[dependencies] +x86_64 = "0.5.2" +``` + +Now we can use the [`Port`] type provided by the crate to create an `exit_qemu` function: + +[`Port`]: https://docs.rs/x86_64/0.5.2/x86_64/instructions/port/struct.Port.html + +```rust +// in src/main.rs + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[repr(u32)] +pub enum QemuExitCode { + Success = 0x10, + Failed = 0x11, +} + +pub unsafe fn exit_qemu(exit_code: QemuExitCode) { + use x86_64::instructions::port::Port; + + let mut port = Port::new(0xf4); + port.write(exit_code as u32); +} +``` + +We mark the function as `unsafe` because it relies on the fact that a special QEMU device is attached to the I/O port with address `0xf4`. The function creates a new [`Port`] at `0xf4`, which is the `iobase` of the `isa-debug-exit` device. Then it writes the the passed exit code to the port. We use `u32` because we specified the `iosize` of the `isa-debug-exit` device as 4 bytes. + +For specifying the exit status, we create a `QemuExitCode` enum. The idea is to exit with the success exit code if all tests succeeded and with the failure exit code otherwise. The enum is marked as `#[repr(u32)]` to represent each variant by an `u32` integer. We use exit code `0x10` for success and `0x11` for failure. The actual exit codes do not matter much, as long as they don't clash with the default exit codes of QEMU. For example, using exit code `0` for success is not a good idea because it becomes `(0 << 1) | 1 = 1` after the transformation, which is the default exit code when QEMU failed to run. So we could not differentiate a QEMU error from a successfull test run. + +We can now update our `test_runner` to exit QEMU after all tests ran: + +```rust +fn test_runner(tests: &[&dyn Fn()]) { + println!("Running {} tests", tests.len()); + for test in tests { + test(); + } + /// new + unsafe { exit_qemu(QemuExitCode::Success) }; +} +``` + +When we run `cargo xtest` now, we see that QEMU immediately closes after executing the tests. The problem is that `cargo test` interprets the test as failed even though we passed our `Success` exit code: + +``` +> cargo xtest + Finished dev [unoptimized + debuginfo] target(s) in 0.03s + Running target/x86_64-blog_os/debug/deps/blog_os-5804fc7d2dd4c9be +Building bootloader + Compiling bootloader v0.5.3 (/home/philipp/Documents/bootloader) + Finished release [optimized + debuginfo] target(s) in 1.07s +Running: `qemu-system-x86_64 -drive format=raw,file=target/x86_64-blog_os/debug/ + deps/bootimage-blog_os-5804fc7d2dd4c9be.bin -device isa-debug-exit,iobase=0xf4, + iosize=0x04` +error: test failed +``` + +The problem is that `cargo test` considers all error codes other than `0` as failure. + +### Success Exit Code + +To work around this, `bootimage` provides a `test-success-exit-code` configuration key that maps a specified exit code to the exit code `0`: + +```toml +[package.metadata.bootimage] +test-args = […] +test-success-exit-code = 33 # (0x10 << 1) | 1 +``` + +With this configuration, `bootimage` maps our success exit code to exit code 0, so that `cargo xtest` correctly recognizes the success case and does no count the test as failed. + +Our test runner now automatically closes QEMU and correctly reports the test results out. We still see the QEMU window open for a very short time, but it does not suffice to read the results. It would be nice if we could print the test results to the console instead, so that we can still see them after QEMU exited. + +## Printing to the Console + + + +### Serial Port + +A simple way to achieve this is by using the [serial port], an old interface standard which is no longer found in modern computers. It is easy to program and QEMU can redirect the bytes sent over serial to the host's standard output or a file. + +[serial port]: https://en.wikipedia.org/wiki/Serial_port + +The chips implementing a serial interface are called [UARTs]. There are [lots of UART models] on x86, but fortunately the only differences between them are some advanced features we don't need. The common UARTs today are all compatible to the [16550 UART], so we will use that model for our testing framework. + +[UARTs]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter +[lots of UART models]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter#UART_models +[16550 UART]: https://en.wikipedia.org/wiki/16550_UART + +We will use the [`uart_16550`] crate to initialize the UART and send data over the serial port. To add it as a dependency, we update our `Cargo.toml` and `main.rs`: + +[`uart_16550`]: https://docs.rs/uart_16550 + +```toml +# in Cargo.toml + +[dependencies] +uart_16550 = "0.1.0" +``` + +The `uart_16550` crate contains a `SerialPort` struct that represents the UART registers, but we still need to construct an instance of it ourselves. For that we create a new `serial` module with the following content: + +```rust +// in src/main.rs + +mod serial; +``` + +```rust +// in src/serial.rs + +use uart_16550::SerialPort; +use spin::Mutex; +use lazy_static::lazy_static; + +lazy_static! { + pub static ref SERIAL1: Mutex = { + let mut serial_port = unsafe { SerialPort::new(0x3F8) }; + serial_port.init(); + Mutex::new(serial_port) + }; +} +``` + +Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static`. However, this time we use `lazy_static` to ensure that the `init` method is called before first use. We're using the port address `0x3F8`, which is the standard port number for the first serial interface. + +[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics + +To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros: + +```rust +#[doc(hidden)] +pub fn _print(args: ::core::fmt::Arguments) { + use core::fmt::Write; + SERIAL1.lock().write_fmt(args).expect("Printing to serial failed"); +} + +/// Prints to the host through the serial interface. +#[macro_export] +macro_rules! serial_print { + ($($arg:tt)*) => { + $crate::serial::_print(format_args!($($arg)*)); + }; +} + +/// Prints to the host through the serial interface, appending a newline. +#[macro_export] +macro_rules! serial_println { + () => ($crate::serial_print!("\n")); + ($fmt:expr) => ($crate::serial_print!(concat!($fmt, "\n"))); + ($fmt:expr, $($arg:tt)*) => ($crate::serial_print!( + concat!($fmt, "\n"), $($arg)*)); +} +``` + +The `SerialPort` type already implements the [`fmt::Write`] trait, so we don't need to provide an implementation. + +[`fmt::Write`]: https://doc.rust-lang.org/nightly/core/fmt/trait.Write.html + +Now we can print to the serial interface instead of the VGA text buffer in our test code: + +```rust +// in src/main.rs + +```rust +fn test_runner(tests: &[&dyn Fn()]) { + serial_println!("Running {} tests", tests.len()); + […] +} + +#[test_case] +fn trivial_assertion() { + serial_print!("trivial assertion... "); + assert_eq!(1, 1); + serial_println("[ok]"); +} +``` + +Note that the `serial_println` macro lives directly under the root namespace because we used the `#[macro_export]` attribute, so importing it through `use crate::serial::serial_println` will not work. + +### QEMU Arguments + +To see the serial output in QEMU, we need use the `-serial` argument to redirect the output to stdout: + +```toml +# in Cargo.toml + +[package.metadata.bootimage] +test-args = [ + "-device", "isa-debug-exit,iobase=0xf4,iosize=0x04", "-serial", "mon:stdio" +] +``` + +When we run `cargo xtest` now, we see the test output directly in the console: + +``` +> cargo xtest +TODO +``` + +However, when a test fails we still see the output inside QEMU because our panic handler still uses `println`. To simulate this, we can change the assertion in our `trivial_assertion` test to `assert_eq!(0, 1)`: + +TODO image + +Note that it's no longer possible to exit QEMU from the console through `Ctrl+c` when `serial mon:stdio` is passed. An alternative keyboard shortcut is `Ctrl+a` and then `x`. Or you can just close the QEMU window manually. + +### Exit QEMU on Panic + +To also exit QEMU on a panic, we can use [conditional compilation] to use a different panic handler in testing mode: + +[conditional compilation]: https://doc.rust-lang.org/1.30.0/book/first-edition/conditional-compilation.html + +```rust +// our existing panic handler +#[cfg(not(test))] // new attribute +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + println!("{}", info); + loop {} +} + +// our panic handler in test mode +#[cfg(test)] +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + serial_println!("[failed]\n"); + serial_println!("Error: {}\n", info); + unsafe { exit_qemu(QemuExitCode::Failed); } + loop {} +} +``` + +For our test panic handler, we use `serial_println` instead of `println` and then exit QEMU with a failure exit code. Note that we still need an endless `loop` after the `exit_qemu` call because the compiler does not know that the `isa-debug-exit` device causes a program exit. + +Now QEMU also exits for failed tests and prints a useful error message on the console: + +``` +> cargo xtest +TODO +``` + +We still see the QEMU window open for a short time, which we don't need anymore. + +### Hiding QEMU + +Since we report out the complete test results using the `isa-debug-exit` device and the serial port, we don't need the QEMU window anymore. We can hide it by passing the `-display none` argument to QEMU: + +```toml +# in Cargo.toml + +[package.metadata.bootimage] +test-args = [ + "-device", "isa-debug-exit,iobase=0xf4,iosize=0x04", "-serial", "mon:stdio", + "-display", "none" +] +``` + +Now QEMU runs completely in the background and no window is opened anymore. This is not only less annoying, but also allows our test framework to run in environments without a graphical user interface, such as CI services or [SSH] connections. + +[SSH]: https://en.wikipedia.org/wiki/Secure_Shell + +## Testing the VGA Buffer + +Now that we have a working test framework, we can create a few tests for our VGA buffer implementation. First, we create a very simple test that ensures that `println` works without panicking: + +```rust +#[test_case] +fn test_println() { + serial_print!("test_println... "); + println!("Test Output"); + serial_println("[ok]"); +} +``` + + + +## Integration Tests + +Our kernel has more features in the future -> ensure that printing works from the beginning on by testing println in minimal environment + +## Split off a Library + +## No Harness + +should panic test + + +# Unit Tests + +## Testing the VGA Module +Now that we have set up the test framework, we can add a first unit test for our `vga_buffer` module: + +```rust +// in src/vga_buffer.rs + +#[cfg(test)] +mod test { + use super::*; + + #[test] + fn foo() {} +} +``` + +We add the test in an inline `test` submodule. This isn't necessary, but a common way to separate test code from the rest of the module. By adding the `#[cfg(test)]` attribute, we ensure that the module is only compiled in test mode. Through `use super::*`, we import all items of the parent module (the `vga_buffer` module), so that we can test them easily. + +The `#[test]` attribute on the `foo` function tells the test framework that the function is an unit test. The framework will find it automatically, even if it's private and inside a private module as in our case: + +``` +> cargo test + Compiling blog_os v0.2.0 (file:///…/blog_os) + Finished dev [unoptimized + debuginfo] target(s) in 2.99 secs + Running target/debug/deps/blog_os-1f08396a9eff0aa7 + +running 1 test +test vga_buffer::test::foo ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out +``` + +We see that the test was found and executed. It didn't panic, so it counts as passed. + +### Constructing a Writer +In order to test the VGA methods, we first need to construct a `Writer` instance. Since we will need such an instance for other tests too, we create a separate function for it: + +```rust +// in src/vga_buffer.rs + +#[cfg(test)] +mod test { + use super::*; + + fn construct_writer() -> Writer { + use std::boxed::Box; + + let buffer = construct_buffer(); + Writer { + column_position: 0, + color_code: ColorCode::new(Color::Blue, Color::Magenta), + buffer: Box::leak(Box::new(buffer)), + } + } + + fn construct_buffer() -> Buffer { … } +} +``` + +We set the initial column position to 0 and choose some arbitrary colors for foreground and background color. The difficult part is the buffer construction, it's described in detail below. We then use [`Box::new`] and [`Box::leak`] to transform the created `Buffer` into a `&'static mut Buffer`, because the `buffer` field needs to be of that type. + +[`Box::new`]: https://doc.rust-lang.org/nightly/std/boxed/struct.Box.html#method.new +[`Box::leak`]: https://doc.rust-lang.org/nightly/std/boxed/struct.Box.html#method.leak + +#### Buffer Construction +So how do we create a `Buffer` instance? The naive approach does not work unfortunately: + +```rust +fn construct_buffer() -> Buffer { + Buffer { + chars: [[Volatile::new(empty_char()); BUFFER_WIDTH]; BUFFER_HEIGHT], + } +} + +fn empty_char() -> ScreenChar { + ScreenChar { + ascii_character: b' ', + color_code: ColorCode::new(Color::Green, Color::Brown), + } +} +``` + +When running `cargo test` the following error occurs: + +``` +error[E0277]: the trait bound `volatile::Volatile: core::marker::Copy` is not satisfied + --> src/vga_buffer.rs:186:21 + | +186 | chars: [[Volatile::new(empty_char); BUFFER_WIDTH]; BUFFER_HEIGHT], + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `core::marker::Copy` is not implemented for `volatile::Volatile` + | + = note: the `Copy` trait is required because the repeated element will be copied +``` + +The problem is that array construction in Rust requires that the contained type is [`Copy`]. The `ScreenChar` is `Copy`, but the `Volatile` wrapper is not. There is currently no easy way to circumvent this without using [`unsafe`], but fortunately there is the [`array_init`] crate that provides a safe interface for such operations. + +[`Copy`]: https://doc.rust-lang.org/core/marker/trait.Copy.html +[`unsafe`]: https://doc.rust-lang.org/book/second-edition/ch19-01-unsafe-rust.html +[`array_init`]: https://docs.rs/array-init + +To use that crate, we add the following to our `Cargo.toml`: + +```toml +[dev-dependencies] +array-init = "0.0.3" +``` + +Note that we're using the [`dev-dependencies`] table instead of the `dependencies` table, because we only need the crate for `cargo test` and not for a normal build. + +[`dev-dependencies`]: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#development-dependencies + +Now we can fix our `construct_buffer` function: + +```rust +fn construct_buffer() -> Buffer { + use array_init::array_init; + + Buffer { + chars: array_init(|_| array_init(|_| Volatile::new(empty_char()))), + } +} +``` + +See the [documentation of `array_init`][`array_init`] for more information about using that crate. + +### Testing `write_byte` +Now we're finally able to write a first unit test that tests the `write_byte` method: + +```rust +// in vga_buffer.rs + +mod test { + […] + + #[test] + fn write_byte() { + let mut writer = construct_writer(); + writer.write_byte(b'X'); + writer.write_byte(b'Y'); + + for (i, row) in writer.buffer.chars.iter().enumerate() { + for (j, screen_char) in row.iter().enumerate() { + let screen_char = screen_char.read(); + if i == BUFFER_HEIGHT - 1 && j == 0 { + assert_eq!(screen_char.ascii_character, b'X'); + assert_eq!(screen_char.color_code, writer.color_code); + } else if i == BUFFER_HEIGHT - 1 && j == 1 { + assert_eq!(screen_char.ascii_character, b'Y'); + assert_eq!(screen_char.color_code, writer.color_code); + } else { + assert_eq!(screen_char, empty_char()); + } + } + } + } +} +``` + +We construct a `Writer`, write two bytes to it, and then check that the right screen characters were updated. When we run `cargo test`, we see that the test is executed and passes: + +``` +running 1 test +test vga_buffer::test::write_byte ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out +``` + +Try to play around a bit with this function and verify that the test fails if you change something, e.g. if you print a third byte without adjusting the `for` loop. + +(If you're getting an “binary operation `==` cannot be applied to type `vga_buffer::ScreenChar`” error, you need to also derive [`PartialEq`] for `ScreenChar` and `ColorCode`). + +[`PartialEq`]: https://doc.rust-lang.org/nightly/core/cmp/trait.PartialEq.html + +### Testing Strings +Let's add a second unit test to test formatted output and newline behavior: + +```rust +// in src/vga_buffer.rs + +mod test { + […] + + #[test] + fn write_formatted() { + use core::fmt::Write; + + let mut writer = construct_writer(); + writeln!(&mut writer, "a").unwrap(); + writeln!(&mut writer, "b{}", "c").unwrap(); + + for (i, row) in writer.buffer.chars.iter().enumerate() { + for (j, screen_char) in row.iter().enumerate() { + let screen_char = screen_char.read(); + if i == BUFFER_HEIGHT - 3 && j == 0 { + assert_eq!(screen_char.ascii_character, b'a'); + assert_eq!(screen_char.color_code, writer.color_code); + } else if i == BUFFER_HEIGHT - 2 && j == 0 { + assert_eq!(screen_char.ascii_character, b'b'); + assert_eq!(screen_char.color_code, writer.color_code); + } else if i == BUFFER_HEIGHT - 2 && j == 1 { + assert_eq!(screen_char.ascii_character, b'c'); + assert_eq!(screen_char.color_code, writer.color_code); + } else if i >= BUFFER_HEIGHT - 2 { + assert_eq!(screen_char.ascii_character, b' '); + assert_eq!(screen_char.color_code, writer.color_code); + } else { + assert_eq!(screen_char, empty_char()); + } + } + } + } +} +``` + +In this test we're using the [`writeln!`] macro to print strings with newlines to the buffer. Most of the for loop is similar to the `write_byte` test and only verifies if the written characters are at the expected place. The new `if i >= BUFFER_HEIGHT - 2` case verifies that the empty lines that are shifted in on a newline have the `writer.color_code`, which is different from the initial color. + +[`writeln!`]: https://doc.rust-lang.org/nightly/core/macro.writeln.html + +### More Tests +We only present two basic tests here as an example, but of course many more tests are possible. For example a test that changes the writer color in between writes. Or a test that checks that the top line is correctly shifted off the screen on a newline. Or a test that checks that non-ASCII characters are handled correctly. + +## Summary +Unit testing is a very useful technique to ensure that certain components have a desired behavior. Even if they cannot show the absence of bugs, they're still an useful tool for finding them and especially for avoiding regressions. + +This post explained how to set up unit testing in a Rust kernel. We now have a functioning test framework and can easily add tests by adding functions with a `#[test]` attribute. To run them, a short `cargo test` suffices. We also added a few basic tests for our VGA buffer as an example how unit tests could look like. + +We also learned a bit about conditional compilation, Rust's [lint system], how to [initialize arrays with non-Copy types], and the `dev-dependencies` section of the `Cargo.toml`. + +[lint system]: #silencing-the-warnings +[initialize arrays with non-Copy types]: #buffer-construction + +## What's next? +We now have a working unit testing framework, which gives us the ability to test individual components. However, unit tests have the disadvantage that they run on the host machine and are thus unable to test how components interact with platform specific parts. For example, we can't test the `println!` macro with an unit test because it wants to write at the VGA text buffer at address `0xb8000`, which only exists in the bare metal environment. + +The next post will close this gap by creating a basic _integration test_ framework, which runs the tests in QEMU and thus has access to platform specific components. This will allow us to test the full system, for example that our kernel boots correctly or that no deadlock occurs on nested `println!` invocations. + + + + +# Integration Tests + +## Overview + +In the previous post we added support for unit tests. The goal of unit tests is to test small components in isolation to ensure that each of them works as intended. The tests are run on the host machine and thus shouldn't rely on architecture specific functionality. + +To test the interaction of the components, both with each other and the system environment, we can write _integration tests_. Compared to unit tests, ìntegration tests are more complex, because they need to run in a realistic environment. What this means depends on the application type. For example, for webserver applications it often means to set up a database instance. For an operating system kernel like ours, it means that we run the tests on the target hardware without an underlying operating system. + +Running on the target architecture allows us to test all hardware specific code such as the VGA buffer or the effects of [page table] modifications. It also allows us to verify that our kernel boots without problems and that no [CPU exception] occurs. + +[page table]: https://en.wikipedia.org/wiki/Page_table +[CPU exception]: https://wiki.osdev.org/Exceptions + +In this post we will implement a very basic test framework that runs integration tests inside instances of the [QEMU] virtual machine. It is not as realistic as running them on real hardware, but it is much simpler and should be sufficient as long as we only use standard hardware that is well supported in QEMU. + +[QEMU]: https://www.qemu.org/ + +## The Serial Port + +The naive way of doing an integration test would be to add some assertions in the code, launch QEMU, and manually check if a panic occured or not. This is very cumbersome and not practical if we have hundreds of integration tests. So we want an automated solution that runs all tests and fails if not all of them pass. + +Such an automated test framework needs to know whether a test succeeded or failed. It can't look at the screen output of QEMU, so we need a different way of retrieving the test results on the host system. A simple way to achieve this is by using the [serial port], an old interface standard which is no longer found in modern computers. It is easy to program and QEMU can redirect the bytes sent over serial to the host's standard output or a file. + +[serial port]: https://en.wikipedia.org/wiki/Serial_port + +The chips implementing a serial interface are called [UARTs]. There are [lots of UART models] on x86, but fortunately the only differences between them are some advanced features we don't need. The common UARTs today are all compatible to the [16550 UART], so we will use that model for our testing framework. + +[UARTs]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter +[lots of UART models]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter#UART_models +[16550 UART]: https://en.wikipedia.org/wiki/16550_UART + +### Port I/O +There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the GPU. + +[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md + +In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`). + +The UART uses port-mapped I/O. Fortunately there are already several crates that provide abstractions for I/O ports and even UARTs, so we don't need to invoke the `in` and `out` assembly instructions manually. + +### Implementation + +We will use the [`uart_16550`] crate to initialize the UART and send data over the serial port. To add it as a dependency, we update our `Cargo.toml` and `main.rs`: + +[`uart_16550`]: https://docs.rs/uart_16550 + +```toml +# in Cargo.toml + +[dependencies] +uart_16550 = "0.1.0" +``` + +The `uart_16550` crate contains a `SerialPort` struct that represents the UART registers, but we still need to construct an instance of it ourselves. For that we create a new `serial` module with the following content: + +```rust +// in src/main.rs + +mod serial; +``` + +```rust +// in src/serial.rs + +use uart_16550::SerialPort; +use spin::Mutex; +use lazy_static::lazy_static; + +lazy_static! { + pub static ref SERIAL1: Mutex = { + let mut serial_port = SerialPort::new(0x3F8); + serial_port.init(); + Mutex::new(serial_port) + }; +} +``` + +Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static`. However, this time we use `lazy_static` to ensure that the `init` method is called before first use. We're using the port address `0x3F8`, which is the standard port number for the first serial interface. + +[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics + +To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros: + +```rust +#[doc(hidden)] +pub fn _print(args: ::core::fmt::Arguments) { + use core::fmt::Write; + SERIAL1.lock().write_fmt(args).expect("Printing to serial failed"); +} + +/// Prints to the host through the serial interface. +#[macro_export] +macro_rules! serial_print { + ($($arg:tt)*) => { + $crate::serial::_print(format_args!($($arg)*)); + }; +} + +/// Prints to the host through the serial interface, appending a newline. +#[macro_export] +macro_rules! serial_println { + () => ($crate::serial_print!("\n")); + ($fmt:expr) => ($crate::serial_print!(concat!($fmt, "\n"))); + ($fmt:expr, $($arg:tt)*) => ($crate::serial_print!( + concat!($fmt, "\n"), $($arg)*)); +} +``` + +The `SerialPort` type already implements the [`fmt::Write`] trait, so we don't need to provide an implementation. + +[`fmt::Write`]: https://doc.rust-lang.org/nightly/core/fmt/trait.Write.html + +Now we can print to the serial interface in our `main.rs`: + +```rust +// in src/main.rs + +mod serial; + +#[cfg(not(test))] +#[no_mangle] +pub extern "C" fn _start() -> ! { + println!("Hello World{}", "!"); // prints to vga buffer + serial_println!("Hello Host{}", "!"); + + loop {} +} +``` + +Note that the `serial_println` macro lives directly under the root namespace because we used the `#[macro_export]` attribute, so importing it through `use crate::serial::serial_println` will not work. + +### QEMU Arguments + +To see the serial output in QEMU, we can use the `-serial` argument to redirect the output to stdout: + +``` +> qemu-system-x86_64 \ + -drive format=raw,file=target/x86_64-blog_os/debug/bootimage-blog_os.bin \ + -serial mon:stdio +warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5] +Hello Host! +``` + +If you chose a different name than `blog_os`, you need to update the paths of course. Note that you can no longer exit QEMU through `Ctrl+c`. As an alternative you can use `Ctrl+a` and then `x`. + +As an alternative to this long command, we can pass the argument to `bootimage run`, with an additional `--` to separate the build arguments (passed to cargo) from the run arguments (passed to QEMU). + +``` +bootimage run -- -serial mon:stdio +``` + +Instead of standard output, QEMU supports [many more target devices][QEMU -serial]. For redirecting the output to a file, the argument is: + +[QEMU -serial]: https://qemu.weilnetz.de/doc/qemu-doc.html#Debug_002fExpert-options + +``` +-serial file:output-file.txt +``` + +## Shutting Down QEMU + +Right now we have an endless loop at the end of our `_start` function and need to close QEMU manually. This does not work for automated tests. We could try to kill QEMU automatically from the host, for example after some special output was sent over serial, but this would be a bit hacky and difficult to get right. The cleaner solution would be to implement a way to shutdown our OS. Unfortunatly this is relatively complex, because it requires implementing support for either the [APM] or [ACPI] power management standard. + +[APM]: https://wiki.osdev.org/APM +[ACPI]: https://wiki.osdev.org/ACPI + +Luckily, there is an escape hatch: QEMU supports a special `isa-debug-exit` device, which provides an easy way to exit QEMU from the guest system. To enable it, we add the following argument to our QEMU command: + +``` +-device isa-debug-exit,iobase=0xf4,iosize=0x04 +``` + +The `iobase` specifies on which port address the device should live (`0xf4` is a [generally unused][list of x86 I/O ports] port on the x86's IO bus) and the `iosize` specifies the port size (`0x04` means four bytes). Now the guest can write a value to the `0xf4` port and QEMU will exit with [exit status] `(passed_value << 1) | 1`. + +[list of x86 I/O ports]: https://wiki.osdev.org/I/O_Ports#The_list +[exit status]: https://en.wikipedia.org/wiki/Exit_status + +To write to the I/O port, we use the [`x86_64`] crate: + +[`x86_64`]: https://docs.rs/x86_64/0.5.2/x86_64/ + +```toml +# in Cargo.toml + +[dependencies] +x86_64 = "0.5.2" +``` + +```rust +// in src/main.rs + +pub unsafe fn exit_qemu() { + use x86_64::instructions::port::Port; + + let mut port = Port::::new(0xf4); + port.write(0); +} +``` + +We mark the function as `unsafe` because it relies on the fact that a special QEMU device is attached to the I/O port with address `0xf4`. For the port type we choose `u32` because the `iosize` is 4 bytes. As value we write a zero, which causes QEMU to exit with exit status `(0 << 1) | 1 = 1`. + +Note that we could also use the exit status instead of the serial interface for sending the test results, for example `1` for success and `2` for failure. However, this wouldn't allow us to send panic messages like the serial interface does and would also prevent us from replacing `exit_qemu` with a proper shutdown someday. Therefore we continue to use the serial interface and just always write a `0` to the port. + +We can now test the QEMU shutdown by calling `exit_qemu` from our `_start` function: + +```rust +#[cfg(not(test))] +#[no_mangle] +pub extern "C" fn _start() -> ! { + println!("Hello World{}", "!"); // prints to vga buffer + serial_println!("Hello Host{}", "!"); + + unsafe { exit_qemu(); } + + loop {} +} +``` + +You should see that QEMU immediately closes after booting when executing: + +``` +bootimage run -- -serial mon:stdio -device isa-debug-exit,iobase=0xf4,iosize=0x04 +``` + +## Hiding QEMU + +We are now able to launch a QEMU instance that writes its output to the serial port and automatically exits itself when it's done. So we no longer need the VGA buffer output or the graphical representation that still pops up. We can disable it by passing the `-display none` parameter to QEMU. The full command looks like this: + +``` +qemu-system-x86_64 \ + -drive format=raw,file=target/x86_64-blog_os/debug/bootimage-blog_os.bin \ + -serial mon:stdio \ + -device isa-debug-exit,iobase=0xf4,iosize=0x04 \ + -display none +``` + +Or, with `bootimage run`: + +``` +bootimage run -- \ + -serial mon:stdio \ + -device isa-debug-exit,iobase=0xf4,iosize=0x04 \ + -display none +``` + +Now QEMU runs completely in the background and no window is opened anymore. This is not only less annoying, but also allows our test framework to run in environments without a graphical user interface, such as [Travis CI]. + +[Travis CI]: https://travis-ci.com/ + +## Test Organization + +Right now we're doing the serial output and the QEMU exit from the `_start` function in our `main.rs` and can no longer run our kernel in a normal way. We could try to fix this by adding an `integration-test` [cargo feature] and using [conditional compilation]: + +[cargo feature]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-features-section +[conditional compilation]: https://doc.rust-lang.org/reference/attributes.html#conditional-compilation + +```toml +# in Cargo.toml + +[features] +integration-test = [] +``` + +```rust +// in src/main.rs + +#[cfg(not(feature = "integration-test"))] // new +#[cfg(not(test))] +#[no_mangle] +pub extern "C" fn _start() -> ! { + println!("Hello World{}", "!"); // prints to vga buffer + + // normal execution + + loop {} +} + +#[cfg(feature = "integration-test")] // new +#[cfg(not(test))] +#[no_mangle] +pub extern "C" fn _start() -> ! { + serial_println!("Hello Host{}", "!"); + + run_test_1(); + run_test_2(); + // run more tests + + unsafe { exit_qemu(); } + + loop {} +} +``` + +However, this approach has a big problem: All tests run in the same kernel instance, which means that they can influence each other. For example, if `run_test_1` misconfigures the system by loading an invalid [page table], it can cause `run_test_2` to fail. This isn't something that we want because it makes it very difficult to find the actual cause of an error. + +[page table]: https://en.wikipedia.org/wiki/Page_table + +Instead, we want our test instances to be as independent as possible. If a test wants to destroy most of the system configuration to ensure that some property still holds in catastrophic situations, it should be able to do so without needing to restore a correct system state afterwards. This means that we need to launch a separate QEMU instance for each test. + +With the above conditional compilation we only have two modes: Run the kernel normally or execute _all_ integration tests. To run each test in isolation we would need a separate cargo feature for each test with that approach, which would result in very complex conditional compilation bounds and confusing code. + +A better solution is to create an additional executable for each test. + +### Additional Test Executables + +Cargo allows to add [additional executables] to a project by putting them inside `src/bin`. We can use that feature to create a separate executable for each integration test. For example, a `test-something` executable could be added like this: + +[additional executables]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-project-layout + +```rust +// src/bin/test-something.rs + +#![cfg_attr(not(test), no_std)] +#![cfg_attr(not(test), no_main)] +#![cfg_attr(test, allow(unused_imports))] + +use core::panic::PanicInfo; + +#[cfg(not(test))] +#[no_mangle] +pub extern "C" fn _start() -> ! { + // run tests + loop {} +} + +#[cfg(not(test))] +#[panic_handler] +fn panic(_info: &PanicInfo) -> ! { + loop {} +} +``` + +By providing a new implementation for `_start` we can create a minimal test case that only tests one specific thing and is independent of the rest. For example, if we don't print anything to the VGA buffer, the test still succeeds even if the `vga_buffer` module is broken. + +We can now run this executable in QEMU by passing a `--bin` argument to `bootimage`: + +``` +bootimage run --bin test-something +``` + +It should build the `test-something.rs` executable instead of `main.rs` and launch an empty QEMU window (since we don't print anything). So this approach allows us to create completely independent executables without cargo features or conditional compilation, and without cluttering our `main.rs`. + +However, there is a problem: This is a completely separate executable, which means that we can't access any functions from our `main.rs`, including `serial_println` and `exit_qemu`. Duplicating the code would work, but we would also need to copy everything we want to test. This would mean that we no longer test the original function but only a possibly outdated copy. + +Fortunately there is a way to share most of the code between our `main.rs` and the testing binaries: We move most of the code from our `main.rs` to a library that we can include from all executables. + +### Split Off A Library + +Cargo supports hybrid projects that are both a library and a binary. We only need to create a `src/lib.rs` file and split the contents of our `main.rs` in the following way: + +```rust +// src/lib.rs + +#![cfg_attr(not(test), no_std)] // don't link the Rust standard library + +// NEW: We need to add `pub` here to make them accessible from the outside +pub mod vga_buffer; +pub mod serial; + +pub unsafe fn exit_qemu() { + use x86_64::instructions::port::Port; + + let mut port = Port::::new(0xf4); + port.write(0); +} +``` + +```rust +// src/main.rs + +#![cfg_attr(not(test), no_std)] +#![cfg_attr(not(test), no_main)] +#![cfg_attr(test, allow(unused_imports))] + +use core::panic::PanicInfo; +use blog_os::println; + +/// This function is the entry point, since the linker looks for a function +/// named `_start` by default. +#[cfg(not(test))] +#[no_mangle] // don't mangle the name of this function +pub extern "C" fn _start() -> ! { + println!("Hello World{}", "!"); + + loop {} +} + +/// This function is called on panic. +#[cfg(not(test))] +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + println!("{}", info); + loop {} +} +``` + +So we move everything except `_start` and `panic` to `lib.rs` and make the `vga_buffer` and `serial` modules public. Everything should work exactly as before, including `bootimage run` and `cargo test`. To run tests only for the library part of our crate and avoid the additional output we can execute `cargo test --lib`. + +### Test Basic Boot + +We are finally able to create our first integration test executable. We start simple and only test that the basic boot sequence works and the `_start` function is called: + +```rust +// in src/bin/test-basic-boot.rs + +#![cfg_attr(not(test), no_std)] +#![cfg_attr(not(test), no_main)] // disable all Rust-level entry points +#![cfg_attr(test, allow(unused_imports))] + +use core::panic::PanicInfo; +use blog_os::{exit_qemu, serial_println}; + +/// This function is the entry point, since the linker looks for a function +/// named `_start` by default. +#[cfg(not(test))] +#[no_mangle] // don't mangle the name of this function +pub extern "C" fn _start() -> ! { + serial_println!("ok"); + + unsafe { exit_qemu(); } + loop {} +} + + +/// This function is called on panic. +#[cfg(not(test))] +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + serial_println!("failed"); + + serial_println!("{}", info); + + unsafe { exit_qemu(); } + loop {} +} +``` + +We don't do something special here, we just print `ok` if `_start` is called and `failed` with the panic message when a panic occurs. Let's try it: + +``` +> bootimage run --bin test-basic-boot -- \ + -serial mon:stdio -display none \ + -device isa-debug-exit,iobase=0xf4,iosize=0x04 +Building kernel + Compiling blog_os v0.2.0 (file:///…/blog_os) + Finished dev [unoptimized + debuginfo] target(s) in 0.19s + Updating registry `https://github.com/rust-lang/crates.io-index` +Creating disk image at target/x86_64-blog_os/debug/bootimage-test-basic-boot.bin +warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5] +ok +``` + +We got our `ok`, so it worked! Try inserting a `panic!()` before the `ok` printing, you should see output like this: + +``` +failed +panicked at 'explicit panic', src/bin/test-basic-boot.rs:19:5 +``` + +### Test Panic + +To test that our panic handler is really invoked on a panic, we create a `test-panic` test: + +```rust +// in src/bin/test-panic.rs + +#![cfg_attr(not(test), no_std)] +#![cfg_attr(not(test), no_main)] +#![cfg_attr(test, allow(unused_imports))] + +use core::panic::PanicInfo; +use blog_os::{exit_qemu, serial_println}; + +#[cfg(not(test))] +#[no_mangle] +pub extern "C" fn _start() -> ! { + panic!(); +} + +#[cfg(not(test))] +#[panic_handler] +fn panic(_info: &PanicInfo) -> ! { + serial_println!("ok"); + + unsafe { exit_qemu(); } + loop {} +} +``` + +This executable is almost identical to `test-basic-boot`, the only difference is that we print `ok` from our panic handler and invoke an explicit `panic()` in our `_start` function. + +## A Test Runner + +The final step is to create a test runner, a program that executes all integration tests and checks their results. The basic steps that it should do are: + +- Look for integration tests in the current project, maybe by some convention (e.g. executables starting with `test-`). +- Run all integration tests and interpret their results. + - Use a timeout to ensure that an endless loop does not block the test runner forever. +- Report the test results to the user and set a successful or failing exit status. + +Such a test runner is useful to many projects, so we decided to add one to the `bootimage` tool. + +### Bootimage Test + +The test runner of the `bootimage` tool can be invoked via `bootimage test`. It uses the following conventions: + +- All executables starting with `test-` are treated as integration tests. +- Tests must print either `ok` or `failed` over the serial port. When printing `failed` they can print additional information such as a panic message (in the next lines). +- Tests are run with a timeout of 1 minute. If the test has not completed in time, it is reported as "timed out". + +The `test-basic-boot` and `test-panic` tests we created above begin with `test-` and follow the `ok`/`failed` conventions, so they should work with `bootimage test`: + +``` +> bootimage test +test-panic + Finished dev [unoptimized + debuginfo] target(s) in 0.01s +Ok + +test-basic-boot + Finished dev [unoptimized + debuginfo] target(s) in 0.01s +Ok + +test-something + Finished dev [unoptimized + debuginfo] target(s) in 0.01s +Timed Out + +The following tests failed: + test-something: TimedOut +``` + +We see that our `test-panic` and `test-basic-boot` succeeded and that the `test-something` test timed out after one minute. We no longer need `test-something`, so we delete it (if you haven't done already). Now `bootimage test` should execute successfully. + +## Summary + +In this post we learned about the serial port and port-mapped I/O and saw how to configure QEMU to print serial output to the command line. We also learned a trick how to exit QEMU without needing to implement a proper shutdown. + +We then split our crate into a library and binary part in order to create additional executables for integration tests. We added two example tests for testing that the `_start` function is correctly called and that a `panic` invokes our panic handler. Finally, we presented `bootimage test` as a basic test runner for our integration tests. + +We now have a working integration test framework and can finally start to implement functionality in our kernel. We will continue to use the test framework over the next posts to test new components we add. + +## What's next? +In the next post, we will explore _CPU exceptions_. These exceptions are thrown by the CPU when something illegal happens, such as a division by zero or an access to an unmapped memory page (a so-called “page fault”). Being able to catch and examine these exceptions is very important for debugging future errors. Exception handling is also very similar to the handling of hardware interrupts, which is required for keyboard support. From 79ce3bd883508f7a8b2f63e06003f32776acaee9 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 17 Apr 2019 12:04:36 +0200 Subject: [PATCH 02/42] Deprecate the old testing posts --- .../posts/{ => deprecated}/04-unit-testing/index.md | 4 ++++ .../posts/{ => deprecated}/05-integration-tests/index.md | 0 2 files changed, 4 insertions(+) rename blog/content/second-edition/posts/{ => deprecated}/04-unit-testing/index.md (98%) rename blog/content/second-edition/posts/{ => deprecated}/05-integration-tests/index.md (100%) diff --git a/blog/content/second-edition/posts/04-unit-testing/index.md b/blog/content/second-edition/posts/deprecated/04-unit-testing/index.md similarity index 98% rename from blog/content/second-edition/posts/04-unit-testing/index.md rename to blog/content/second-edition/posts/deprecated/04-unit-testing/index.md index deea9afa..63e68f3e 100644 --- a/blog/content/second-edition/posts/04-unit-testing/index.md +++ b/blog/content/second-edition/posts/deprecated/04-unit-testing/index.md @@ -18,6 +18,10 @@ This blog is openly developed on [GitHub]. If you have any problems or questions +## Requirements + +In this post we explore how to execute `cargo test` on the host system. This only works if you don't have a default target set in your `.cargo/config` file. If you don't have a `.cargo/config` file in your project, you're fine too. + ## Unit Tests for `no_std` Binaries Rust has a [built-in test framework] that is capable of running unit tests without the need to set anything up. Just create a function that checks some results through assertions and add the `#[test]` attribute to the function header. Then `cargo test` will automatically find and execute all test functions of your crate. diff --git a/blog/content/second-edition/posts/05-integration-tests/index.md b/blog/content/second-edition/posts/deprecated/05-integration-tests/index.md similarity index 100% rename from blog/content/second-edition/posts/05-integration-tests/index.md rename to blog/content/second-edition/posts/deprecated/05-integration-tests/index.md From ae46a98cdbe78cda4505d5f26885824777928485 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 17 Apr 2019 12:16:54 +0200 Subject: [PATCH 03/42] Add introduciotn for printing to the console section --- blog/content/second-edition/posts/04-testing/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index c6d646e9..6720a403 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -244,11 +244,11 @@ Our test runner now automatically closes QEMU and correctly reports the test res ## Printing to the Console - +To see the test output on the console, we need to send the data from our kernel to the host system somehow. There are various ways to achieve this, for example by sending the data over a TCP network interface. However, setting up a networking stack is a quite complex task, so we will choose a simpler solution instead. ### Serial Port -A simple way to achieve this is by using the [serial port], an old interface standard which is no longer found in modern computers. It is easy to program and QEMU can redirect the bytes sent over serial to the host's standard output or a file. +A simple way to send the data is to use the [serial port], an old interface standard which is no longer found in modern computers. It is easy to program and QEMU can redirect the bytes sent over serial to the host's standard output or a file. [serial port]: https://en.wikipedia.org/wiki/Serial_port From 66102b3aba02fabc67d7ae468107af024e39edf6 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 17 Apr 2019 18:54:31 +0200 Subject: [PATCH 04/42] Use version 0.2.0 of uart_16550 --- blog/content/second-edition/posts/04-testing/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 6720a403..ba8ea7fa 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -266,7 +266,7 @@ We will use the [`uart_16550`] crate to initialize the UART and send data over t # in Cargo.toml [dependencies] -uart_16550 = "0.1.0" +uart_16550 = "0.2.0" ``` The `uart_16550` crate contains a `SerialPort` struct that represents the UART registers, but we still need to construct an instance of it ourselves. For that we create a new `serial` module with the following content: From f0c5326b194eacbf3597733b30d39a23ace8d979 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Wed, 17 Apr 2019 19:00:10 +0200 Subject: [PATCH 05/42] Rename section --- blog/content/second-edition/posts/04-testing/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index ba8ea7fa..a150e487 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -375,9 +375,9 @@ TODO image Note that it's no longer possible to exit QEMU from the console through `Ctrl+c` when `serial mon:stdio` is passed. An alternative keyboard shortcut is `Ctrl+a` and then `x`. Or you can just close the QEMU window manually. -### Exit QEMU on Panic +### Print a Error Message on Panic -To also exit QEMU on a panic, we can use [conditional compilation] to use a different panic handler in testing mode: +To exit QEMU with an error message on a panic, we can use [conditional compilation] to use a different panic handler in testing mode: [conditional compilation]: https://doc.rust-lang.org/1.30.0/book/first-edition/conditional-compilation.html From 441ef219375a7230d972a1e4d211426096b91e9a Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 18 Apr 2019 17:43:17 +0200 Subject: [PATCH 06/42] Continue post --- .../second-edition/posts/04-testing/index.md | 152 +++++++++++++++++- 1 file changed, 145 insertions(+), 7 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index a150e487..1ec1cdd6 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -432,28 +432,166 @@ Now QEMU runs completely in the background and no window is opened anymore. This ## Testing the VGA Buffer -Now that we have a working test framework, we can create a few tests for our VGA buffer implementation. First, we create a very simple test that ensures that `println` works without panicking: +Now that we have a working test framework, we can create a few tests for our VGA buffer implementation. First, we create a very simple test to verify that `println` works without panicking: ```rust +// in src/vga_buffer.rs + #[test_case] -fn test_println() { +fn test_println_simple() { serial_print!("test_println... "); - println!("Test Output"); + println!("test_println_simple output"); serial_println("[ok]"); } ``` +The test just prints something to the VGA buffer. If it finishes without panicking, it means that the `println` invocation did not panic either. +To ensure that no panic occurs even if many lines are printed and lines are shifted off the screen, we can create another test: + +```rust +// in src/vga_buffer.rs + +#[test_case] +fn test_println_many() { + serial_print!("test_println... "); + for _ in 0..1000 { + println!("test_println_many output"); + } + serial_println("[ok]"); +} +``` + +We can also create a test function to verify that the printed lines really appear on the screen: + +```rust +// in src/vga_buffer.rs + +#[test_case] +fn check_println_output() { + serial_print!("test_println... "); + + let s = "Some test string that fits on a single line"; + println!("{}", s); + for (i, c) in s.chars().enumerate() { + let screen_char = WRITER.lock().chars[BUFFER_HEIGHT - 2][i].load(); + assert_eq!(char::from(screen_char.ascii_character), c); + } + + serial_println("[ok]"); +} +``` + +The function defines a test string, prints it using `println`, and then iterates over the screen characters of the static `WRITER`, which represents the vga text buffer. Since `println` prints to the last screen line and then immediately appends a newline, the string should appear on line `BUFFER_HEIGHT - 2`. + +By using [`enumerate`], we count the number of iterations in the variable `i`, which we then use for loading the screen character corresponding to `c`. By comparing the `ascii_character` of the screen character with `c`, we ensure that each character of the string really appears in the vga text buffer. + +[`enumerate`]: https://doc.rust-lang.org/core/iter/trait.Iterator.html#method.enumerate + +As you can imagine, we could create many more test functions, for example a function that tests that no panic occurs when printing very long lines and that they're wrapped correctly. Or a function for testing that newlines, non-printable characters, and non-unicode charactes are handled correctly. + +For the rest of this post, however, we will explain how to create _integration tests_ to test the interaction of different components together. ## Integration Tests -Our kernel has more features in the future -> ensure that printing works from the beginning on by testing println in minimal environment +The convention for [integration tests] in Rust is to put them into a `tests` directory in the project root (i.e. next to the `src` directory). Both the default test framework and custom test frameworks will automatically pick up and execute all tests in that directory. -## Split off a Library +[integration tests]: https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests -## No Harness +All integration tests are their own executables and completely separate from our `main.rs`. This means that each test needs to define its own entry point function. Let's create an example integration test named `basic_boot` to see how it works in detail: -should panic test +```rust +// in tests/basic_boot.rs + +#![no_std] +#![no_main] + +#![feature(custom_test_frameworks)] +#![test_runner(blog_os::test_runner)] +#![reexport_test_harness_main = "test_main"] + +use core::panic::PanicInfo; + +#[no_mangle] // don't mangle the name of this function +pub extern "C" fn _start() -> ! { + test_main(); + + loop {} +} + +fn test_runner(tests: &[&dyn Fn()]) { + unimplemented!(); +} + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + unimplemented!(); +} +``` + +Since integration tests are separate executables we need to provide all the crate attributes (`no_std`, `no_main`, `test_runner`, etc.) again. We also need to create a new entry point function `_start`, which calls the test entry point function `test_main`. We don't need any `cfg(test)` attributes because integration test executables are never built in non-test mode. + +We use the [`unimplemented`] macro that always panics as a placeholder for the `test_runner` and the `panic` function. Ideally, we want to implement these functions exactly as we did in our `main.rs` using the `serial_println` macro and the `exit_qemu` function. The problem is that we don't have access to these functions since tests are built completely separately of our `main.rs` executable. + +[`unimplemented`]: https://doc.rust-lang.org/core/macro.unimplemented.html + +### Create a Library + +The solution is to split off a library from our `main.rs`, which can be included by other crates and integration test executables. To do this, we create a new `src/lib.rs` file and move most of our `main.rs` to it: + +```rust +// src/lib.rs + +TODO +``` + +As you can see, we moved all module declarations, the `exit_qemu` function, and our test runner into the library. Since the test runner also runs for our library, it needs to define its own entry point function in test-mode. To share the implementation of the entry point between our `main.rs` and our `lib.rs`, we move it into a new `run` function. + +The remaining code of our `src/main.rs` is: + +```rust +// src/main.rs + +TODO +``` + +We add a new `use` statement that imports all the used functions and macros from our library component. The library is called like your crate, which is named `blog_os` in our case. From the `_start` entry point we call the `run` function of our `lib.rs` to use the same environment for our `main.rs` and our `lib.rs` tests. + +### Completing the Integration Test + +Like our `src/main.rs`, our `tests/basic_boot.rs` executable can import types from our new library. This allows us to import the missing components to complete our test. + +```rust +// in tests/basic_boot.rs + +TODO +``` + +Instead of reimplementing the `test_runner`, we use the `test_runner` function of the library. We deliberatly don't call the `run` function from `_start` to let the tests run in a minimal boot environment. This way, we can test that certain features don't depend on initialization code that we will add to our `run` function in future posts. + +For example, we can test that `println` works right after boot without needing any initialization: + +```rust +TODO +``` + +This test is very similar to the TODO vga buffer test that we created earlier in this post. The important difference is that TODO runs at the end of the `run` function and TODO runs directly after boot without running any initialization code beforehand. + +At this stage, a test like this doesn't seem very useful. However, when our kernel becomes more featureful in the future, integration tests like this will be useful for testing certain features in well defined environments. For example, we might want to prepare certain page table mappings and that are used in our tests. + +### Testing Our Panic Handler + +Another thing that we can test with an integration test is our panic handler function. The idea is the following: + +- Deliberately cause a panic in the test +- Add assertions in the panic handler that check the panic message and the file/line information +- Exit with a success exit code at the end of the panic handler + +This is similar to a should panic test in the default Rust test framework. The difference is that can't continue the test after our panic handler was called because we don't have support for unwinding and the catch_panic function. + +For cases like this, where more than a single test are not useful, we can use the `no harness` feature to omit the test runner completely. + +#### No Harness # Unit Tests From ade3e36856621f1f96e7d2d4a215592e69b619f7 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 18 Apr 2019 20:47:26 +0200 Subject: [PATCH 07/42] Continue post --- .../second-edition/posts/04-testing/index.md | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 1ec1cdd6..4c549808 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -593,6 +593,46 @@ For cases like this, where more than a single test are not useful, we can use th #### No Harness +The [`no_harness`] flag can be set for an integration test in the `Cargo.toml`. It disables both the default test runner and the custom test runner feature, so that the test is treated like a normal executable. + +Let's use the `no_harness` feature to create a panic handler test. First, we create the test at `tests/panic_handler.rs`: + +```rust +TODO +``` + +The code is similar to the `basic_boot` test with the difference that no test attributes are needed and no runner function is called. We immediately exit with an error from the `_start` entry point and the panic handler for now and first try to get it to compile. + +If you run `cargo xtest` now, you will get an error that the `test` crate is missing. This error occurs because we didn't set a custom test framework, so that the compiler tries to use the default test framework, which is unavailable for our panic. By setting the `no_harness` flag for the test in our `Cargo.toml`, we can fix this error: + +```toml +# in Cargo.toml + +[[test]] +name = "panic_handler" +harness = false +``` + +Now the test compiles fine, but fails of course since we always exit with an error exit code. + +#### Implement a Proper Test + +Let's finish the implementation of our panic handler test: + +```rust +TODO +``` + +We immediately `panic` in our `_start` function with a panic message. In the panic handler, we verify that the reported message and file/line information are correct. At the end of the panic handler, we exit with a success exit code because our panic handler works as intended then. We don't need a `qemu_exit` call at the end of our `_start` function, since the Rust compiler knows for sure that the code after the `panic` is unreachable. + +## Summary + +## What's next? +In the next post, we will explore _CPU exceptions_. These exceptions are thrown by the CPU when something illegal happens, such as a division by zero or an access to an unmapped memory page (a so-called “page fault”). Being able to catch and examine these exceptions is very important for debugging future errors. Exception handling is also very similar to the handling of hardware interrupts, which is required for keyboard support. + + + + # Unit Tests From 736160a910858eb6feda9f6c1901aef3c6f3a6d2 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Sat, 20 Apr 2019 11:49:50 +0200 Subject: [PATCH 08/42] Some improvements --- .../second-edition/posts/04-testing/index.md | 26 +++++++++++-------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 4c549808..7e8bbb11 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -48,7 +48,7 @@ Since the `test` crate depends on the standard library, it is not available for ### Custom Test Frameworks -Fortunately, Rust supports replacing the default test framework through the unstable [`custom_test_frameworks`] feature. This feature requires no external libraries and thus also works in `#[no_std]` environments. It works by collecting all functions annotated with a `#[test_case]` attribute and then invoking a specified runner function with the list of tests as argument. Thus it gives the implementation maximal control over the test process. +Fortunately, Rust supports replacing the default test framework through the unstable [`custom_test_frameworks`] feature. This feature requires no external libraries and thus also works in `#[no_std]` environments. It works by collecting all functions annotated with a `#[test_case]` attribute and then invoking a user-specified runner function with the list of tests as argument. Thus it gives the implementation maximal control over the test process. [`custom_test_frameworks`]: https://doc.rust-lang.org/unstable-book/language-features/custom-test-frameworks.html @@ -121,11 +121,13 @@ fn trivial_assertion() { } ``` -Of course the test succeeds and we see the `trivial assertion... [ok]` output on the screen. The problem is that QEMU never exits so that `cargo xtest` runs forever. +When we run `cargo xtest` now, we see the `trivial assertion... [ok]` output on the screen, which indicates that the test succeeded. + +After executing the tests, our `test_runner` returns to the `test_main` function, which in turn returns to our `_start` entry point function. At the end of `_start`, we enter an endless loop because the entry point function is not allowed to return. This is a problem, because we want `cargo xtest` to exit after running all tests. ## Exiting QEMU -Right now we have an endless loop at the end of our `_start` function and need to close QEMU manually. The clean solution to this would be to implement a proper way to shutdown our OS. Unfortunatly this is relatively complex, because it requires implementing support for either the [APM] or [ACPI] power management standard. +Right now we have an endless loop at the end of our `_start` function and need to close QEMU manually on each execution of `cargo xtest`. This is unfortunate because we also want to run `cargo xtest` in scripts without user interaction. The clean solution to this would be to implement a proper way to shutdown our OS. Unfortunatly this is relatively complex, because it requires implementing support for either the [APM] or [ACPI] power management standard. [APM]: https://wiki.osdev.org/APM [ACPI]: https://wiki.osdev.org/ACPI @@ -293,7 +295,9 @@ lazy_static! { } ``` -Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static`. However, this time we use `lazy_static` to ensure that the `init` method is called before first use. We're using the port address `0x3F8`, which is the standard port number for the first serial interface. +Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static` writer instance. By using `lazy_static` we can ensure that the `init` method is called exactly once on its first use. + +Like the `isa-debug-exit` device, the UART is programmed using port I/O. Since the UART is more complex, it uses multiple I/O ports for programming different device registers. The `SerialPort::new` function expects the address of the first I/O port of the UART as argument, from which it can calculate the addresses of all needed ports. We're passing the port address `0x3F8`, which is the standard port number for the first serial interface. [vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics @@ -324,7 +328,7 @@ macro_rules! serial_println { } ``` -The `SerialPort` type already implements the [`fmt::Write`] trait, so we don't need to provide an implementation. +The implementation is very similar to the implementation of our `print` and `println` macros. Since the `SerialPort` type already implements the [`fmt::Write`] trait, we don't need to provide our own implementation. [`fmt::Write`]: https://doc.rust-lang.org/nightly/core/fmt/trait.Write.html @@ -351,7 +355,7 @@ Note that the `serial_println` macro lives directly under the root namespace bec ### QEMU Arguments -To see the serial output in QEMU, we need use the `-serial` argument to redirect the output to stdout: +To see the serial output from QEMU, we need use the `-serial` argument to redirect the output to stdout: ```toml # in Cargo.toml @@ -567,7 +571,7 @@ Like our `src/main.rs`, our `tests/basic_boot.rs` executable can import types fr TODO ``` -Instead of reimplementing the `test_runner`, we use the `test_runner` function of the library. We deliberatly don't call the `run` function from `_start` to let the tests run in a minimal boot environment. This way, we can test that certain features don't depend on initialization code that we will add to our `run` function in future posts. +Instead of reimplementing the test runner, we use the `test_runner` function of the library. We deliberatly don't call the `run` function from `_start` to let the tests run in a minimal boot environment. This way, we can test that certain features don't depend on initialization code that we will add to our `run` function in future posts. For example, we can test that `println` works right after boot without needing any initialization: @@ -577,7 +581,7 @@ TODO This test is very similar to the TODO vga buffer test that we created earlier in this post. The important difference is that TODO runs at the end of the `run` function and TODO runs directly after boot without running any initialization code beforehand. -At this stage, a test like this doesn't seem very useful. However, when our kernel becomes more featureful in the future, integration tests like this will be useful for testing certain features in well defined environments. For example, we might want to prepare certain page table mappings and that are used in our tests. +At this stage, a test like this doesn't seem very useful. However, when our kernel becomes more featureful in the future, integration tests like this will be useful for testing certain features in well defined environments. For example, we might want to prepare certain page table mappings that are then used in the tests. ### Testing Our Panic Handler @@ -593,9 +597,9 @@ For cases like this, where more than a single test are not useful, we can use th #### No Harness -The [`no_harness`] flag can be set for an integration test in the `Cargo.toml`. It disables both the default test runner and the custom test runner feature, so that the test is treated like a normal executable. +The `harness` flag defines whether a test runner is used for an integration test. When it's set to `false`, both the default test runner and the custom test runner feature are disabled, so that the test is treated like a normal executable. -Let's use the `no_harness` feature to create a panic handler test. First, we create the test at `tests/panic_handler.rs`: +Let's create a panic handler test with a disabled `harness` flag. First, we create the test at `tests/panic_handler.rs`: ```rust TODO @@ -603,7 +607,7 @@ TODO The code is similar to the `basic_boot` test with the difference that no test attributes are needed and no runner function is called. We immediately exit with an error from the `_start` entry point and the panic handler for now and first try to get it to compile. -If you run `cargo xtest` now, you will get an error that the `test` crate is missing. This error occurs because we didn't set a custom test framework, so that the compiler tries to use the default test framework, which is unavailable for our panic. By setting the `no_harness` flag for the test in our `Cargo.toml`, we can fix this error: +If you run `cargo xtest` now, you will get an error that the `test` crate is missing. This error occurs because we didn't set a custom test framework, so that the compiler tries to use the default test framework, which is unavailable for our panic. By setting the `harness` flag to `false` for the test in our `Cargo.toml`, we can fix this error: ```toml # in Cargo.toml From 10718f2996cde36e0af45e1e7e3587cd7faeec69 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Mon, 22 Apr 2019 13:24:27 +0200 Subject: [PATCH 09/42] Continue post --- .../second-edition/posts/04-testing/index.md | 266 ++++++++++++++---- .../posts/04-testing/qemu-failed-test.png | Bin 0 -> 9098 bytes .../04-testing/qemu-test-runner-output.png | Bin 0 -> 8346 bytes 3 files changed, 217 insertions(+), 49 deletions(-) create mode 100644 blog/content/second-edition/posts/04-testing/qemu-failed-test.png create mode 100644 blog/content/second-edition/posts/04-testing/qemu-test-runner-output.png diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 7e8bbb11..24e3a0bb 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -38,7 +38,7 @@ We can see this when we try to run `cargo xtest` in our project: ``` > cargo xtest - Compiling blog_os v0.1.0 (/home/philipp/Documents/blog_os/code) + Compiling blog_os v0.1.0 (/…/blog_os) error[E0463]: can't find crate for `test` ``` @@ -64,6 +64,7 @@ To implement a custom test framework for our kernel, we add the following to our #![feature(custom_test_frameworks)] #![test_runner(crate::test_runner)] +#[cfg(test)] fn test_runner(tests: &[&dyn Fn()]) { println!("Running {} tests", tests.len()); for test in tests { @@ -72,17 +73,13 @@ fn test_runner(tests: &[&dyn Fn()]) { } ``` -Our runner just prints a short debug message and then calls each test function in the list. The argument type `&[&dyn Fn()]` is a [_slice_] of [_trait object_] references of the [_Fn()_] trait. It is basically a list of references to types that can be called like a function. +Our runner just prints a short debug message and then calls each test function in the list. The argument type `&[&dyn Fn()]` is a [_slice_] of [_trait object_] references of the [_Fn()_] trait. It is basically a list of references to types that can be called like a function. Since the function is useless for non-test runs, we use the `#[cfg(test)]` attribute to include it only for tests. [_slice_]: https://doc.rust-lang.org/std/primitive.slice.html [_trait object_]: https://doc.rust-lang.org/1.30.0/book/first-edition/trait-objects.html [_Fn()_]: https://doc.rust-lang.org/std/ops/trait.Fn.html -When we run `cargo xtest` now, we see that it now succeeds. However, we still see our "Hello World" instead of the message from our `test_runner`: - -TODO image - -The reason is that our `_start` function is still used as entry point. The custom test frameworks feature generates a `main` function that calls `test_runner`, but this function is ignored because we use the `#[no_main]` attribute and provide our own entry point. +When we run `cargo xtest` now, we see that it now succeeds. However, we still see our "Hello World" instead of the message from our `test_runner`. The reason is that our `_start` function is still used as entry point. The custom test frameworks feature generates a `main` function that calls `test_runner`, but this function is ignored because we use the `#[no_main]` attribute and provide our own entry point. To fix this, we first need to change the name of the generated function to something different than `main` through the `reexport_test_harness_main` attribute. Then we can call the renamed function from our `_start` function: @@ -104,11 +101,7 @@ pub extern "C" fn _start() -> ! { We set the name of the test framework entry function to `test_main` and call it from our `_start` entry point. We use [conditional compilation] to add the call to `test_main` only in test contexts because the function is not generated on a normal run. -When we now execute `cargo xtest`, we see the message from our `test_runner` on the screen: - -TODO image - -We are now ready to create our first test function: +When we now execute `cargo xtest`, we see the "Running 0 tests" message from our `test_runner` on the screen. We are now ready to create our first test function: ```rust // in src/main.rs @@ -117,11 +110,15 @@ We are now ready to create our first test function: fn trivial_assertion() { print!("trivial assertion... "); assert_eq!(1, 1); - println("[ok]"); + println!("[ok]"); } ``` -When we run `cargo xtest` now, we see the `trivial assertion... [ok]` output on the screen, which indicates that the test succeeded. +When we run `cargo xtest` now, we see the following output: + +![QEMU printing "Hello World!", "Running 1 tests", and "trivial assertion... [ok]"](qemu-test-runner-output.png) + +The `tests` slice passed to our `test_runner` function now contains a reference to the `trivial_assertion` function. From the `trivial assertion... [ok]` output on the screen we see that the test was called and that it succeeded. After executing the tests, our `test_runner` returns to the `test_main` function, which in turn returns to our `_start` entry point function. At the end of `_start`, we enter an endless loop because the entry point function is not allowed to return. This is a problem, because we want `cargo xtest` to exit after running all tests. @@ -222,10 +219,10 @@ When we run `cargo xtest` now, we see that QEMU immediately closes after executi Building bootloader Compiling bootloader v0.5.3 (/home/philipp/Documents/bootloader) Finished release [optimized + debuginfo] target(s) in 1.07s -Running: `qemu-system-x86_64 -drive format=raw,file=target/x86_64-blog_os/debug/ +Running: `qemu-system-x86_64 -drive format=raw,file=/…/target/x86_64-blog_os/debug/ deps/bootimage-blog_os-5804fc7d2dd4c9be.bin -device isa-debug-exit,iobase=0xf4, iosize=0x04` -error: test failed +error: test failed, to rerun pass '--bin blog_os' ``` The problem is that `cargo test` considers all error codes other than `0` as failure. @@ -337,7 +334,7 @@ Now we can print to the serial interface instead of the VGA text buffer in our t ```rust // in src/main.rs -```rust +#[cfg(test)] fn test_runner(tests: &[&dyn Fn()]) { serial_println!("Running {} tests", tests.len()); […] @@ -347,7 +344,7 @@ fn test_runner(tests: &[&dyn Fn()]) { fn trivial_assertion() { serial_print!("trivial assertion... "); assert_eq!(1, 1); - serial_println("[ok]"); + serial_println!("[ok]"); } ``` @@ -370,12 +367,23 @@ When we run `cargo xtest` now, we see the test output directly in the console: ``` > cargo xtest -TODO + Finished dev [unoptimized + debuginfo] target(s) in 0.02s + Running target/x86_64-blog_os/debug/deps/blog_os-7b7c37b4ad62551a +Building bootloader + Finished release [optimized + debuginfo] target(s) in 0.02s +Running: `qemu-system-x86_64 -drive format=raw,file=/…/target/x86_64-blog_os/debug/ + deps/bootimage-blog_os-7b7c37b4ad62551a.bin -device + isa-debug-exit,iobase=0xf4,iosize=0x04 -serial mon:stdio` +Running 1 tests +trivial assertion... [ok] ``` However, when a test fails we still see the output inside QEMU because our panic handler still uses `println`. To simulate this, we can change the assertion in our `trivial_assertion` test to `assert_eq!(0, 1)`: -TODO image +![QEMU printing "Hello World!" and "panicked at 'assertion failed: `(left == right)` + left: `0`, right: `1`', src/main.rs:55:5](qemu-failed-test.png) + +We see that the panic message is still printed to the VGA buffer, while the other test output is printed to the serial port. The panic message is quite useful, so it would be useful to see it in the console too. Note that it's no longer possible to exit QEMU from the console through `Ctrl+c` when `serial mon:stdio` is passed. An alternative keyboard shortcut is `Ctrl+a` and then `x`. Or you can just close the QEMU window manually. @@ -411,10 +419,22 @@ Now QEMU also exits for failed tests and prints a useful error message on the co ``` > cargo xtest -TODO + Finished dev [unoptimized + debuginfo] target(s) in 0.02s + Running target/x86_64-blog_os/debug/deps/blog_os-7b7c37b4ad62551a +Building bootloader + Finished release [optimized + debuginfo] target(s) in 0.02s +Running: `qemu-system-x86_64 -drive format=raw,file=/…/target/x86_64-blog_os/debug/ + deps/bootimage-blog_os-7b7c37b4ad62551a.bin -device + isa-debug-exit,iobase=0xf4,iosize=0x04 -serial mon:stdio` +Running 1 tests +trivial assertion... [failed] + +Error: panicked at 'assertion failed: `(left == right)` + left: `0`, + right: `1`', src/main.rs:65:5 ``` -We still see the QEMU window open for a short time, which we don't need anymore. +Since we see all test output on the console now, we no longer need the QEMU window that pops up for a short time. So we can hide it completely. ### Hiding QEMU @@ -434,6 +454,8 @@ Now QEMU runs completely in the background and no window is opened anymore. This [SSH]: https://en.wikipedia.org/wiki/Secure_Shell +At this point, we no longer need the `trivial_assertion` test, so we can delete it. + ## Testing the VGA Buffer Now that we have a working test framework, we can create a few tests for our VGA buffer implementation. First, we create a very simple test to verify that `println` works without panicking: @@ -441,15 +463,18 @@ Now that we have a working test framework, we can create a few tests for our VGA ```rust // in src/vga_buffer.rs +#[cfg(test)] +use crate::{serial_print, serial_println}; + #[test_case] fn test_println_simple() { serial_print!("test_println... "); println!("test_println_simple output"); - serial_println("[ok]"); + serial_println!("[ok]"); } ``` -The test just prints something to the VGA buffer. If it finishes without panicking, it means that the `println` invocation did not panic either. +The test just prints something to the VGA buffer. If it finishes without panicking, it means that the `println` invocation did not panic either. Since we only need the `serial_println` import in test mode, we add the `cfg(test)` attribute to avoid the unused import warning for a normal `cargo xbuild`. To ensure that no panic occurs even if many lines are printed and lines are shifted off the screen, we can create another test: @@ -458,11 +483,11 @@ To ensure that no panic occurs even if many lines are printed and lines are shif #[test_case] fn test_println_many() { - serial_print!("test_println... "); - for _ in 0..1000 { + serial_print!("test_println_many... "); + for _ in 0..200 { println!("test_println_many output"); } - serial_println("[ok]"); + serial_println!("[ok]"); } ``` @@ -472,17 +497,17 @@ We can also create a test function to verify that the printed lines really appea // in src/vga_buffer.rs #[test_case] -fn check_println_output() { - serial_print!("test_println... "); +fn test_println_output() { + serial_print!("test_println_output... "); let s = "Some test string that fits on a single line"; println!("{}", s); for (i, c) in s.chars().enumerate() { - let screen_char = WRITER.lock().chars[BUFFER_HEIGHT - 2][i].load(); + let screen_char = WRITER.lock().buffer.chars[BUFFER_HEIGHT - 2][i].read(); assert_eq!(char::from(screen_char.ascii_character), c); } - serial_println("[ok]"); + serial_println!("[ok]"); } ``` @@ -509,9 +534,8 @@ All integration tests are their own executables and completely separate from our #![no_std] #![no_main] - #![feature(custom_test_frameworks)] -#![test_runner(blog_os::test_runner)] +#![test_runner(crate::test_runner)] #![reexport_test_harness_main = "test_main"] use core::panic::PanicInfo; @@ -529,37 +553,153 @@ fn test_runner(tests: &[&dyn Fn()]) { #[panic_handler] fn panic(info: &PanicInfo) -> ! { - unimplemented!(); + loop {} } ``` -Since integration tests are separate executables we need to provide all the crate attributes (`no_std`, `no_main`, `test_runner`, etc.) again. We also need to create a new entry point function `_start`, which calls the test entry point function `test_main`. We don't need any `cfg(test)` attributes because integration test executables are never built in non-test mode. +Since integration tests are separate executables, we need to provide all the crate attributes (`no_std`, `no_main`, `test_runner`, etc.) again. We also need to create a new entry point function `_start`, which calls the test entry point function `test_main`. We don't need any `cfg(test)` attributes because integration test executables are never built in non-test mode. -We use the [`unimplemented`] macro that always panics as a placeholder for the `test_runner` and the `panic` function. Ideally, we want to implement these functions exactly as we did in our `main.rs` using the `serial_println` macro and the `exit_qemu` function. The problem is that we don't have access to these functions since tests are built completely separately of our `main.rs` executable. +We use the [`unimplemented`] macro that always panics as a placeholder for the `test_runner` function and just `loop` in the `panic` handler for now. Ideally, we want to implement these functions exactly as we did in our `main.rs` using the `serial_println` macro and the `exit_qemu` function. The problem is that we don't have access to these functions since tests are built completely separately of our `main.rs` executable. [`unimplemented`]: https://doc.rust-lang.org/core/macro.unimplemented.html +If you run `cargo xtest` at this stage, you will get an endless loop because the panic handler itself panics again. Remember the `Ctrl-a` and then `x` keyboard shortcut for exiting QEMU. + ### Create a Library -The solution is to split off a library from our `main.rs`, which can be included by other crates and integration test executables. To do this, we create a new `src/lib.rs` file and move most of our `main.rs` to it: +To make the required functions available to our integration test, we need to split off a library from our `main.rs`, which can be included by other crates and integration test executables. To do this, we create a new `src/lib.rs` file: ```rust // src/lib.rs -TODO +#![no_std] ``` -As you can see, we moved all module declarations, the `exit_qemu` function, and our test runner into the library. Since the test runner also runs for our library, it needs to define its own entry point function in test-mode. To share the implementation of the entry point between our `main.rs` and our `lib.rs`, we move it into a new `run` function. +Like the `main.rs`, the `lib.rs` is a special file that is automatically recognized by cargo. The library is a separate compilation unit, so we need to specify the `#![no_std]` attribute again. -The remaining code of our `src/main.rs` is: +To make our library work with `cargo xtest`, we need to also add the test functions and attributes: + +```rust +// in src/lib.rs + +#![cfg_attr(test, no_main)] +#![feature(custom_test_frameworks)] +#![test_runner(crate::test_runner)] +#![reexport_test_harness_main = "test_main"] + +use core::panic::PanicInfo; + +pub fn test_runner(tests: &[&dyn Fn()]) { + serial_println!("Running {} tests", tests.len()); + for test in tests { + test(); + } + unsafe { exit_qemu(QemuExitCode::Success) }; +} + +pub fn test_panic_handler(info: &PanicInfo) -> ! { + serial_println!("[failed]\n"); + serial_println!("Error: {}\n", info); + unsafe { + exit_qemu(QemuExitCode::Failed); + } + loop {} +} + +/// Entry point for `cargo xtest` +#[cfg(test)] +#[no_mangle] +pub extern "C" fn _start() -> ! { + test_main(); + loop {} +} + +#[cfg(test)] +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + test_panic_handler(info) +} +``` + +The above code adds a `_start` entry point and panic handler that are compiled in test mode using the `cfg(test)` attribute. By using the [`cfg_attr`] crate attribute, we can conditionally enable the `no_main` attribute in test mode. + +To make our `test_runner` available to executables and integration tests, we don't apply the `cfg(test)` attribute to it and make it public. We also factor out the implementation of our panic handler into a public `test_panic_handler` function, so that it is available for executables too. + +[`cfg_attr`]: https://doc.rust-lang.org/reference/conditional-compilation.html#the-cfg_attr-attribute + +We also move over the `QemuExitCode` enum and the `exit_qemu` function and make them public: + +```rust +// in src/lib.rs + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[repr(u32)] +pub enum QemuExitCode { + Success = 0x10, + Failed = 0x11, +} + +pub unsafe fn exit_qemu(exit_code: QemuExitCode) { + use x86_64::instructions::port::Port; + + let mut port = Port::new(0xf4); + port.write(exit_code as u32); +} +``` + +Now executables and integration tests can import these functions from the library and don't need to define their own implementations. To also make `println` and `serial_println` available, we move the module declarations too: + +```rust +// in src/lib.rs + +pub mod serial; +pub mod vga_buffer; +``` + +We make the modules public to make them usable from outside of our library. This is also required for making our `println` and `serial_println` macros usable, since they use the `_print` functions of the modules. + +Now we can update our `main.rs` to use the library: ```rust // src/main.rs -TODO +#![no_std] +#![no_main] +#![feature(custom_test_frameworks)] +#![test_runner(blog_os::test_runner)] +#![reexport_test_harness_main = "test_main"] + +use core::panic::PanicInfo; +use blog_os::println; + +#[no_mangle] +pub extern "C" fn _start() -> ! { + println!("Hello World{}", "!"); + + #[cfg(test)] + test_main(); + + loop {} +} + +/// This function is called on panic. +#[cfg(not(test))] +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + println!("{}", info); + loop {} +} + +#[cfg(test)] +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + blog_os::test_panic_handler(info) +} ``` -We add a new `use` statement that imports all the used functions and macros from our library component. The library is called like your crate, which is named `blog_os` in our case. From the `_start` entry point we call the `run` function of our `lib.rs` to use the same environment for our `main.rs` and our `lib.rs` tests. +The library is usable like a normal external crate. It is called like our crate, which is `blog_os` in our case. The above code uses the `blog_os::test_runner` function in the `test_runner` attribute and the `blog_os::test_panic_handler` function in our `cfg(test)` panic handler. It also imports the `println` macro to make it available to our `_start` and `panic` functions. + +At this point, `cargo xrun` and `cargo xtest` should work again. Of course, `cargo xtest` still loops endlessly (remember the `ctrl+a` and then `x` shortcut to exit). Let's fix this by using the required library functions in our integration test. ### Completing the Integration Test @@ -568,22 +708,50 @@ Like our `src/main.rs`, our `tests/basic_boot.rs` executable can import types fr ```rust // in tests/basic_boot.rs -TODO +#![test_runner(blog_os::test_runner)] + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + blog_os::test_panic_handler(info) +} ``` -Instead of reimplementing the test runner, we use the `test_runner` function of the library. We deliberatly don't call the `run` function from `_start` to let the tests run in a minimal boot environment. This way, we can test that certain features don't depend on initialization code that we will add to our `run` function in future posts. +Instead of reimplementing the test runner, we use the `test_runner` function from our library. For our `panic` handler, we call the `blog_os::test_panic_handler` function like we did in our `main.rs`. -For example, we can test that `println` works right after boot without needing any initialization: +Now `cargo xtest` exits normally again. When you run it, you see that it builds and runs the tests for our `lib.rs`, `main.rs`, and `basic_boot.rs` separately after each other. For the `main.rs` and the `basic_boot` integration test, it reports "Running 0 tests" since these files don't have any functions annotated with `#[test_case]`. + +We can now add tests to our `basic_boot.rs`. For example, we can test that `println` works without panicking, like we did did in the vga buffer tests: ```rust -TODO +// in tests/basic_boot.rs + +#[test_case] +fn test_println() { + serial_print!("test_println... "); + println!("test_println output"); + serial_println!("[ok]"); +} ``` -This test is very similar to the TODO vga buffer test that we created earlier in this post. The important difference is that TODO runs at the end of the `run` function and TODO runs directly after boot without running any initialization code beforehand. +When we run `cargo xtest` now, we see that it finds and executes the test function. -At this stage, a test like this doesn't seem very useful. However, when our kernel becomes more featureful in the future, integration tests like this will be useful for testing certain features in well defined environments. For example, we might want to prepare certain page table mappings that are then used in the tests. +The test might seem a bit useless right now since it's almost identical to one of the VGA buffer tests. However, in the future the `_start` functions of our `main.rs` and `lib.rs` might grow and call various initialization routines before running the `test_main` function, so that the two tests are executed in very different environments. -### Testing Our Panic Handler +By testing `println` in a `basic_boot` environment without calling any initialization routines in `_start`, we can ensure that `println` works right after booting. This is important because we rely on it e.g. for printing panic messages. + +### Future Tests + +The power of integration tests is that they're treated as completely separate executables. This gives them complete control over the environment, which makes it possible to test that the code interacts correctly with the CPU or hardware devices. + +Our `basic_boot` test is a very simple example for an integration test. In the future, our kernel will become much more featureful and interact with the hardware in various ways. By adding integration tests, we can ensure that these interactions work (and keep working) as expected. Some ideas for possible future tests are: + +- **CPU Exceptions**: When the code performs invalid operations (e.g. divides by zero), the CPU throws an exception. The kernel can register handler functions for such exceptions. An integration test could verify that the correct exception handler is called when a CPU execption occurs or that the execution continues correctly after resolvable exceptions. +- **Page Tables**: Page tables define which memory regions are valid and accessible. By modifying the page tables, it is possible to allocate new memory regions, for example when launching programs. An integration test could perform some modifications of the page tables in the `_start` function and then verify that the modifications have the desired effects in `#[test_case]` functions. +- **Userspace Programs**: Userspace programs are programs with limited access to the system's resources. For example, they don't have access to kernel data structures or to the memory of other programs. An integration test could launch userspace programs that perform forbidden operations and verify that the kernel prevents them all. + +As you can imagine, many more tests are possible. By adding such tests, we can ensure that we don't break them accidentally when we add new features to our kernel or refactor our code. This is especially important when our kernel becomes larger and more complex. + +## Testing Our Panic Handler Another thing that we can test with an integration test is our panic handler function. The idea is the following: diff --git a/blog/content/second-edition/posts/04-testing/qemu-failed-test.png b/blog/content/second-edition/posts/04-testing/qemu-failed-test.png new file mode 100644 index 0000000000000000000000000000000000000000..aebfbc99b33110c126c0d9f3ae789235c4bb01eb GIT binary patch literal 9098 zcmeAS@N?(olHy`uVBq!ia0y~yV7kP>z_^x!je&u|`rBk>1_lO}VkgfK4h{~E8jh3> z1_lO+64!{5;QX|b^2DN4hTO!GRNdm_qSVy9;*9)~6VpC;F)%1Fc)B=-RLpsMw|YX# z^+fq+_upRMQzp(WpmKoK^&pFAV2%9|bDf4sjtU%^J0ILVoVq$id)1Uw^`xVlbY@P<^XC_q#@5!Gf7gcco++DO{ch*-_h0@zJG1k7*v$iX^J|kgU;NC#P@w$lhq_(l z+WL2!)Wev*bUweo@7+}G+|NZpxv@#N-|B0Zq`sV+TmO_>ICsCR?A0<>wR`*4d`Nk_ zzHH~;TfBPrlb+vR|NnVk_Kr1ui>~F|lbw7$^zD}$>n%P^Qr^F2UGBcE+t#OczhB)M zz45F2$#1vwcb;r{AMnt;aO+QR;j3GBzctS|t9Spris$BE^JD&9^?M!h`zc@c&ZOVI zadQ^*z4guyTy!(@^la^ak#>`|+$r8I)3s|$ZKUgKzjoQ{TR(oAe0}@pXZc0%#ANC= zAN;*@^V{q88&_}6ygRuy{oAtHZy#9a9{Rqy=J(g5W;YU=tjzR`KC&NXY2R7e`F30V zqt4}%V&qnY@89uzZGKMPzMntJd>I%Tq_4faoE~<#WNMG|myeIlV{eBsG8|AWcy{Jy z%^%_UYof02Qe#+mU!;9G2Q7I zv~*S2y6-D5nMlWf-~6fOvdQYNaawyyZ4W=pShQ_hRA^M@(yQw(Wo^Bb^|mJG=bM|w zFV~m}FfeeL_)cDyX|(F~mBoQ<&55rrW>|)pXNI0`+8;G|x2*ZD%)MJG z&A?#3>8gP38e<8arTe!`v3|eDxcp9`()r4BYj^&9_Smxam(RS~Z!>NF{Sf~7>-zqA zE93O%=B3*#nlp9c(~lR5E}Ka5wZHs6z39=A&OJYlmOuS**8KU4iOM?*9y*#Ph zzSiXapJ(RlSE#>k`jBCA>e8j4e}}sLpYWF7nPlBBe>T@W?bDam)|$U(e*QW5c=@M& z_Ek!ISH@kE&EGuPAolLxYvyL|{c<-$+56lU>*#$mw%A9)zo-LmF zIh&KAfH{));^M$x-pwVu@7}3CZ@cs9w8xUYuX@}{UscTrTRpjK_tdi8Q-hbEdvaL* zrw3cJW7XYvpE|x&+;Y2YBFV#+xjnk_b@cmlx!Y~^OZOJqp11j&WA|-ie|n~`ZE0<$ zOO=Vtv&zeF|9ttrM@PNSXX#3J&)DTFUx%y>yOgywYwPDDt#$v-x$mE`T)cht!5X3G z=i?>VUn;vl>#{B=+&Dkof3DwUlb2g=dz$iHdmCnLe!s>hIC$@)UEx=o=ggmfx_s|n z*~;hhYhCv}{moYt{b$xcgsc{`}czL#{5VUpGJe@7ejM`}cl}{<+=$=l*B4 z%b&GmGCXKAikWxn;@gtX-?rbMUbg&HLgoZD-&v*yw`GN{3|e~ob(%@erwh*VX9`R_ zmS6T=9je>@A~7m->h}GA_kCHmYxUP9J559@b8Qbh&gHLpu<*)Rjk$j1rSXd&8kGBN z{k-}6J`>}!3STchTJ|>o>$1%OrJv2~o^w}L-B@z{?b55a>L&i@7T0^1mv#Knqa?q1 zHEQmCXHwJB)b8H1F!}!b@0Z_yYhKSxFMeH|ck;`Zke`0D0`+uk%)kFXa{nCv-V(jT zo9~}dp8tC?Pk&u>+^1i^KQCTiSsM4L{$EMm!-XB^ zo3-nt?{*pg-K&hgzAAqrTmOvzbGE;#`z}x8H|Z~1-=B%E`a3b?s>bpBdEUYLQ@6`9 zJg~ZE;yWvF^1(|%?|#nPec9x&&Kdr2_f_F?p`mN-+73JJx*K=z(=_uZ&hozstk!;v zdA@7i<}2@?=iaZ*k6N3OSeH1vs_kXj?w!BinXkQeIcw{*VFaO+Z|AV>m?&`Ri zDnHNE|BI=6IlbP}_wJK3GdEYBE6&rL_snq4`{&=v|7`tp!25a8H{<_W|8JkSoulry zHe~UW7pvHw$d0wkCgSO|L{{Hu;`IGH+--0Dett8L*Kjpn&pZ_kyG;>y1-~AdL$+>$T`I-A( z?x{Z=b^pe)n_p$uGBEIUUtDoA!Q_N&`J5A9zLY$BF22dgb@95_PZ@0DLKjEv3)rw~ z6%X6tGiS~`J0h$8Y3*|VW!K-Onn*nTc02$4#GN~zR!wS`|L4(m_+i`ceed_ZKb>F@ zvG)0slaps=`KMaQJUe*%`KLW;cFo4=lT_57Pycl0kb-{gxuua3@wT?}mG74M$;>>J z@vtV(zR2eP)6e}+{#U1E+MG^eI_+5XTz3189aDB?8rj-T-v3G2e$MoLPppeRK5A7t zsU7z%_UA+SUkm?q*Z=g6l;YX6VZ)3~FO?7LT)RK<_q}gTTc*pMT)+SM;h$meJ@T^D zJtzNrUT?bp$H(hW>UVzMy7Rs7{#o{a=5D=xP4bJoB)`{oquLy+%t^ca`tFvPTzzq| z=F9iA;_uP#&fk9jBjw-S?XPVED0YFAr+`#%?X*qWcFetNa%L7#Btr%v@?-%!aux1G;-8cW96 zOs}%=Yx39c`(!-7O2_`+tLM*`Za?#Eb-LK6e!I^u|9)TJXRzJB|1^*OywiKn&pnxM zzjwmDmdd=!J5|qL=7;oOdR=zuZP}++;rn!+pPT2s+V}P-f-~8Wl|DPWrR%c84WM41KI`_Qz{qKst+VA~O&fEUz`g84k zy`_HTgL`*&erf%4`FpKpe%;~OpWd3w`YvC2ee0f+#|*7z+psef)UH~!WTm^Mk@@tc zOY0_Sr(3P@Il1G@AC1t^R1>MRyt1;IkMYO8o;iCqH8b<&CMz*9F}>C4M=j#})}Ngn zyKec^uvdX|^JlLMil5^jeBE=>bo=)eb&uwLep2o2suk@9zo_g)p(GAz%ZHT&AJK1mf-gTeL7GzFZxza~H zQ%o+2&-~BE+-`@B1YW_aW^EGc4{<#-yT=~>s<+Hw&b)M(e&&|KLX3N5x z3;KQEGBD)H_)glDIZ11Eu)(TTr{4d2r*3}tnQngN<5wA9uasDSE!wtvdh;hxUQ^#` zvwYp(Dw{0Zf8VcMDYCs<@%QD;v<*^kCNHu3ytd}~MaO@$&G#t%e}CM5wtUrT?a!Nj zg|t2Qi_MA%P2DB8rf-@5qKYi{?YGZnn4BuGIQ4wrkEb=yyW^+S?cZNxaQWw(=ldrd zESM6udamE{U z`@R12#G_L!FNN+3-dhsASSLlTcys8ipFd;H81&aYQ2u$$_Wu0+KeoR+G4b5>&)WR8 zriWkcKk>&Y=Ox4DJ3~ zYo;G;{dv^nY(h5d+%?y(yuyts^(>DYwMM>6(UuMb%il$JKsD?vJW-gSN-tR zDaqIFi^8vz?%i`Ic5>Z*`*~)UvX-ujvfO{+{FN1cm#?gTD*yAc`loz(%h~cZpS3^l z`~PL>ou6~fpYG1vwPjb#)vTw74kfKxWm-5dI?ie5&5y}tWomcx)Zdm({r}wd{mFFu z??=y^JC|BkrZ@lZpXs0U>pqHK&f9+Jb*YiDap;!*Em4{Ce@E@B+T{28$m&nOo?nlY zS+mFPR8qa!`s%;kmwoN)|4cour~1bG{f$qH&n?(LCGql-lRnSVR+)HvN3M!`Y4i2s zp8uc!%GlPaaL@l6AZK^eZ@1)<{a2REntM9u@{(2a&L;PNzg!YGW&56&*Ld`Fo*Zgz z<;ky3zW3=-*{q+(Kli3jDqH9d)yA6Ie#8hYH@RO zN5;g&{H?0{m*loM(d}@e+hVKqb8~!G-#uqxvAyWqo1Isyj&1)PSAFr((r$^#YKI?Y zRMwZCn`gVdX7zimw`JD#C3S|rudXb1e05PF-gfhZ`9D9+{B!nwb?v>%YIFOK2WEfv zm$RI#KX+cd?q!owxxE+scVC|O;YPXP*5tG_wd4Kw=PaLBWwbZXeE+Z4`={0am;dK{ zN#!&5=a*lrR=!r-ez&ap*WaUatjqI`|Mpn*#j5|Qv-}S)xgV?bPk#UZqJH-Ae<#;J z<@eK5U7K8a()!=-KObJFU9Nfk(B{j`)-9X&WV!!#dH3_Q{T%W2dkwv7jrRYS{%;t6 zrX+3I$Qpay=kgo31{Zm5R(~W{v-`-RB|8sl2 z`u!j8{%h&~pSXW|{hwv)pGb>8|MXUWul9Cd-KQ7t|4Fb5FE!*I+ z=JVP!XV0dXNG)CUYD-kX~hyA9LN_YxC!f!QOWhPrGf}9%Qri@td2)GiORZo4UTHR7_0F zNQU?F>%R8m)7iW;x8HhMw)^KZbNSOpyT!Yi)2_aK_NKjF@&6ZB`+55d&wqb+1Na zWxKc9oK3Fzet4H$zt?$voh6~6J1y+zr_Zl6Gv2(}bE=Q8|I@?i&ptoc=olBUHHd$D zQBvBJGxi&;)T>IK&AekOrMlPZ>mP%)dHVBrU3j7_e#^4%bBLJuw#e$L$kOl8_iucu zy8G5UFR84|43tpcRekI(e!cd*@9M0lFPG0hwYdLZq2D~4nLmGaA1#f)l#;2q^XJq1 zlYhUfD~p8E51 zDbKCDGIL+2Jl?%>{gun--dv2^b((XpB=6=M@@WQ9al#v~dTqacwq$+j(KSL04bqOQ zub#`WIJIJ3@%7m9b$t%&_g0=tH4n|)8q`0_=GP1P<#%5$xjxrxwduaS-xD(T?D^Ck z_a!}Ot5Mv=WM5m)Y0Im6-WSYRcIx`RZ(CJDtM=wq1wWq6@J;-;<+chn`_kJ-WwO~; zpMH>g+13zbKG@Xm;ci5m2!NS--qvft^Yry z?$yEQqEGgJ%j*6%&p)Z$Z#(&C;#}Tm3z-=TrkZ$OGI?3DJ2EQjRE9+ycU4(g+2ztD z&!0d4{A9BK(z|)7SyjI_adW=jbuwuG{EI8jbqa4!PfJVN*|)X++xltK&r3g@eZR)0 zG&a4eN-K2r)VTMTjy8Y({Q0F`+?n0|b5Fls_RDkco>Rv^$tJB@-py=cUpg*WxiMS z`<}PCl@CO{fBs;XKY6roVj@Gs-zz4*uP$l??~MymUG6{EZ~y)&A-+*>!)End&zLuV z{`t-4=bhSo-fnhWxz^s5E3XQ~hCbdpdG)HOeLpswTp7Ilxc942&(&AGV%1B7(>Ly) zVZ8kOlaB>gS7}Wpc08gF`3pTy-);+cJLp zr=^bi?``d;*KaNeu5#~Ndi`~3U1H%Q%k!nN{q9S=^Y7QbHIe+%9+`D_N`Cc4?&9db z6?XrZKfjdOeUX9TcW&yFotNJ(@v?4ysId58fsJ}qb++@!fk@L~~caR{Q%eY1*`3 zlDx-Dw??h~xMkA4kY&r38O!LTuDtI*`J|qa&!mS1CVZFPa=uru`(U!t-S^chujQ&P z%nUKwzItn2?&OspTYP%;&o@CgVk6_LHh*0hK2LtV;M|nEvCEEc=aB5T>JR_6Bjo%k znU}e@qpq)-+ih z=ij^bpZihoq`|=O#^s~cCD0I_^_O6428IVwFJ!E%Y~KGr;Cy|TECWM9hRu%ye`nvh z!Oy^OW0C?d0|P5dCIf?pf(J8$1EZr90|OVwB?g87hY4&94NOfY3=ARyOBfjzG>j@8 z4S~@R7!85Z5Eu=Cferx&bAe_hyr28Mult9m^Ah5oF&USRREFgH9e_d?unQw9e1=9RJx3;_!^Pd+5w+Y|RT zQ<6O~K6`n@o1;@il+Q3QwBJuvzU2FP-PPk&*@vH;N`Cvf`sISR*?)_7XP#g3dEWJX zb$k7q85u0j=t)n!mmyNN_psHhUE$*I-lkQH^D{BT*q51XRTg9lf5FbsAbq>9`AcF+ zMEC)LYm5wh>tfg5+IX|(Rddo`o5zRE-v3zeU7z7VZ_Xoyz2BI2Z)MD1>|k}0<3m_# z*{1b(XQjkjOJ2NpvM}~+w+17_j$NzR<#+6_{Le18>)Kleh6d-y2if11JBu+eXec7r z;Riy~S*6Mj|74a4Y5Jo0teusEdpCXV9`?y!9xl0WK6#Ek z!|%7V*fQmNp00c5o5k1NT=DOXYc=1k!nfb!Grw8ix!=Ke%QEeVq{SMmlTYmrT&tU~ z_Wy#rZ~SUs$KPK!?`vu9_g`U8&2Inqdwp}Y{p9?u^S`ctuV&%1>1^5Cn=8(^J^i-m zjLP$4e!tSUKR@qx?&{oGOKfMqP1nD*Y5wy+j=`UH7Mkw&H-G*3t=;0XUHeymy}5ha zzPIOYUV5L}a5(4dxwhGVcNVM5&y%*b_msG`uFrFx_RX|QRbEqMzF!TwWnUed_O@92 z{QD<==YN^;?N`Kh<4lKnOYTnIw=Z;NcICPE*Xs7Xx31jJ=e6G_OP#S|`~3UWfA{XW zyorJRU^D+6d%218u56V#c!V8?OZK1d{cH7{f6>~$zY2dJ6>Oa$#k`?jjqlR$TF0w#xBB^iH(#@#ye&O9ATGLY zdFr;RI%D%w{13AW^X_QW?w;6O^5(bc*2h^n{0F|3nJCo;s<&(HzxbMcXFa3Yrw7d6^Y>-3Gnkv%i!+#gf57}*e%?x1hM3r88yMdBXzwiY z(_t{5_f=eUeJx*ucBzp36o#Ikl?(=!!PaF5KvVQ8^{a)Jd{64TyJUOlz4r!Dq3`dOt$naHbsN{^XG%;AGI8xm zS;ym8Kd+YDWxngs-4n(P2fnT@)-AI-IrsXu>e^b_X<1AR3HEQRC)|5ucGdUuI$rmJ z>vsFE`Q^XfGoLYf1v|rzf3cGVU#7f%$rt)HE>VP`f%)w$)|am>72?kyXJa^UW8sb) z1^P0_W-pNVUctn`z$^$_3m|jMjG5t%2A|7>eRA(4cHI2)ZtHqRhJwF4vdmL%zIgrQ z(}A64Ki2L4QLy>#_1Rw*9sV1!z4GtI+^cd741C*H8@X3)*UorjQX{uy!>;gL!6S?e zeA}|iHY!F7F90!_4c(;}82F^2t$gbO+ckPj3(m7kp1v$!a?m z3;nju+8x_5V~gz}j-_gr<=*{c6&+Ywl`Wt$K;y z1@`y2G!hiyQDlan_0teoP= z7hn03W}9sa^|-6EBkxO`Pt^G(zwe##JGa?0ZvK~;JLj&xnE&?M=4&doJ7y=}^0j3s z*jx3X@%QOvWpAGI2ixyhKd)D>>dvbD<)7l-96dVY_5Qg1e;qeZ?)LUR_xfy|I3}&uyApZ(sa%)~%XBW28MOLk3y}>oPXT5{w2Wh;MUFWmY1^s_vnQ%GZaL;{oPTbWWmf(@FO;Sbr}Q0 ufjU8IcMIpLRDNjuy}lIGYYq7MpRqAvciQ*OQtAv03=E#GelF{r5}E+tUCvkl literal 0 HcmV?d00001 diff --git a/blog/content/second-edition/posts/04-testing/qemu-test-runner-output.png b/blog/content/second-edition/posts/04-testing/qemu-test-runner-output.png new file mode 100644 index 0000000000000000000000000000000000000000..1295f2e18b86aff6861aaad63e9e078b398eac4c GIT binary patch literal 8346 zcmeAS@N?(olHy`uVBq!ia0y~yV7kP>z_^x!je&u|`rBk>1_lO}VkgfK4h{~E8jh3> z1_lO+64!{5;QX|b^2DN4hTO!GRNdm_qSVy9;*9)~6VpC;F)%1Fc)B=-RLpsMw{n8o zbI1E1!+Pp8}9vX zk~ZH{w5xc}oRIYRM{{1!-*n-ncYglg%keiZ{YsZyY;~O}I&Oc?!?kbgOXs~_AOG-* z>9+mRa!s?cerJ-}5>6nAP$4wZ;8K)Aol)-|OD|fBBO; zujiLe)va@Gt?yiIynoBmjpy?J`hI(6v;Vw`=k9OzmS4kf%O?NT-)?y-^Y#3PM|FD& z-z`1A$z}Il+kF4uxj!c^nd<+}hTnB=%>JAOy!A5MY_C0gonO1>?fZX{|K5o0`CB&m zeR1{c{F+C5{+Z-fZ2I8;O)~uUvHM$e>*hawogb4kS#+Dhv1i>mKka#TU|CkTbzO6!K%_17ane%v2IeP%lmt^XNz}-GcXiLTGagT z0EgMv;=RfY3!8eQSFCP1U;Z%nS{uBW`b7xk}6T@{-KiCX>x_XH8vsGIx?z z=+>%z{!_Qz{j+EO)#taj#Y%sFawuIs-DX}+X8h}2UURKVmwUxtU-|m2e6(HBlZ;*2 z)*K8BthcuL-rhFT#CO(7mBkk=&ZcY&I6Xp{thU zUf$;Gw)mvjv6*`Dp`owN&MHgYf9$M^Z?Vyj*H$mD%`MqA_sU&AtJ3t_j(2VM_wzC| ztiBoL(m%V$?J!T<;X5^-&+dG=Z12u@yIyZjKmSg~s>Fhw@6Gwg{q_^}YhQZLeEI$I z+tTIkQ#MznR90o|%)Q-n{4o#P;Wu}8n{U5YQ|o zGUw<)i^ZOkLrQn;jeGVi-S~FwU(aPHA3a=bX=hdYb(Q(_*JbCI_+8@jOZk5>aMiz> zb$gf154K-tcG|uF*$ai4ceiTqPFtliS;oFXr`uP5_qym4Ia>>FT5(sGr5)^7WjGL; zqPs|aGEdv$FI9W*6o0?F^V_YvGks?HsQJ!Ho3rcPvdb$kzr12u@uA?}_kZ8hO{A9I zf4x>_;yl0G8MnseyzMh-ndfbjm9MN;z78o>_m`1>!fy9rrp;cPIPTADEy~{fFj-Rm zCPL#M!}RIwmIWUIZf*13e%W(XW<>cbzi)4D&Yd~4?&FQ@S%H1JdQV>8`!j9NSLwKU z>vtBWzk2HY^YZyxllPU^cYl7kd~)`)NlO=RuDs0u@ARME^|pVRCF-9&|J*14dv8kG zoTJnC+1h{J8$bWz@jA8tAJpxqWnItm*&b;Z^;GozkLN#smw)-RaQ=^bf4)5b{Zc1> z{{A29>on?L{**tzzwUSaiREI?jDxpI%E&&MdjE6q&wJtalmGv7{%`W$YKOuy#)j3W zDpIxF*WW%p`+iNn)m^Q{4>fXcZFzlY+p5(eD=)qNZX~(A^2NgMDK?TzZ@*o-ZuQg- zx#jCt?fH53`!f5>(bDQ~Pp&-i`nY8OESnz>oHK8E?z(&L);6`rg?Y1HK2d+aQ@8l7 z#@T|OJ-_3>r*97V`swce&o6Ck^De*t8hrivz7zjHK2|@MpLP7xr=o9fZ_k&t7Bez8 zpB`5)b+-ASLi0g|^Ka}Yyk5WCZ1ZO0r_!0~mixZ!x%2(+e~tTX_fF;W*Prfv-=}}K%If;g%vn6_&3j(X ze&-ph?%nPETq1o=-1_Oe-&Os+Wlck5B6+~cE;c{Qi+pW1$Z+XQ9#|4099o?m6*@AE2Ta>{e<{mS-V_}`s0 z-~0FWr_T4)>2}x7`G(K3nfU$xBmF1L&3;y^?E^}xB4@l!*%Xobee3lldACiz^8S3b z|HH07*Zk{DCg+(iK00&HyXf~PeM@I=Tf4dX{JNA)@3&ar-I8nlS2O$ZxovMN?_cC% zcu-_GbBm;}8>rx#Id|^aM~{NSmS>pwZWcOld39EZwmMf>=+uiDb9V0h`DSls{=CYn z_+NAD)E0Z@?fH1@O+-Y5qpLz z70>638(Y;_cy7usoblIt@tv}}CnF`|Ew;>%-Rt-5-HekdFC+5qY_zU_$7=m)|9Af7 zn{~D&s4m(5|Ddiu|K%;YY3a{DneTg@yYsDT{A6MEK6THyf0x#uzP<0W_2k%utNu)`j+&>_W&FLG%kH1-=Vco0 z{J3w=*U-(Cf6DDAJ++RT|GNCG&#dMCw^x@v72C9&{oBv`Qeq4biZ)hFddU54pMtly z_e>u(?epf@?i0i3+RU<<8*tmi_uIQ#$9Xl6PJX%dQe@)Z>ijdGK9$V58v1_k_t}2Z z&mXqiPq>(|rD*0JHT^xA`5=WM#-Yb1d%ml?7jsO#Z<(aO_lK8HJ?kDV?tl6_uii0l zcbQ$h#hxkh|9=So+`YaoFmKLWJ!88u2AtugL6^vz)cd_-_K+>C=^0PsdM-fA{Uu`>#ebd%CwxJgIVa-uiQAuZLvr`*XAA zmxsBrwSV8``I65+&#yB%Tpc%ecKn}1A^VN@vR75z!StFe3Ezulj7`S;lD|9W};Y43OY)y?_s=hpvUcm65w`s&(! zzh=#Sx$SJ5ZT{v;x92?P85v?`uF}e!?OCRylVTyGVp+TF@T)03lh6AHg@&q4E);wA ze7XAnRI77`4mq_QeyEwOqpLgD=-jHzc)#;!XU9t}_nVt`>B;J})h6m_gou)zVt1#nKMu3Oe%ZnA{IU``n$2G+i`JYpJnQ%>-Jrc zIiW89FG4PQ?!7znPADI@dKu&>Zity*s%xb(gM{rAtc_y1b|Jc#}N?A`Zwr{3o8 zQ2TBFBcSg6%AI?X=7extZ@Qp%{Mvc*{P6D!?>;d$+se$aqiT!itZlxl*7zP=u}W+I z@7?m2kFCn@RUW@}@zucKz8_(GCOK;1#nKye?|B2|^>bW($ zylm;&gD=acs$|YKGu~aEs+QX)8}GCG*P$mTYkq|P%f7dNkCpo_ul!~6<7`g`J=x0Wm_D_kgd%-OkUT@az?YVWsgw|DOFP6=@xWm_I z_RpO!zf9QXH>*tVb8+aTz_!I}<^)E4vSVb>v%h*OmTkJ&^vl~WXUc|#KA%^;uP`_6 z%c`qCFL`@=XWn`l_+shNr!S4PPwVOP2ZdhVExsy{-i3e+a6J^pAh`J_d6cXwn|)Tv9Cf~HNMe!Xt*jtZs8CoLv_w3zHU`EYyx zl6NIWyw9!N`($FJ)2nvB|EjJ#&!*Eb(^$?wY}@rGCnhe|-u`RdrI#h2+>cMYyv);B za_P*TPpyZ&e|DaiJ@>ulbN1(J($zoz{k}h;{_}~4HQz2gx@DAm`^m+}%_Y0$>hAmg zdB!b6bMxtUcYlAfa``-;{nt;g|MzSC)B1nw|4-d=^3kQFUw`XnEj!C=erLwP1WWgR zxx3%YCTabBmt6V2dH(so^`BLLUMv5z)#h7%oo@Yx{25nvf10%Y&)Yq3O+RPOue9K= ztmUDrVm$Ngot#OB5BJfC=C|KD$~SC9Yq%YOd!+Uxw!Fp-k z?vrMh?7sWv_ICZfe&(gGE+3m27j5gLpUdC<{_D(}-;=jIKKim@Zk^mcd%ykL zA5T%fnI3Gj_3?>`i)YT3elEIxhpo4_x1|5Em*1DQA8%irXjQ7dH%>qP=c#1N!bdKJ z31NHV`u4m}|MWioPwmflyZ2k(|1Mnq^v1!(lIP!_ocGi3*5di+>;JC&KkfYT`cDld zwYhn_f6u$Kt!}|b`KKT6S82?DcQ$QZ^io+~Q@GowIPC|1aj>`RaR9UUVH?#9U-Sv9?d7u5$mt_WB4Yitl&dVb`b&|tuPh-#9KK)7ZHYFDA z{IaM1{OMcbwd-#1aY_GMcj|s?Uw(N-`?Afla{HWFF9Tm5UApv2=v3kEZp#h-9iBaV zK3Qw#%(*9j^XWfpj9t8E!>Y?Y?KiE>J;mln%oet_++)ApN6mA;=VlAPyRV-wK7M=8 z{|fcp-_N8}zLdE>FD~#dsHf8Mx{Vm@@w_Xx7U}S)na7WA$6rxeey?@$;(daZojjqjHT-D*jpGWoo8Lxchv;M`#yg4e`!M4lFEqtP%cE|ronxyr&PXF)B zm%a?_r<>nI+0BvP`_QFq>*B4aALQ=(e>bi#Z10}~!Pdr6H>NH*Te0Qm*45Xg`P?s0 zc=z>`^RJ|=V~=^*Li2vUwx13hW;Q90C=PP7i9zA;0=I4{(mnE;xoH;kINGb917D+y~nLe}L+^tT2 z_Uu{Mv55cY|6M)x^t0N}Q{i&{mrZ8Qo$DET{pEN6c{V$C?3i)yQ|F%A>lgQ=rA;u= z4*wak)7d99)bs91cXs2S{_f?eCcBmY*6E-2PTx28M-`}_DZckbt9{V*+49r(UI{6k zo2P&C|4aX;CtGKKUUmKc%-#3DEW7jnQ=;9MdG1AvUY{^vIB@jJw%l1KRgC}E1*x|4 z$(>3t@L6S6y4-K}OVe56;^M^*4jh!RC{Xx&&*zu1akgM==;dtn@X)PApAu&`v-A7v zuMAf$ja{BOX<66Up`}&go$>G2L)`?A@w%z9W&dgaa1z%peU2b1~ ztB36|&+F_@q55@!c^Pu&m;d{cu}fBxm!V+eTDi)1-*&CMbTY%{REo*GbDeQBpIuyO z{qoDMm*v&k(^t-3xBH_|<>mkWx5_3x%$V~rV~^3?#h#&8SH8Boo2S0FPQNsCzR=^n z|ITFozGjlk*8G&Gz1ip3<&Cd)&GXZKc|IjA&Ft|-gTn_Ea<`RDN}SYld~&8o`L+uC zkKX>X(=IJrxvC^vkeA^@OlI*XuOoB)%BxB$*Piz~_U_f!rRPg(UtE2@{CbDqriXf$ zFGsid^jrC_e^XdA=c?tU=Xb6z^So>(*{>$a!=7F4aHsENN>JU;-QKRD;V++js^e#1 z2yeP7B|ZP?lYQxzA3b{HdN;28SG|%51H-n(Kdde>Fcg^XezUxcf#E>$!rj@=-|hK* zuzlX^zYGiwr;jmzzhB-a{hN`2AzgI|Bg2A*32Y1vOioe^3|t(U3=A3yqk^LWGMX4h zGs0-eFj^`O{Tktd@1$*ix2brB{ylYzd$#TN(ABT!<^El(Q@roo7rArK_vV()elAsS zJ^$k7uRj?W0^UVhD(#*6VRz8ug4OR|$DCfv|Mu&CHiie660+DC7(`y=r9V8|*JF44 zvXNeE?%MfDg^J6TSRCSEDEMjpw4>y9-R)b+i@zG~{1x&4Pix7_-0a82&o|pg&;7pV z-&gHYDISIcPv1;YzBF65_3n!z+gVi${q9)L|2o%(gQ4K*jcZr>Tv*n9vSeV0xw}pF z()7irzj8?=$uc~+R2mol*5Y!_EAOMnDvwRSz2~v_-_NWJJ7(otF1bEGVY}T2Gww@k zWf<*l8{NKn?dPpi+qMh@l}p{Of_g-?zT6`>^nNl|2K)gH0a~{QbUD zk(+^GiohiXh5!c-W(EgFMP3F5))o_xFV)ib{kgTYVC_@yx7%)sXVqVe|M%`@nc;R; zhJvjtCL7=1apY^Sf9XQI{_dYUfAKLg%#)q|NB5BaROOXTA`A^*x5-?RUT!kyEnh?E zSMB93&&#IY-0>!DeSYbj-t|-c&h8L!e`&t1dRgtw9O>J!&z$nO85%CnE!*_)=bcH_ zwK?-LyH12nySXFh<+f}4`#*QS;7evZb}y zHREO7+W&u#T|Zay=y}+*odLI*7!r!FGJn5+eIKN0wL_ytiGe}<5U6#P>i@QYNtS`( z{4{N=CFZklJk{&*J~z+4;}%ch36xnOp~N zZreHa?VQY++^O2PZ@qc(e&)UBKXo=weV?R#|7F_gX?xbc{d052ot&q?tu9-IANtMC zaBhvC-+%SzXa8~j^4UI1kF$5J)pd*2zbg&%!t2)MuKgc#Zp)q4s5I^SN5l4A?tPq_ zw)=PS#(n+WiQ4b~=DcQL2;a77rR;Psm6$UD*4MX%^&eYjzcGKayxY00XTr8`uk+F4 zRgz*@r}x~{{nt*OwzV>n`)}Nil0Wr&`u%rC>DB*yjz}{cI9hsfy>jikmps#Vuk~Tv zF=?57!KC*h4D0mLd0u{X_toF>^7>QZ56-^!2YAf4|SZ|M!9O`LA^h3Opa@zEfYKnjt5W&k-*0G>{?vbVK8b^Ur;-*&GB7YOc)I$ztaD0e F0sximbsGQx literal 0 HcmV?d00001 From 6257baaea505556d26634d1155201a02eb8051c0 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Mon, 22 Apr 2019 20:20:06 +0200 Subject: [PATCH 10/42] Some improvements --- blog/content/second-edition/posts/04-testing/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 24e3a0bb..80d9ff43 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -563,7 +563,7 @@ We use the [`unimplemented`] macro that always panics as a placeholder for the ` [`unimplemented`]: https://doc.rust-lang.org/core/macro.unimplemented.html -If you run `cargo xtest` at this stage, you will get an endless loop because the panic handler itself panics again. Remember the `Ctrl-a` and then `x` keyboard shortcut for exiting QEMU. +If you run `cargo xtest` at this stage, you will get an endless loop because the panic handler loops endlessly. You need to use the `Ctrl-a` and then `x` keyboard shortcut for exiting QEMU. ### Create a Library @@ -621,10 +621,10 @@ fn panic(info: &PanicInfo) -> ! { } ``` -The above code adds a `_start` entry point and panic handler that are compiled in test mode using the `cfg(test)` attribute. By using the [`cfg_attr`] crate attribute, we can conditionally enable the `no_main` attribute in test mode. - To make our `test_runner` available to executables and integration tests, we don't apply the `cfg(test)` attribute to it and make it public. We also factor out the implementation of our panic handler into a public `test_panic_handler` function, so that it is available for executables too. +Since our `lib.rs` is tested independently of our `main.rs`, we need to add a `_start` entry point and a panic handler when the library is compiled in test mode. By using the [`cfg_attr`] crate attribute, we conditionally enable the `no_main` attribute in this case. + [`cfg_attr`]: https://doc.rust-lang.org/reference/conditional-compilation.html#the-cfg_attr-attribute We also move over the `QemuExitCode` enum and the `exit_qemu` function and make them public: @@ -720,7 +720,7 @@ Instead of reimplementing the test runner, we use the `test_runner` function fro Now `cargo xtest` exits normally again. When you run it, you see that it builds and runs the tests for our `lib.rs`, `main.rs`, and `basic_boot.rs` separately after each other. For the `main.rs` and the `basic_boot` integration test, it reports "Running 0 tests" since these files don't have any functions annotated with `#[test_case]`. -We can now add tests to our `basic_boot.rs`. For example, we can test that `println` works without panicking, like we did did in the vga buffer tests: +We can now add tests to our `basic_boot.rs`. For example, we can test that `println` works without panicking, like we did in the vga buffer tests: ```rust // in tests/basic_boot.rs From 028d31cc5bd01d0006f80da5eaaf189b495604a6 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Mon, 22 Apr 2019 22:03:53 +0200 Subject: [PATCH 11/42] Make exit_qemu safe --- .../second-edition/posts/04-testing/index.md | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 80d9ff43..3d5eb457 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -185,15 +185,17 @@ pub enum QemuExitCode { Failed = 0x11, } -pub unsafe fn exit_qemu(exit_code: QemuExitCode) { +pub fn exit_qemu(exit_code: QemuExitCode) { use x86_64::instructions::port::Port; - let mut port = Port::new(0xf4); - port.write(exit_code as u32); + unsafe { + let mut port = Port::new(0xf4); + port.write(exit_code as u32); + } } ``` -We mark the function as `unsafe` because it relies on the fact that a special QEMU device is attached to the I/O port with address `0xf4`. The function creates a new [`Port`] at `0xf4`, which is the `iobase` of the `isa-debug-exit` device. Then it writes the the passed exit code to the port. We use `u32` because we specified the `iosize` of the `isa-debug-exit` device as 4 bytes. +The function creates a new [`Port`] at `0xf4`, which is the `iobase` of the `isa-debug-exit` device. Then it writes the the passed exit code to the port. We use `u32` because we specified the `iosize` of the `isa-debug-exit` device as 4 bytes. Both operations are unsafe, because writing to an I/O port can generally result in arbitrary behavior. For specifying the exit status, we create a `QemuExitCode` enum. The idea is to exit with the success exit code if all tests succeeded and with the failure exit code otherwise. The enum is marked as `#[repr(u32)]` to represent each variant by an `u32` integer. We use exit code `0x10` for success and `0x11` for failure. The actual exit codes do not matter much, as long as they don't clash with the default exit codes of QEMU. For example, using exit code `0` for success is not a good idea because it becomes `(0 << 1) | 1 = 1` after the transformation, which is the default exit code when QEMU failed to run. So we could not differentiate a QEMU error from a successfull test run. @@ -206,7 +208,7 @@ fn test_runner(tests: &[&dyn Fn()]) { test(); } /// new - unsafe { exit_qemu(QemuExitCode::Success) }; + exit_qemu(QemuExitCode::Success); } ``` @@ -294,7 +296,7 @@ lazy_static! { Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static` writer instance. By using `lazy_static` we can ensure that the `init` method is called exactly once on its first use. -Like the `isa-debug-exit` device, the UART is programmed using port I/O. Since the UART is more complex, it uses multiple I/O ports for programming different device registers. The `SerialPort::new` function expects the address of the first I/O port of the UART as argument, from which it can calculate the addresses of all needed ports. We're passing the port address `0x3F8`, which is the standard port number for the first serial interface. +Like the `isa-debug-exit` device, the UART is programmed using port I/O. Since the UART is more complex, it uses multiple I/O ports for programming different device registers. The unsafe `SerialPort::new` function expects the address of the first I/O port of the UART as argument, from which it can calculate the addresses of all needed ports. We're passing the port address `0x3F8`, which is the standard port number for the first serial interface. [vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics @@ -408,7 +410,7 @@ fn panic(info: &PanicInfo) -> ! { fn panic(info: &PanicInfo) -> ! { serial_println!("[failed]\n"); serial_println!("Error: {}\n", info); - unsafe { exit_qemu(QemuExitCode::Failed); } + exit_qemu(QemuExitCode::Failed); loop {} } ``` @@ -594,15 +596,13 @@ pub fn test_runner(tests: &[&dyn Fn()]) { for test in tests { test(); } - unsafe { exit_qemu(QemuExitCode::Success) }; + exit_qemu(QemuExitCode::Success); } pub fn test_panic_handler(info: &PanicInfo) -> ! { serial_println!("[failed]\n"); serial_println!("Error: {}\n", info); - unsafe { - exit_qemu(QemuExitCode::Failed); - } + exit_qemu(QemuExitCode::Failed); loop {} } @@ -639,11 +639,13 @@ pub enum QemuExitCode { Failed = 0x11, } -pub unsafe fn exit_qemu(exit_code: QemuExitCode) { +pub fn exit_qemu(exit_code: QemuExitCode) { use x86_64::instructions::port::Port; - let mut port = Port::new(0xf4); - port.write(exit_code as u32); + unsafe { + let mut port = Port::new(0xf4); + port.write(exit_code as u32); + } } ``` From 00b3ded7d6534c959a210703e33aae0366b6180b Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Mon, 22 Apr 2019 22:49:03 +0200 Subject: [PATCH 12/42] Add missing imports --- blog/content/second-edition/posts/04-testing/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 3d5eb457..7a251f5c 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -727,6 +727,8 @@ We can now add tests to our `basic_boot.rs`. For example, we can test that `prin ```rust // in tests/basic_boot.rs +use blog_os::{println, serial_print, serial_println}; + #[test_case] fn test_println() { serial_print!("test_println... "); From 22261d83037338abf8b1deff46cfb9c213530a45 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Tue, 23 Apr 2019 16:48:25 +0200 Subject: [PATCH 13/42] Write section about testing the panic handler --- .../second-edition/posts/04-testing/index.md | 176 ++++++++++++++++-- 1 file changed, 161 insertions(+), 15 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 7a251f5c..f84996bc 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -757,24 +757,36 @@ As you can imagine, many more tests are possible. By adding such tests, we can e ## Testing Our Panic Handler -Another thing that we can test with an integration test is our panic handler function. The idea is the following: +Another thing that we can test with an integration test is that our panic handler is called correctly. The idea is to deliberately cause a panic in the test function and exit with a success exit code in the panic handler. -- Deliberately cause a panic in the test -- Add assertions in the panic handler that check the panic message and the file/line information -- Exit with a success exit code at the end of the panic handler +Since we exit from our panic handler, the panicking test never returns to the test runner. For this reason, it does not make sense to add more than one test because subsequent tests are never executed. For cases like this, where only a single test function exists, we can disable the test runner completely and run our test directly in the `_start` function. -This is similar to a should panic test in the default Rust test framework. The difference is that can't continue the test after our panic handler was called because we don't have support for unwinding and the catch_panic function. - -For cases like this, where more than a single test are not useful, we can use the `no harness` feature to omit the test runner completely. - -#### No Harness +### No Harness The `harness` flag defines whether a test runner is used for an integration test. When it's set to `false`, both the default test runner and the custom test runner feature are disabled, so that the test is treated like a normal executable. -Let's create a panic handler test with a disabled `harness` flag. First, we create the test at `tests/panic_handler.rs`: +Let's create a panic handler test with a disabled `harness` flag. First, we create a skeleton for the test at `tests/panic_handler.rs`: ```rust -TODO +// in tests/panic_handler.rs + +#![no_std] +#![no_main] + +use core::panic::PanicInfo; +use blog_os::{QemuExitCode, exit_qemu}; + +#[no_mangle] +pub extern "C" fn _start() -> ! { + exit_qemu(QemuExitCode::Failed); + loop {} +} + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + exit_qemu(QemuExitCode::Failed); + loop {} +} ``` The code is similar to the `basic_boot` test with the difference that no test attributes are needed and no runner function is called. We immediately exit with an error from the `_start` entry point and the panic handler for now and first try to get it to compile. @@ -791,15 +803,149 @@ harness = false Now the test compiles fine, but fails of course since we always exit with an error exit code. -#### Implement a Proper Test +### Implementing the Test -Let's finish the implementation of our panic handler test: +Let's complete the implementation of our panic handler test: ```rust -TODO +// in tests/panic_handler.rs + +use blog_os::{serial_print, serial_println, QemuExitCode, exit_qemu}; + +const MESSAGE: &str = "Example panic message from panic_handler test"; +const PANIC_LINE: u32 = 14; // adjust this when moving the `panic!` call + +#[no_mangle] +pub extern "C" fn _start() -> ! { + serial_print!("panic_handler... "); + panic!(MESSAGE); // must be in line `PANIC_LINE` +} + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + serial_println!("[ok]"); + exit_qemu(QemuExitCode::Success); + loop {} +} ``` -We immediately `panic` in our `_start` function with a panic message. In the panic handler, we verify that the reported message and file/line information are correct. At the end of the panic handler, we exit with a success exit code because our panic handler works as intended then. We don't need a `qemu_exit` call at the end of our `_start` function, since the Rust compiler knows for sure that the code after the `panic` is unreachable. +We immediately `panic` in our `_start` function with a `MESSAGE`. In the panic handler, we exit with a success exit code. We don't need a `qemu_exit` call at the end of our `_start` function, since the Rust compiler knows for sure that the code after the `panic` is unreachable. If we run the test with `cargo xtest --test panic_handler` now, we see that it succeeds as expected. + +We will need the `MESSAGE` and `PANIC_LINE` constants in the next section. The `PANIC_LINE` constant specifies the line number that contains the `panic!` invocation, which is `14` in our case (but it might be different for you). + +### Checking the `PanicInfo` + +To ensure that the given `PanicInfo` is correct, we can extend the `panic` function to check that the reported message and file/line information are correct: + +```rust +// in tests/panic_handler.rs + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + check_message(info); + check_location(info); + + // same as before + serial_println!("[ok]"); + exit_qemu(QemuExitCode::Success); + loop {} +} +``` + +We will show the implementation of `check_message` and `check_location` in a moment. Before that, we create a `fail` helper function that can be used to print an error message and exit QEMU with an failure exit code: + +```rust +// in tests/panic_handler.rs + +fn fail(error: &str) -> ! { + serial_println!("[failed]"); + serial_println!("{}", error); + exit_qemu(QemuExitCode::Failed); + loop {} +} +``` + +Now we can implement our `check_location` function: + +```rust +// in tests/panic_handler.rs + +fn check_location(info: &PanicInfo) { + let location = info.location().unwrap_or_else(|| fail("no location")); + if location.file() != file!() { + fail("file name wrong"); + } + if location.line() != PANIC_LINE { + fail("file line wrong"); + } +} +``` + +The function takes queries the location information from the `PanicInfo` and fails if it does not exist. It then checks that the reported file name is correct by comparing it with the output of the compiler-provided [`file!`] macro. To check the reported line number, it compares it with the `PANIC_LINE` constant that we manually defined above. + +[`file!`]: https://doc.rust-lang.org/core/macro.file.html + +#### Checking the Panic Message + +Checking the reported panic message is a bit more complicated. The reason is that the [`PanicInfo::message`] function returns a [`fmt::Arguments`] instance that can't be compared with our `MESSAGE` string directly. To work around this, we need to create a `CompareMessage` struct: + +[`PanicInfo::message`]: https://doc.rust-lang.org/core/macro.file.html +[`fmt::Arguments`]: https://doc.rust-lang.org/core/fmt/struct.Arguments.html + +```rust +// in tests/panic_handler.rs + +/// Compares a `fmt::Arguments` instance with the `MESSAGE` string. +/// +/// To use this type, write the `fmt::Arguments` instance to it using the +/// `write` macro. If a message component matches `MESSAGE`, the equals +/// field is set to true. +struct CompareMessage { + equals: bool, +} + +impl fmt::Write for CompareMessage { + fn write_str(&mut self, s: &str) -> fmt::Result { + if s == MESSAGE { + self.equals = true; + } + Ok(()) + } +} +``` + +The trick is to implement the [`fmt::Write`] trait, which is called by the [`write`] macro with `&str` arguments. This makes it possible to compare the panic arguments with `MESSAGE`. By the way, this is the same trait that we implemented for our VGA buffer writer in order to print to the screen. + +[`fmt::Write`]: https://doc.rust-lang.org/core/fmt/trait.Write.html +[`write`]: https://doc.rust-lang.org/core/macro.write.html + +The above code only works for messages with a single component. This means that it works for `panic!("some message")`, but not for `panic!("some {}", message)`. This isn't ideal, but good enough for our test. + +With the `CompareMessage` type, we can finally implement our `check_message` function: + +```rust +// in tests/panic_handler.rs + +#![feature(panic_info_message)] // at the top of the file + +fn check_message(info: &PanicInfo) { + let message = info.message().unwrap_or_else(|| fail("no message")); + let mut compare_message = CompareMessage { equals: false }; + write!(&mut compare_message, "{}", message) + .unwrap_or_else(|_| fail("write failed")); + if !compare_message.equals { + fail("message not equal to expected message"); + } +} +``` + +The function uses the [`PanicInfo::message`] function to get the panic message. If no message is reported, it calls `fail` to fail the test. Since the function is unstable, we need to add the `#![feature(panic_info_message)]` attribute at the top of our test file. + +[`PanicInfo::message`]: https://doc.rust-lang.org/core/panic/struct.PanicInfo.html#method.message + +After querying the message, the function constructs a `CompareMessage` instance and writes the message to it using the `write!` macro. Afterwards it reads the `equals` field and fails the test if the panic message does not equal `MESSAGE`. + +Now we can run the test using `cargo xtest --test panic_handler`. We see that it passes, which means that the reported panic info is correct. If we use a wrong line number in `PANIC_LINE` or panic with an additional character through `panic!("{}x", MESSAGE)`, we see that the test indeed fails. ## Summary From 077b583effe41f36c23a05b9aacaf3c4ab675822 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Tue, 23 Apr 2019 16:49:24 +0200 Subject: [PATCH 14/42] Delete content from old unit testing post --- .../second-edition/posts/04-testing/index.md | 771 +----------------- 1 file changed, 9 insertions(+), 762 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index f84996bc..285b736c 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -956,229 +956,17 @@ In the next post, we will explore _CPU exceptions_. These exceptions are thrown -# Unit Tests +TODO: +- timeout? +- quiet? +- macro +- update date -## Testing the VGA Module -Now that we have set up the test framework, we can add a first unit test for our `vga_buffer` module: -```rust -// in src/vga_buffer.rs -#[cfg(test)] -mod test { - use super::*; +## OLD - #[test] - fn foo() {} -} -``` - -We add the test in an inline `test` submodule. This isn't necessary, but a common way to separate test code from the rest of the module. By adding the `#[cfg(test)]` attribute, we ensure that the module is only compiled in test mode. Through `use super::*`, we import all items of the parent module (the `vga_buffer` module), so that we can test them easily. - -The `#[test]` attribute on the `foo` function tells the test framework that the function is an unit test. The framework will find it automatically, even if it's private and inside a private module as in our case: - -``` -> cargo test - Compiling blog_os v0.2.0 (file:///…/blog_os) - Finished dev [unoptimized + debuginfo] target(s) in 2.99 secs - Running target/debug/deps/blog_os-1f08396a9eff0aa7 - -running 1 test -test vga_buffer::test::foo ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out -``` - -We see that the test was found and executed. It didn't panic, so it counts as passed. - -### Constructing a Writer -In order to test the VGA methods, we first need to construct a `Writer` instance. Since we will need such an instance for other tests too, we create a separate function for it: - -```rust -// in src/vga_buffer.rs - -#[cfg(test)] -mod test { - use super::*; - - fn construct_writer() -> Writer { - use std::boxed::Box; - - let buffer = construct_buffer(); - Writer { - column_position: 0, - color_code: ColorCode::new(Color::Blue, Color::Magenta), - buffer: Box::leak(Box::new(buffer)), - } - } - - fn construct_buffer() -> Buffer { … } -} -``` - -We set the initial column position to 0 and choose some arbitrary colors for foreground and background color. The difficult part is the buffer construction, it's described in detail below. We then use [`Box::new`] and [`Box::leak`] to transform the created `Buffer` into a `&'static mut Buffer`, because the `buffer` field needs to be of that type. - -[`Box::new`]: https://doc.rust-lang.org/nightly/std/boxed/struct.Box.html#method.new -[`Box::leak`]: https://doc.rust-lang.org/nightly/std/boxed/struct.Box.html#method.leak - -#### Buffer Construction -So how do we create a `Buffer` instance? The naive approach does not work unfortunately: - -```rust -fn construct_buffer() -> Buffer { - Buffer { - chars: [[Volatile::new(empty_char()); BUFFER_WIDTH]; BUFFER_HEIGHT], - } -} - -fn empty_char() -> ScreenChar { - ScreenChar { - ascii_character: b' ', - color_code: ColorCode::new(Color::Green, Color::Brown), - } -} -``` - -When running `cargo test` the following error occurs: - -``` -error[E0277]: the trait bound `volatile::Volatile: core::marker::Copy` is not satisfied - --> src/vga_buffer.rs:186:21 - | -186 | chars: [[Volatile::new(empty_char); BUFFER_WIDTH]; BUFFER_HEIGHT], - | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `core::marker::Copy` is not implemented for `volatile::Volatile` - | - = note: the `Copy` trait is required because the repeated element will be copied -``` - -The problem is that array construction in Rust requires that the contained type is [`Copy`]. The `ScreenChar` is `Copy`, but the `Volatile` wrapper is not. There is currently no easy way to circumvent this without using [`unsafe`], but fortunately there is the [`array_init`] crate that provides a safe interface for such operations. - -[`Copy`]: https://doc.rust-lang.org/core/marker/trait.Copy.html -[`unsafe`]: https://doc.rust-lang.org/book/second-edition/ch19-01-unsafe-rust.html -[`array_init`]: https://docs.rs/array-init - -To use that crate, we add the following to our `Cargo.toml`: - -```toml -[dev-dependencies] -array-init = "0.0.3" -``` - -Note that we're using the [`dev-dependencies`] table instead of the `dependencies` table, because we only need the crate for `cargo test` and not for a normal build. - -[`dev-dependencies`]: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#development-dependencies - -Now we can fix our `construct_buffer` function: - -```rust -fn construct_buffer() -> Buffer { - use array_init::array_init; - - Buffer { - chars: array_init(|_| array_init(|_| Volatile::new(empty_char()))), - } -} -``` - -See the [documentation of `array_init`][`array_init`] for more information about using that crate. - -### Testing `write_byte` -Now we're finally able to write a first unit test that tests the `write_byte` method: - -```rust -// in vga_buffer.rs - -mod test { - […] - - #[test] - fn write_byte() { - let mut writer = construct_writer(); - writer.write_byte(b'X'); - writer.write_byte(b'Y'); - - for (i, row) in writer.buffer.chars.iter().enumerate() { - for (j, screen_char) in row.iter().enumerate() { - let screen_char = screen_char.read(); - if i == BUFFER_HEIGHT - 1 && j == 0 { - assert_eq!(screen_char.ascii_character, b'X'); - assert_eq!(screen_char.color_code, writer.color_code); - } else if i == BUFFER_HEIGHT - 1 && j == 1 { - assert_eq!(screen_char.ascii_character, b'Y'); - assert_eq!(screen_char.color_code, writer.color_code); - } else { - assert_eq!(screen_char, empty_char()); - } - } - } - } -} -``` - -We construct a `Writer`, write two bytes to it, and then check that the right screen characters were updated. When we run `cargo test`, we see that the test is executed and passes: - -``` -running 1 test -test vga_buffer::test::write_byte ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out -``` - -Try to play around a bit with this function and verify that the test fails if you change something, e.g. if you print a third byte without adjusting the `for` loop. - -(If you're getting an “binary operation `==` cannot be applied to type `vga_buffer::ScreenChar`” error, you need to also derive [`PartialEq`] for `ScreenChar` and `ColorCode`). - -[`PartialEq`]: https://doc.rust-lang.org/nightly/core/cmp/trait.PartialEq.html - -### Testing Strings -Let's add a second unit test to test formatted output and newline behavior: - -```rust -// in src/vga_buffer.rs - -mod test { - […] - - #[test] - fn write_formatted() { - use core::fmt::Write; - - let mut writer = construct_writer(); - writeln!(&mut writer, "a").unwrap(); - writeln!(&mut writer, "b{}", "c").unwrap(); - - for (i, row) in writer.buffer.chars.iter().enumerate() { - for (j, screen_char) in row.iter().enumerate() { - let screen_char = screen_char.read(); - if i == BUFFER_HEIGHT - 3 && j == 0 { - assert_eq!(screen_char.ascii_character, b'a'); - assert_eq!(screen_char.color_code, writer.color_code); - } else if i == BUFFER_HEIGHT - 2 && j == 0 { - assert_eq!(screen_char.ascii_character, b'b'); - assert_eq!(screen_char.color_code, writer.color_code); - } else if i == BUFFER_HEIGHT - 2 && j == 1 { - assert_eq!(screen_char.ascii_character, b'c'); - assert_eq!(screen_char.color_code, writer.color_code); - } else if i >= BUFFER_HEIGHT - 2 { - assert_eq!(screen_char.ascii_character, b' '); - assert_eq!(screen_char.color_code, writer.color_code); - } else { - assert_eq!(screen_char, empty_char()); - } - } - } - } -} -``` - -In this test we're using the [`writeln!`] macro to print strings with newlines to the buffer. Most of the for loop is similar to the `write_byte` test and only verifies if the written characters are at the expected place. The new `if i >= BUFFER_HEIGHT - 2` case verifies that the empty lines that are shifted in on a newline have the `writer.color_code`, which is different from the initial color. - -[`writeln!`]: https://doc.rust-lang.org/nightly/core/macro.writeln.html - -### More Tests -We only present two basic tests here as an example, but of course many more tests are possible. For example a test that changes the writer color in between writes. Or a test that checks that the top line is correctly shifted off the screen on a newline. Or a test that checks that non-ASCII characters are handled correctly. - -## Summary +### Summary Unit testing is a very useful technique to ensure that certain components have a desired behavior. Even if they cannot show the absence of bugs, they're still an useful tool for finding them and especially for avoiding regressions. This post explained how to set up unit testing in a Rust kernel. We now have a functioning test framework and can easily add tests by adding functions with a `#[test]` attribute. To run them, a short `cargo test` suffices. We also added a few basic tests for our VGA buffer as an example how unit tests could look like. @@ -1188,547 +976,7 @@ We also learned a bit about conditional compilation, Rust's [lint system], how t [lint system]: #silencing-the-warnings [initialize arrays with non-Copy types]: #buffer-construction -## What's next? -We now have a working unit testing framework, which gives us the ability to test individual components. However, unit tests have the disadvantage that they run on the host machine and are thus unable to test how components interact with platform specific parts. For example, we can't test the `println!` macro with an unit test because it wants to write at the VGA text buffer at address `0xb8000`, which only exists in the bare metal environment. - -The next post will close this gap by creating a basic _integration test_ framework, which runs the tests in QEMU and thus has access to platform specific components. This will allow us to test the full system, for example that our kernel boots correctly or that no deadlock occurs on nested `println!` invocations. - - - - -# Integration Tests - -## Overview - -In the previous post we added support for unit tests. The goal of unit tests is to test small components in isolation to ensure that each of them works as intended. The tests are run on the host machine and thus shouldn't rely on architecture specific functionality. - -To test the interaction of the components, both with each other and the system environment, we can write _integration tests_. Compared to unit tests, ìntegration tests are more complex, because they need to run in a realistic environment. What this means depends on the application type. For example, for webserver applications it often means to set up a database instance. For an operating system kernel like ours, it means that we run the tests on the target hardware without an underlying operating system. - -Running on the target architecture allows us to test all hardware specific code such as the VGA buffer or the effects of [page table] modifications. It also allows us to verify that our kernel boots without problems and that no [CPU exception] occurs. - -[page table]: https://en.wikipedia.org/wiki/Page_table -[CPU exception]: https://wiki.osdev.org/Exceptions - -In this post we will implement a very basic test framework that runs integration tests inside instances of the [QEMU] virtual machine. It is not as realistic as running them on real hardware, but it is much simpler and should be sufficient as long as we only use standard hardware that is well supported in QEMU. - -[QEMU]: https://www.qemu.org/ - -## The Serial Port - -The naive way of doing an integration test would be to add some assertions in the code, launch QEMU, and manually check if a panic occured or not. This is very cumbersome and not practical if we have hundreds of integration tests. So we want an automated solution that runs all tests and fails if not all of them pass. - -Such an automated test framework needs to know whether a test succeeded or failed. It can't look at the screen output of QEMU, so we need a different way of retrieving the test results on the host system. A simple way to achieve this is by using the [serial port], an old interface standard which is no longer found in modern computers. It is easy to program and QEMU can redirect the bytes sent over serial to the host's standard output or a file. - -[serial port]: https://en.wikipedia.org/wiki/Serial_port - -The chips implementing a serial interface are called [UARTs]. There are [lots of UART models] on x86, but fortunately the only differences between them are some advanced features we don't need. The common UARTs today are all compatible to the [16550 UART], so we will use that model for our testing framework. - -[UARTs]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter -[lots of UART models]: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter#UART_models -[16550 UART]: https://en.wikipedia.org/wiki/16550_UART - -### Port I/O -There are two different approaches for communicating between the CPU and peripheral hardware on x86, **memory-mapped I/O** and **port-mapped I/O**. We already used memory-mapped I/O for accessing the [VGA text buffer] through the memory address `0xb8000`. This address is not mapped to RAM, but to some memory on the GPU. - -[VGA text buffer]: ./second-edition/posts/03-vga-text-buffer/index.md - -In contrast, port-mapped I/O uses a separate I/O bus for communication. Each connected peripheral has one or more port numbers. To communicate with such an I/O port there are special CPU instructions called `in` and `out`, which take a port number and a data byte (there are also variations of these commands that allow sending an `u16` or `u32`). - -The UART uses port-mapped I/O. Fortunately there are already several crates that provide abstractions for I/O ports and even UARTs, so we don't need to invoke the `in` and `out` assembly instructions manually. - -### Implementation - -We will use the [`uart_16550`] crate to initialize the UART and send data over the serial port. To add it as a dependency, we update our `Cargo.toml` and `main.rs`: - -[`uart_16550`]: https://docs.rs/uart_16550 - -```toml -# in Cargo.toml - -[dependencies] -uart_16550 = "0.1.0" -``` - -The `uart_16550` crate contains a `SerialPort` struct that represents the UART registers, but we still need to construct an instance of it ourselves. For that we create a new `serial` module with the following content: - -```rust -// in src/main.rs - -mod serial; -``` - -```rust -// in src/serial.rs - -use uart_16550::SerialPort; -use spin::Mutex; -use lazy_static::lazy_static; - -lazy_static! { - pub static ref SERIAL1: Mutex = { - let mut serial_port = SerialPort::new(0x3F8); - serial_port.init(); - Mutex::new(serial_port) - }; -} -``` - -Like with the [VGA text buffer][vga lazy-static], we use `lazy_static` and a spinlock to create a `static`. However, this time we use `lazy_static` to ensure that the `init` method is called before first use. We're using the port address `0x3F8`, which is the standard port number for the first serial interface. - -[vga lazy-static]: ./second-edition/posts/03-vga-text-buffer/index.md#lazy-statics - -To make the serial port easily usable, we add `serial_print!` and `serial_println!` macros: - -```rust -#[doc(hidden)] -pub fn _print(args: ::core::fmt::Arguments) { - use core::fmt::Write; - SERIAL1.lock().write_fmt(args).expect("Printing to serial failed"); -} - -/// Prints to the host through the serial interface. -#[macro_export] -macro_rules! serial_print { - ($($arg:tt)*) => { - $crate::serial::_print(format_args!($($arg)*)); - }; -} - -/// Prints to the host through the serial interface, appending a newline. -#[macro_export] -macro_rules! serial_println { - () => ($crate::serial_print!("\n")); - ($fmt:expr) => ($crate::serial_print!(concat!($fmt, "\n"))); - ($fmt:expr, $($arg:tt)*) => ($crate::serial_print!( - concat!($fmt, "\n"), $($arg)*)); -} -``` - -The `SerialPort` type already implements the [`fmt::Write`] trait, so we don't need to provide an implementation. - -[`fmt::Write`]: https://doc.rust-lang.org/nightly/core/fmt/trait.Write.html - -Now we can print to the serial interface in our `main.rs`: - -```rust -// in src/main.rs - -mod serial; - -#[cfg(not(test))] -#[no_mangle] -pub extern "C" fn _start() -> ! { - println!("Hello World{}", "!"); // prints to vga buffer - serial_println!("Hello Host{}", "!"); - - loop {} -} -``` - -Note that the `serial_println` macro lives directly under the root namespace because we used the `#[macro_export]` attribute, so importing it through `use crate::serial::serial_println` will not work. - -### QEMU Arguments - -To see the serial output in QEMU, we can use the `-serial` argument to redirect the output to stdout: - -``` -> qemu-system-x86_64 \ - -drive format=raw,file=target/x86_64-blog_os/debug/bootimage-blog_os.bin \ - -serial mon:stdio -warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5] -Hello Host! -``` - -If you chose a different name than `blog_os`, you need to update the paths of course. Note that you can no longer exit QEMU through `Ctrl+c`. As an alternative you can use `Ctrl+a` and then `x`. - -As an alternative to this long command, we can pass the argument to `bootimage run`, with an additional `--` to separate the build arguments (passed to cargo) from the run arguments (passed to QEMU). - -``` -bootimage run -- -serial mon:stdio -``` - -Instead of standard output, QEMU supports [many more target devices][QEMU -serial]. For redirecting the output to a file, the argument is: - -[QEMU -serial]: https://qemu.weilnetz.de/doc/qemu-doc.html#Debug_002fExpert-options - -``` --serial file:output-file.txt -``` - -## Shutting Down QEMU - -Right now we have an endless loop at the end of our `_start` function and need to close QEMU manually. This does not work for automated tests. We could try to kill QEMU automatically from the host, for example after some special output was sent over serial, but this would be a bit hacky and difficult to get right. The cleaner solution would be to implement a way to shutdown our OS. Unfortunatly this is relatively complex, because it requires implementing support for either the [APM] or [ACPI] power management standard. - -[APM]: https://wiki.osdev.org/APM -[ACPI]: https://wiki.osdev.org/ACPI - -Luckily, there is an escape hatch: QEMU supports a special `isa-debug-exit` device, which provides an easy way to exit QEMU from the guest system. To enable it, we add the following argument to our QEMU command: - -``` --device isa-debug-exit,iobase=0xf4,iosize=0x04 -``` - -The `iobase` specifies on which port address the device should live (`0xf4` is a [generally unused][list of x86 I/O ports] port on the x86's IO bus) and the `iosize` specifies the port size (`0x04` means four bytes). Now the guest can write a value to the `0xf4` port and QEMU will exit with [exit status] `(passed_value << 1) | 1`. - -[list of x86 I/O ports]: https://wiki.osdev.org/I/O_Ports#The_list -[exit status]: https://en.wikipedia.org/wiki/Exit_status - -To write to the I/O port, we use the [`x86_64`] crate: - -[`x86_64`]: https://docs.rs/x86_64/0.5.2/x86_64/ - -```toml -# in Cargo.toml - -[dependencies] -x86_64 = "0.5.2" -``` - -```rust -// in src/main.rs - -pub unsafe fn exit_qemu() { - use x86_64::instructions::port::Port; - - let mut port = Port::::new(0xf4); - port.write(0); -} -``` - -We mark the function as `unsafe` because it relies on the fact that a special QEMU device is attached to the I/O port with address `0xf4`. For the port type we choose `u32` because the `iosize` is 4 bytes. As value we write a zero, which causes QEMU to exit with exit status `(0 << 1) | 1 = 1`. - -Note that we could also use the exit status instead of the serial interface for sending the test results, for example `1` for success and `2` for failure. However, this wouldn't allow us to send panic messages like the serial interface does and would also prevent us from replacing `exit_qemu` with a proper shutdown someday. Therefore we continue to use the serial interface and just always write a `0` to the port. - -We can now test the QEMU shutdown by calling `exit_qemu` from our `_start` function: - -```rust -#[cfg(not(test))] -#[no_mangle] -pub extern "C" fn _start() -> ! { - println!("Hello World{}", "!"); // prints to vga buffer - serial_println!("Hello Host{}", "!"); - - unsafe { exit_qemu(); } - - loop {} -} -``` - -You should see that QEMU immediately closes after booting when executing: - -``` -bootimage run -- -serial mon:stdio -device isa-debug-exit,iobase=0xf4,iosize=0x04 -``` - -## Hiding QEMU - -We are now able to launch a QEMU instance that writes its output to the serial port and automatically exits itself when it's done. So we no longer need the VGA buffer output or the graphical representation that still pops up. We can disable it by passing the `-display none` parameter to QEMU. The full command looks like this: - -``` -qemu-system-x86_64 \ - -drive format=raw,file=target/x86_64-blog_os/debug/bootimage-blog_os.bin \ - -serial mon:stdio \ - -device isa-debug-exit,iobase=0xf4,iosize=0x04 \ - -display none -``` - -Or, with `bootimage run`: - -``` -bootimage run -- \ - -serial mon:stdio \ - -device isa-debug-exit,iobase=0xf4,iosize=0x04 \ - -display none -``` - -Now QEMU runs completely in the background and no window is opened anymore. This is not only less annoying, but also allows our test framework to run in environments without a graphical user interface, such as [Travis CI]. - -[Travis CI]: https://travis-ci.com/ - -## Test Organization - -Right now we're doing the serial output and the QEMU exit from the `_start` function in our `main.rs` and can no longer run our kernel in a normal way. We could try to fix this by adding an `integration-test` [cargo feature] and using [conditional compilation]: - -[cargo feature]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-features-section -[conditional compilation]: https://doc.rust-lang.org/reference/attributes.html#conditional-compilation - -```toml -# in Cargo.toml - -[features] -integration-test = [] -``` - -```rust -// in src/main.rs - -#[cfg(not(feature = "integration-test"))] // new -#[cfg(not(test))] -#[no_mangle] -pub extern "C" fn _start() -> ! { - println!("Hello World{}", "!"); // prints to vga buffer - - // normal execution - - loop {} -} - -#[cfg(feature = "integration-test")] // new -#[cfg(not(test))] -#[no_mangle] -pub extern "C" fn _start() -> ! { - serial_println!("Hello Host{}", "!"); - - run_test_1(); - run_test_2(); - // run more tests - - unsafe { exit_qemu(); } - - loop {} -} -``` - -However, this approach has a big problem: All tests run in the same kernel instance, which means that they can influence each other. For example, if `run_test_1` misconfigures the system by loading an invalid [page table], it can cause `run_test_2` to fail. This isn't something that we want because it makes it very difficult to find the actual cause of an error. - -[page table]: https://en.wikipedia.org/wiki/Page_table - -Instead, we want our test instances to be as independent as possible. If a test wants to destroy most of the system configuration to ensure that some property still holds in catastrophic situations, it should be able to do so without needing to restore a correct system state afterwards. This means that we need to launch a separate QEMU instance for each test. - -With the above conditional compilation we only have two modes: Run the kernel normally or execute _all_ integration tests. To run each test in isolation we would need a separate cargo feature for each test with that approach, which would result in very complex conditional compilation bounds and confusing code. - -A better solution is to create an additional executable for each test. - -### Additional Test Executables - -Cargo allows to add [additional executables] to a project by putting them inside `src/bin`. We can use that feature to create a separate executable for each integration test. For example, a `test-something` executable could be added like this: - -[additional executables]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-project-layout - -```rust -// src/bin/test-something.rs - -#![cfg_attr(not(test), no_std)] -#![cfg_attr(not(test), no_main)] -#![cfg_attr(test, allow(unused_imports))] - -use core::panic::PanicInfo; - -#[cfg(not(test))] -#[no_mangle] -pub extern "C" fn _start() -> ! { - // run tests - loop {} -} - -#[cfg(not(test))] -#[panic_handler] -fn panic(_info: &PanicInfo) -> ! { - loop {} -} -``` - -By providing a new implementation for `_start` we can create a minimal test case that only tests one specific thing and is independent of the rest. For example, if we don't print anything to the VGA buffer, the test still succeeds even if the `vga_buffer` module is broken. - -We can now run this executable in QEMU by passing a `--bin` argument to `bootimage`: - -``` -bootimage run --bin test-something -``` - -It should build the `test-something.rs` executable instead of `main.rs` and launch an empty QEMU window (since we don't print anything). So this approach allows us to create completely independent executables without cargo features or conditional compilation, and without cluttering our `main.rs`. - -However, there is a problem: This is a completely separate executable, which means that we can't access any functions from our `main.rs`, including `serial_println` and `exit_qemu`. Duplicating the code would work, but we would also need to copy everything we want to test. This would mean that we no longer test the original function but only a possibly outdated copy. - -Fortunately there is a way to share most of the code between our `main.rs` and the testing binaries: We move most of the code from our `main.rs` to a library that we can include from all executables. - -### Split Off A Library - -Cargo supports hybrid projects that are both a library and a binary. We only need to create a `src/lib.rs` file and split the contents of our `main.rs` in the following way: - -```rust -// src/lib.rs - -#![cfg_attr(not(test), no_std)] // don't link the Rust standard library - -// NEW: We need to add `pub` here to make them accessible from the outside -pub mod vga_buffer; -pub mod serial; - -pub unsafe fn exit_qemu() { - use x86_64::instructions::port::Port; - - let mut port = Port::::new(0xf4); - port.write(0); -} -``` - -```rust -// src/main.rs - -#![cfg_attr(not(test), no_std)] -#![cfg_attr(not(test), no_main)] -#![cfg_attr(test, allow(unused_imports))] - -use core::panic::PanicInfo; -use blog_os::println; - -/// This function is the entry point, since the linker looks for a function -/// named `_start` by default. -#[cfg(not(test))] -#[no_mangle] // don't mangle the name of this function -pub extern "C" fn _start() -> ! { - println!("Hello World{}", "!"); - - loop {} -} - -/// This function is called on panic. -#[cfg(not(test))] -#[panic_handler] -fn panic(info: &PanicInfo) -> ! { - println!("{}", info); - loop {} -} -``` - -So we move everything except `_start` and `panic` to `lib.rs` and make the `vga_buffer` and `serial` modules public. Everything should work exactly as before, including `bootimage run` and `cargo test`. To run tests only for the library part of our crate and avoid the additional output we can execute `cargo test --lib`. - -### Test Basic Boot - -We are finally able to create our first integration test executable. We start simple and only test that the basic boot sequence works and the `_start` function is called: - -```rust -// in src/bin/test-basic-boot.rs - -#![cfg_attr(not(test), no_std)] -#![cfg_attr(not(test), no_main)] // disable all Rust-level entry points -#![cfg_attr(test, allow(unused_imports))] - -use core::panic::PanicInfo; -use blog_os::{exit_qemu, serial_println}; - -/// This function is the entry point, since the linker looks for a function -/// named `_start` by default. -#[cfg(not(test))] -#[no_mangle] // don't mangle the name of this function -pub extern "C" fn _start() -> ! { - serial_println!("ok"); - - unsafe { exit_qemu(); } - loop {} -} - - -/// This function is called on panic. -#[cfg(not(test))] -#[panic_handler] -fn panic(info: &PanicInfo) -> ! { - serial_println!("failed"); - - serial_println!("{}", info); - - unsafe { exit_qemu(); } - loop {} -} -``` - -We don't do something special here, we just print `ok` if `_start` is called and `failed` with the panic message when a panic occurs. Let's try it: - -``` -> bootimage run --bin test-basic-boot -- \ - -serial mon:stdio -display none \ - -device isa-debug-exit,iobase=0xf4,iosize=0x04 -Building kernel - Compiling blog_os v0.2.0 (file:///…/blog_os) - Finished dev [unoptimized + debuginfo] target(s) in 0.19s - Updating registry `https://github.com/rust-lang/crates.io-index` -Creating disk image at target/x86_64-blog_os/debug/bootimage-test-basic-boot.bin -warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5] -ok -``` - -We got our `ok`, so it worked! Try inserting a `panic!()` before the `ok` printing, you should see output like this: - -``` -failed -panicked at 'explicit panic', src/bin/test-basic-boot.rs:19:5 -``` - -### Test Panic - -To test that our panic handler is really invoked on a panic, we create a `test-panic` test: - -```rust -// in src/bin/test-panic.rs - -#![cfg_attr(not(test), no_std)] -#![cfg_attr(not(test), no_main)] -#![cfg_attr(test, allow(unused_imports))] - -use core::panic::PanicInfo; -use blog_os::{exit_qemu, serial_println}; - -#[cfg(not(test))] -#[no_mangle] -pub extern "C" fn _start() -> ! { - panic!(); -} - -#[cfg(not(test))] -#[panic_handler] -fn panic(_info: &PanicInfo) -> ! { - serial_println!("ok"); - - unsafe { exit_qemu(); } - loop {} -} -``` - -This executable is almost identical to `test-basic-boot`, the only difference is that we print `ok` from our panic handler and invoke an explicit `panic()` in our `_start` function. - -## A Test Runner - -The final step is to create a test runner, a program that executes all integration tests and checks their results. The basic steps that it should do are: - -- Look for integration tests in the current project, maybe by some convention (e.g. executables starting with `test-`). -- Run all integration tests and interpret their results. - - Use a timeout to ensure that an endless loop does not block the test runner forever. -- Report the test results to the user and set a successful or failing exit status. - -Such a test runner is useful to many projects, so we decided to add one to the `bootimage` tool. - -### Bootimage Test - -The test runner of the `bootimage` tool can be invoked via `bootimage test`. It uses the following conventions: - -- All executables starting with `test-` are treated as integration tests. -- Tests must print either `ok` or `failed` over the serial port. When printing `failed` they can print additional information such as a panic message (in the next lines). -- Tests are run with a timeout of 1 minute. If the test has not completed in time, it is reported as "timed out". - -The `test-basic-boot` and `test-panic` tests we created above begin with `test-` and follow the `ok`/`failed` conventions, so they should work with `bootimage test`: - -``` -> bootimage test -test-panic - Finished dev [unoptimized + debuginfo] target(s) in 0.01s -Ok - -test-basic-boot - Finished dev [unoptimized + debuginfo] target(s) in 0.01s -Ok - -test-something - Finished dev [unoptimized + debuginfo] target(s) in 0.01s -Timed Out - -The following tests failed: - test-something: TimedOut -``` - -We see that our `test-panic` and `test-basic-boot` succeeded and that the `test-something` test timed out after one minute. We no longer need `test-something`, so we delete it (if you haven't done already). Now `bootimage test` should execute successfully. - -## Summary +### Summary In this post we learned about the serial port and port-mapped I/O and saw how to configure QEMU to print serial output to the command line. We also learned a trick how to exit QEMU without needing to implement a proper shutdown. @@ -1736,5 +984,4 @@ We then split our crate into a library and binary part in order to create additi We now have a working integration test framework and can finally start to implement functionality in our kernel. We will continue to use the test framework over the next posts to test new components we add. -## What's next? -In the next post, we will explore _CPU exceptions_. These exceptions are thrown by the CPU when something illegal happens, such as a division by zero or an access to an unmapped memory page (a so-called “page fault”). Being able to catch and examine these exceptions is very important for debugging future errors. Exception handling is also very similar to the handling of hardware interrupts, which is required for keyboard support. + From 6f4383d004f75e10d334b273319550e426770be2 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 25 Apr 2019 14:52:41 +0200 Subject: [PATCH 15/42] Write requirements section --- blog/content/second-edition/posts/04-testing/index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/blog/content/second-edition/posts/04-testing/index.md b/blog/content/second-edition/posts/04-testing/index.md index 285b736c..5cedcda9 100644 --- a/blog/content/second-edition/posts/04-testing/index.md +++ b/blog/content/second-edition/posts/04-testing/index.md @@ -20,9 +20,11 @@ This blog is openly developed on [GitHub]. If you have any problems or questions ## Requirements -This post assumes that you have a `.cargo/config` file with a default target... TODO +This post assumes that you have followed the latest version of the ["_A Minimal Rust Kernel_"] post. It requires that you have a `.cargo/config` file that [sets a default target] and [defines a runner executable]. -Earlier posts since XX, bootimage runner +["_A Minimal Rust Kernel_"]: ./second-edition/posts/02-minimal-rust-kernel/index.md +[sets a default target]: ./second-edition/posts/02-minimal-rust-kernel/index.md#set-a-default-target +[defines a runner executable]: ./second-edition/posts/02-minimal-rust-kernel/index.md#using-cargo-run ## Testing in Rust From 38b904c6fb914b7645f0290594356b6813452007 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 25 Apr 2019 16:14:23 +0200 Subject: [PATCH 16/42] Update CPU Exceptions post for new test system --- .../posts/06-cpu-exceptions/index.md | 78 +++++++++++-------- 1 file changed, 45 insertions(+), 33 deletions(-) diff --git a/blog/content/second-edition/posts/06-cpu-exceptions/index.md b/blog/content/second-edition/posts/06-cpu-exceptions/index.md index 33056e67..c83091df 100644 --- a/blog/content/second-edition/posts/06-cpu-exceptions/index.md +++ b/blog/content/second-edition/posts/06-cpu-exceptions/index.md @@ -371,8 +371,21 @@ pub fn init_idt() { Note how this solution requires no `unsafe` blocks. The `lazy_static!` macro does use `unsafe` behind the scenes, but it is abstracted away in a safe interface. -### Testing it -Now we should be able to handle breakpoint exceptions! Let's try it in our `_start` function: +### Running it + +The last step for making exceptions work in our kernel it to call the `init_idt` function from our `main.rs`. Instead of calling it directly, we introduce a general `init` function in our `lib.rs`: + +```rust +// in src/lib.rs + +pub fn init() { + interrupts::init_idt(); +} +``` + +With this function we now have a central place for initialization routines that can be shared between the different `_start` functions in our `main.rs`, `lib.rs`, and integration tests. + +Now we can update the `_start` function of our `main.rs` to call `init` and then trigger a breakpoint exception: ```rust // in src/main.rs @@ -382,10 +395,10 @@ Now we should be able to handle breakpoint exceptions! Let's try it in our `_sta pub extern "C" fn _start() -> ! { println!("Hello World{}", "!"); - blog_os::interrupts::init_idt(); + blog_os::init(); // new // invoke a breakpoint exception - x86_64::instructions::interrupts::int3(); + x86_64::instructions::interrupts::int3(); // new println!("It did not crash!"); loop {} @@ -402,47 +415,46 @@ We see that the interrupt stack frame tells us the instruction and stack pointer ### Adding a Test -Let's create an integration test that ensures that the above continues to work. For that we create a file named `test-exception-breakpoint.rs`: +Let's create a test that ensures that the above continues to work. First, we update the `_start` function to also call `init`: ```rust -// in src/bin/test-exception-breakpoint.rs +// in src/lib.rs -#![no_std] -#![cfg_attr(not(test), no_main)] -#![cfg_attr(test, allow(dead_code, unused_macros, unused_imports))] - -use core::panic::PanicInfo; -use blog_os::{exit_qemu, serial_println}; - -#[cfg(not(test))] +#[cfg(test)] #[no_mangle] pub extern "C" fn _start() -> ! { - blog_os::interrupts::init_idt(); + init(); // new - x86_64::instructions::interrupts::int3(); + test_main(); - serial_println!("ok"); - - unsafe { exit_qemu(); } - loop {} -} - - -#[cfg(not(test))] -#[panic_handler] -fn panic(info: &PanicInfo) -> ! { - serial_println!("failed"); - - serial_println!("{}", info); - - unsafe { exit_qemu(); } loop {} } ``` -It is similar to our `main.rs`, but instead of printing "It did not crash!" to the VGA buffer, it prints "ok" to the serial output and calls `exit_qemu`. This allows the `bootimage` tool to detect that our code successfully continued after invoking the `int3` instruction. If our `panic_handler` is called, we instead print `failed` to signalize the failure to `bootimage`. +Remember, this `_start` function is used when running `cargo xtest --lib`, since Rust's tests the `lib.rs` completely independent of the `main.rs`. We need to call `init` here to set up an IDT before running the tests. -You can try this new test by running `bootimage test`. +Now we can create a `test_breakpoint_exception` test: + +```rust +// in src/interrupts.rs + +#[cfg(test)] +use crate::{serial_print, serial_println}; + +#[test_case] +fn test_breakpoint_exception() { + serial_print!("test_breakpoint_exception..."); + // invoke a breakpoint exception + x86_64::instructions::interrupts::int3(); + serial_println!("[ok]"); +} +``` + +Apart from printing status messages through the [serial port], the test invokes the `int3` function to trigger a breakpoint exception. By checking that the execution continues afterwards, we verify that our breakpoint handler is working correctly. + +[serial port]: ./second-edition/posts/04-testing/index.md#serial-port + +You can try this new test by running `cargo xtest --lib`. You should see `test_breakpoint_exception...[ok]` in the output. ### Fixing `cargo test` on Windows From 97bd58720e27d59b18e1629b5d78be85a57096ec Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 25 Apr 2019 16:38:56 +0200 Subject: [PATCH 17/42] Update Double Faults post for new test system --- blog/content/second-edition/posts/07-double-faults/index.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/blog/content/second-edition/posts/07-double-faults/index.md b/blog/content/second-edition/posts/07-double-faults/index.md index f41ce34b..589fb0a6 100644 --- a/blog/content/second-edition/posts/07-double-faults/index.md +++ b/blog/content/second-edition/posts/07-double-faults/index.md @@ -410,9 +410,10 @@ That's it! Now the CPU should switch to the double fault stack whenever a double From now on we should never see a triple fault again! -To ensure that we don't accidentally break the above, we should add a integration test for this. We don't show the code here for space reasons, but you can find it [on Github][stack overflow test]. The idea is to do a `serial_println!("ok");` from the double fault handler to ensure that it is called. The rest of the file is is very similar to our `main.rs`. +To ensure that we don't accidentally break the above, we should add a integration test for this. We don't show the code here for space reasons, but you can find it [on Github][stack overflow test]. The idea is to do a `exit_qemu(QemuExitCode::Success)` from the double fault handler to ensure that it is called. Like our `panic_handler` test, the test requires [disabling the test harness]. -[stack overflow test]: https://github.com/phil-opp/blog_os/blob/post-07/src/bin/test-exception-double-fault-stack-overflow.rs +[stack overflow test]: https://github.com/phil-opp/blog_os/blob/post-07/src/tests/stack_overflow.rs +[disabling the test harness]: ./second-edition/posts/04-testing/index.md#no-harness ## Summary In this post we learned what a double fault is and under which conditions it occurs. We added a basic double fault handler that prints an error message and added an integration test for it. From 9437656d5cab1e715a0ff91f9ba4d02c2227dff2 Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 25 Apr 2019 16:41:01 +0200 Subject: [PATCH 18/42] Update I/O Port links in hardware interrupts post --- .../second-edition/posts/08-hardware-interrupts/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blog/content/second-edition/posts/08-hardware-interrupts/index.md b/blog/content/second-edition/posts/08-hardware-interrupts/index.md index 55739f2a..d8c8388b 100644 --- a/blog/content/second-edition/posts/08-hardware-interrupts/index.md +++ b/blog/content/second-edition/posts/08-hardware-interrupts/index.md @@ -66,7 +66,7 @@ This graphic shows the typical assignment of interrupt lines. We see that most o Each controller can be configured through two [I/O ports], one “command” port and one “data” port. For the primary controller these ports are `0x20` (command) and `0x21` (data). For the secondary controller they are `0xa0` (command) and `0xa1` (data). For more information on how the PICs can be configured see the [article on osdev.org]. -[I/O ports]: ./second-edition/posts/05-integration-tests/index.md#port-i-o +[I/O ports]: ./second-edition/posts/04-testing/index.md#i-o-ports [article on osdev.org]: https://wiki.osdev.org/8259_PIC ### Implementation @@ -497,7 +497,7 @@ We now see that a `k` appears on the screen when we press a key. However, this o To find out _which_ key was pressed, we need to query the keyboard controller. We do this by reading from the data port of the PS/2 controller, which is the [I/O port] with number `0x60`: -[I/O port]: ./second-edition/posts/05-integration-tests/index.md#port-i-o +[I/O port]: ./second-edition/posts/04-testing/index.md#i-o-ports ```rust // in src/interrupts.rs From 701542b40fe025caf79573502b91e7331738c19d Mon Sep 17 00:00:00 2001 From: Philipp Oppermann Date: Thu, 25 Apr 2019 16:47:05 +0200 Subject: [PATCH 19/42] Move Testing post to bare bones category and update post numbers --- blog/static/css/main.css | 8 -------- blog/templates/second-edition/index.html | 9 ++------- 2 files changed, 2 insertions(+), 15 deletions(-) diff --git a/blog/static/css/main.css b/blog/static/css/main.css index a303d19c..c13ac3a6 100644 --- a/blog/static/css/main.css +++ b/blog/static/css/main.css @@ -67,10 +67,6 @@ main img { border: 2px solid #66f; } -.posts.testing { - border: 2px solid #0a0; -} - .posts.memory-management { border: 2px solid #fc0 } @@ -106,10 +102,6 @@ main img { color: #55d; } -.post-category.testing { - color: #090; -} - .post-category.memory-management { color: #990; } diff --git a/blog/templates/second-edition/index.html b/blog/templates/second-edition/index.html index ae0a9cc8..d25ca71f 100644 --- a/blog/templates/second-edition/index.html +++ b/blog/templates/second-edition/index.html @@ -26,25 +26,20 @@ {{ macros::post_link(page=posts.0) }} {{ macros::post_link(page=posts.1) }} {{ macros::post_link(page=posts.2) }} - - -
Testing
-
{{ macros::post_link(page=posts.3) }} - {{ macros::post_link(page=posts.4) }}
Exceptions
+ {{ macros::post_link(page=posts.4) }} {{ macros::post_link(page=posts.5) }} {{ macros::post_link(page=posts.6) }} - {{ macros::post_link(page=posts.7) }}
Memory Management
+ {{ macros::post_link(page=posts.7) }} {{ macros::post_link(page=posts.8) }} - {{ macros::post_link(page=posts.9) }}