Compare commits

...

29 Commits

Author SHA1 Message Date
seewishnew
e8ea607b48 Merge 128d456923 into 4b023bb432 2024-01-05 10:05:36 -08:00
Philipp Oppermann
4b023bb432 Merge pull request #1262 from acyanbird/main
fix testing and paging introduction chapter errors in zh-CN translation
2024-01-05 18:00:30 +01:00
acyanbird
b1b35833d6 fix zh-CN paging introduction 2024-01-05 16:27:14 +00:00
acyanbird
34120a0409 fix zh-CN testing code err 2024-01-05 16:16:08 +00:00
Philipp Oppermann
6367e931e5 Merge pull request #1082 from TornaxO7/post_04_pub_fn_test_runner
Adding fix to make the test_runner functions pub
2023-12-28 17:20:22 +01:00
Philipp Oppermann
02fe09d56f Merge pull request #1254 from swnakamura/change-translator-name
Change one of translator's github ID
2023-12-08 19:46:42 +01:00
woodyZootopia
f4ab296b8b Change one of translator's github id
woodyZootopia changed their github id to swnakamura. This commit changes
the translators section accordingly
2023-12-02 15:04:01 +09:00
Philipp Oppermann
db4068826b Merge pull request #1167 from swnakamura/translate-11allocatordesign-ja
Translate post 11 "allocator design" into Japanese
2023-11-27 14:40:47 +01:00
woodyZootopia
9b1791a48d Refine the translation of post 11 2023-11-26 13:22:08 +09:00
Philipp Oppermann
61d074cc6c Merge pull request #1253 from keith666666/main
Fix: broken link of generator
2023-11-25 18:16:44 +01:00
Your Name
417c22556d fix: broken link of generator 2023-11-25 18:15:03 +08:00
Philipp Oppermann
5b4d04e337 Fix datetime comparison error in before_build.py 2023-10-17 09:57:11 +02:00
Philipp Oppermann
684ef64767 Merge pull request #1242 from phil-opp/improve-comment-link
Add `in:title` qualifier for discussion links
2023-09-14 20:30:36 +02:00
Philipp Oppermann
87d0ce5fa2 Add in:title qualifier for discussion links
Reduces the number of false positives in search.
2023-09-14 20:23:53 +02:00
Philipp Oppermann
5b67cb05ff Merge pull request #1237 from xzmeng/issue-1199
minimal-rust-kernel: fix missing .toml in zh-CN translation
2023-09-06 20:04:10 +02:00
Meng Xiangzhuo
81b7829657 minimal-rust-kernel: fix missing .toml in zh-CN translation 2023-08-27 03:48:25 +08:00
Philipp Oppermann
1ddeb129ac Merge pull request #1235 from Connortsui20/post-02-beginner-cargo-tips
Grammar fix
2023-08-21 09:26:13 +02:00
woodyZootopia
53d181d57b Rebase to the latest main 2023-08-21 13:34:57 +09:00
woodyZootopia
b634a24f4b Finish translation 2023-08-21 13:33:14 +09:00
woodyZootopia
4ef59648be Add Japanese article 2023-08-21 13:33:14 +09:00
Connortsui20
73628c1d05 Grammar fix 2023-08-18 08:12:39 -04:00
Philipp Oppermann
2e3230eca2 Merge pull request #1234 from Connortsui20/post-02-beginner-cargo-tips
Update post 2 with beginner friendly cargo tips
2023-08-17 16:30:54 +02:00
Connortsui20
63dc179cc7 shorter .cargo explanation 2023-08-17 09:13:43 -04:00
Connortsui20
211544af00 Better clarification 2023-08-17 08:42:54 -04:00
Connortsui20
1ff26bb4b6 Update post 2 with beginner friendly cargo tips
As a relatively new person to Rust, I confused the `.cargo/config.toml` with the global cargo config in my home directory (`~/.cargo/config.toml). While this is pretty obviously wrong in hindsight, since I've never used the `[unstable]` options before, I didn't realize my mistake until this next thing that tripped me up.

For `cargo install bootimage`, I think it is reasonable to tell the reader to go into a different directory to execute the command, since it might be the case that the reader has never dealt with different targets before and would have no idea that running `cargo install` for the new target that they just made in a json would be wrong (at least this was me).

This could be worded differently than the changes I made, but I think that the addition of these could only benefit a confused reader.
2023-08-16 18:57:37 -04:00
Vishnu C
128d456923 Minor corrections 2022-12-28 01:48:13 -08:00
Vishnu C
0652ed79c3 Minor edits and formatting corrections 2022-12-28 01:40:54 -08:00
Vishnu C
7500cac640 Adds code and documentation to rectify potential leaky headers in linked list allocator 2022-12-28 01:23:09 -08:00
TornaxO7
7ce356f99d Adding fix to make the test_runner functions pub 2022-02-23 02:26:05 +01:00
20 changed files with 1376 additions and 38 deletions

View File

@@ -8,7 +8,7 @@ from github import Github
g = Github() g = Github()
one_month_ago = datetime.datetime.now() - datetime.timedelta(days=32) one_month_ago = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(days=32)
def filter_date(issue): def filter_date(issue):
return issue.closed_at > one_month_ago return issue.closed_at > one_month_ago

View File

@@ -9,7 +9,7 @@ chapter = "Bare Bones"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "7212ffaa8383122b1eb07fe1854814f99d2e1af4" translation_based_on_commit = "7212ffaa8383122b1eb07fe1854814f99d2e1af4"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["woodyZootopia", "JohnTitor"] translators = ["swnakamura", "JohnTitor"]
+++ +++
この記事では、Rustで最小限の64bitカーネルを作ります。前の記事で作った[フリースタンディングなRustバイナリ][freestanding Rust binary]を下敷きにして、何かを画面に出力する、ブータブルディスクイメージを作ります。 この記事では、Rustで最小限の64bitカーネルを作ります。前の記事で作った[フリースタンディングなRustバイナリ][freestanding Rust binary]を下敷きにして、何かを画面に出力する、ブータブルディスクイメージを作ります。

View File

@@ -260,7 +260,7 @@ That's where the [`build-std` feature] of cargo comes in. It allows to recompile
[`build-std` feature]: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#build-std [`build-std` feature]: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#build-std
[nightly Rust compilers]: #installing-rust-nightly [nightly Rust compilers]: #installing-rust-nightly
To use the feature, we need to create a [cargo configuration] file at `.cargo/config.toml` with the following content: To use the feature, we need to create a local [cargo configuration] file at `.cargo/config.toml` (the `.cargo` folder should be next to your `src` folder) with the following content:
```toml ```toml
# in .cargo/config.toml # in .cargo/config.toml
@@ -410,7 +410,7 @@ Adding the bootloader as a dependency is not enough to actually create a bootabl
[post-build scripts]: https://github.com/rust-lang/cargo/issues/545 [post-build scripts]: https://github.com/rust-lang/cargo/issues/545
To solve this problem, we created a tool named `bootimage` that first compiles the kernel and bootloader, and then links them together to create a bootable disk image. To install the tool, execute the following command in your terminal: To solve this problem, we created a tool named `bootimage` that first compiles the kernel and bootloader, and then links them together to create a bootable disk image. To install the tool, go into your home directory (or any directory outside of your cargo project) and execute the following command in your terminal:
``` ```
cargo install bootimage cargo install bootimage
@@ -418,7 +418,7 @@ cargo install bootimage
For running `bootimage` and building the bootloader, you need to have the `llvm-tools-preview` rustup component installed. You can do so by executing `rustup component add llvm-tools-preview`. For running `bootimage` and building the bootloader, you need to have the `llvm-tools-preview` rustup component installed. You can do so by executing `rustup component add llvm-tools-preview`.
After installing `bootimage` and adding the `llvm-tools-preview` component, we can create a bootable disk image by executing: After installing `bootimage` and adding the `llvm-tools-preview` component, you can create a bootable disk image by going back into your cargo project directory and executing:
``` ```
> cargo bootimage > cargo bootimage

View File

@@ -426,7 +426,7 @@ warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
要让在 QEMU 中运行内核更轻松,我们可以设置在 cargo 配置文件中设置 `runner` 配置项: 要让在 QEMU 中运行内核更轻松,我们可以设置在 cargo 配置文件中设置 `runner` 配置项:
```toml ```toml
# in .cargo/config # in .cargo/config.toml
[target.'cfg(target_os = "none")'] [target.'cfg(target_os = "none")']
runner = "bootimage runner" runner = "bootimage runner"

View File

@@ -9,7 +9,7 @@ chapter = "Bare Bones"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "bd6fbcb1c36705b2c474d7fcee387bfea1210851" translation_based_on_commit = "bd6fbcb1c36705b2c474d7fcee387bfea1210851"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["woodyZootopia", "JohnTitor"] translators = ["swnakamura", "JohnTitor"]
+++ +++
[VGAテキストモード][VGA text mode]は画面にテキストを出力するシンプルな方法です。この記事では、すべてのunsafeな要素を別のモジュールにカプセル化することで、それを安全かつシンプルに扱えるようにするインターフェースを作ります。また、Rustの[フォーマッティングマクロ][formatting macros]のサポートも実装します。 [VGAテキストモード][VGA text mode]は画面にテキストを出力するシンプルな方法です。この記事では、すべてのunsafeな要素を別のモジュールにカプセル化することで、それを安全かつシンプルに扱えるようにするインターフェースを作ります。また、Rustの[フォーマッティングマクロ][formatting macros]のサポートも実装します。

View File

@@ -77,7 +77,7 @@ error[E0463]: can't find crate for `test`
#![test_runner(crate::test_runner)] #![test_runner(crate::test_runner)]
#[cfg(test)] #[cfg(test)]
fn test_runner(tests: &[&dyn Fn()]) { pub fn test_runner(tests: &[&dyn Fn()]) {
println!("Running {} tests", tests.len()); println!("Running {} tests", tests.len());
for test in tests { for test in tests {
test(); test();

View File

@@ -9,7 +9,7 @@ chapter = "Bare Bones"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "dce5c9825bd4e7ea6c9530e999c9d58f80c585cc" translation_based_on_commit = "dce5c9825bd4e7ea6c9530e999c9d58f80c585cc"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["woodyZootopia", "JohnTitor"] translators = ["swnakamura", "JohnTitor"]
+++ +++
この記事では、`no_std`な実行環境における<ruby>単体テスト<rp> (</rp><rt>unit test</rt><rp>) </rp></ruby>と<ruby>結合テスト<rp> (</rp><rt>integration test</rt><rp>) </rp></ruby>について学びます。Rustではカスタムテストフレームワークがサポートされているので、これを使ってカーネルの中でテスト関数を実行します。QEMUの外へとテストの結果を通知するため、QEMUと`bootimage`の様々な機能を使います。 この記事では、`no_std`な実行環境における<ruby>単体テスト<rp> (</rp><rt>unit test</rt><rp>) </rp></ruby>と<ruby>結合テスト<rp> (</rp><rt>integration test</rt><rp>) </rp></ruby>について学びます。Rustではカスタムテストフレームワークがサポートされているので、これを使ってカーネルの中でテスト関数を実行します。QEMUの外へとテストの結果を通知するため、QEMUと`bootimage`の様々な機能を使います。
@@ -81,7 +81,7 @@ error[E0463]: can't find crate for `test`
#![test_runner(crate::test_runner)] #![test_runner(crate::test_runner)]
#[cfg(test)] #[cfg(test)]
fn test_runner(tests: &[&dyn Fn()]) { pub fn test_runner(tests: &[&dyn Fn()]) {
println!("Running {} tests", tests.len()); println!("Running {} tests", tests.len());
for test in tests { for test in tests {
test(); test();

View File

@@ -73,7 +73,7 @@ To implement a custom test framework for our kernel, we add the following to our
#![test_runner(crate::test_runner)] #![test_runner(crate::test_runner)]
#[cfg(test)] #[cfg(test)]
fn test_runner(tests: &[&dyn Fn()]) { pub fn test_runner(tests: &[&dyn Fn()]) {
println!("Running {} tests", tests.len()); println!("Running {} tests", tests.len());
for test in tests { for test in tests {
test(); test();

View File

@@ -77,7 +77,7 @@ error[E0463]: can't find crate for `test`
#![test_runner(crate::test_runner)] #![test_runner(crate::test_runner)]
#[cfg(test)] #[cfg(test)]
fn test_runner(tests: &[&dyn Fn()]) { pub fn test_runner(tests: &[&dyn Fn()]) {
println!("Running {} tests", tests.len()); println!("Running {} tests", tests.len());
for test in tests { for test in tests {
test(); test();
@@ -989,7 +989,7 @@ harness = false
#![no_main] #![no_main]
use core::panic::PanicInfo; use core::panic::PanicInfo;
use blog_os::{QemuExitCode, exit_qemu, serial_println}; use blog_os::{QemuExitCode, exit_qemu, serial_println, serial_print};
#[no_mangle] #[no_mangle]
pub extern "C" fn _start() -> ! { pub extern "C" fn _start() -> ! {

View File

@@ -9,7 +9,7 @@ chapter = "Interrupts"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "a8a6b725cff2e485bed76ff52ac1f18cec08cc7b" translation_based_on_commit = "a8a6b725cff2e485bed76ff52ac1f18cec08cc7b"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["woodyZootopia"] translators = ["swnakamura"]
+++ +++
CPU例外は、例えば無効なメモリアドレスにアクセスしたときやゼロ除算したときなど、様々なミスによって発生します。それらに対処するために、ハンドラ関数を提供する **<ruby>割り込み記述子表<rp> (</rp><rt>interrupt descriptor table</rt><rp>) </rp></ruby>** を設定しなくてはなりません。この記事を読み終わる頃には、私達のカーネルは[ブレークポイント例外][breakpoint exceptions]を捕捉し、その後通常の実行を継続できるようになっているでしょう。 CPU例外は、例えば無効なメモリアドレスにアクセスしたときやゼロ除算したときなど、様々なミスによって発生します。それらに対処するために、ハンドラ関数を提供する **<ruby>割り込み記述子表<rp> (</rp><rt>interrupt descriptor table</rt><rp>) </rp></ruby>** を設定しなくてはなりません。この記事を読み終わる頃には、私達のカーネルは[ブレークポイント例外][breakpoint exceptions]を捕捉し、その後通常の実行を継続できるようになっているでしょう。

View File

@@ -9,7 +9,7 @@ chapter = "Interrupts"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "81d4f49f153eb5f390681f5c13018dd2aa6be0b1" translation_based_on_commit = "81d4f49f153eb5f390681f5c13018dd2aa6be0b1"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["shimomura1004", "woodyZootopia"] translators = ["shimomura1004", "swnakamura"]
+++ +++
この記事では、ハードウェア割り込みを正しく CPU に転送するためにプログラム可能な割り込みコントローラの設定を行います。これらの割り込みに対処するため、例外ハンドラのときに行ったのと同じように割り込み記述子表に新しいエントリを追加しなくてはいけません。ここでは周期タイマ割り込みの受け方と、キーボードからの入力の受け方を学びます。 この記事では、ハードウェア割り込みを正しく CPU に転送するためにプログラム可能な割り込みコントローラの設定を行います。これらの割り込みに対処するため、例外ハンドラのときに行ったのと同じように割り込み記述子表に新しいエントリを追加しなくてはいけません。ここでは周期タイマ割り込みの受け方と、キーボードからの入力の受け方を学びます。

View File

@@ -9,7 +9,7 @@ chapter = "Memory Management"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "3315bfe2f63571f5e6e924d58ed32afd8f39f892" translation_based_on_commit = "3315bfe2f63571f5e6e924d58ed32afd8f39f892"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["woodyZootopia", "JohnTitor"] translators = ["swnakamura", "JohnTitor"]
+++ +++
この記事では**ページング**を紹介します。これは、私達のオペレーティングシステムにも使う、とても一般的なメモリ管理方式です。なぜメモリの分離が必要なのか、**セグメンテーション**がどういう仕組みなのか、**仮想メモリ**とは何なのか、ページングがいかにしてメモリ<ruby>断片化<rp> (</rp><rt>フラグメンテーション</rt><rp>) </rp></ruby>の問題を解決するのかを説明します。また、x86_64アーキテクチャにおける、マルチレベルページテーブルのレイアウトについても説明します。 この記事では**ページング**を紹介します。これは、私達のオペレーティングシステムにも使う、とても一般的なメモリ管理方式です。なぜメモリの分離が必要なのか、**セグメンテーション**がどういう仕組みなのか、**仮想メモリ**とは何なのか、ページングがいかにしてメモリ<ruby>断片化<rp> (</rp><rt>フラグメンテーション</rt><rp>) </rp></ruby>の問題を解決するのかを説明します。また、x86_64アーキテクチャにおける、マルチレベルページテーブルのレイアウトについても説明します。

View File

@@ -149,7 +149,7 @@ x86_64 平台使用4级页表页大小为4KiB无论层级每个页表
![Bits 012 are the page offset, bits 1221 the level 1 index, bits 2130 the level 2 index, bits 3039 the level 3 index, and bits 3948 the level 4 index](x86_64-table-indices-from-address.svg) ![Bits 012 are the page offset, bits 1221 the level 1 index, bits 2130 the level 2 index, bits 3039 the level 3 index, and bits 3948 the level 4 index](x86_64-table-indices-from-address.svg)
我们可以看到,每个表索引号占据9个字节,这当然是有道理的,每个表都有 2^9 = 512 个条目低12位用来表示内存页的偏移量2^12 bytes = 4KiB而上文提到页大小为4KiB。第48-64位毫无用处这也就意味着 x86_64 并非真正的64位因为它实际上支持48位地址。 我们可以看到,每个表索引号占据 9 个比特,这当然是有道理的,每个表都有 2^9 = 512 个条目低12位用来表示内存页的偏移量2^12 bytes = 4KiB而上文提到页大小为4KiB。第 48-64 位毫无用处,这也就意味着 x86_64 并非真正的 64 位,因为它实际上支持 48 位地址。
[5-level page table]: https://en.wikipedia.org/wiki/Intel_5-level_paging [5-level page table]: https://en.wikipedia.org/wiki/Intel_5-level_paging
@@ -191,7 +191,7 @@ x86_64 平台使用4级页表页大小为4KiB无论层级每个页表
- 1个4级页表 - 1个4级页表
- 512个3级页表因为4级页表可以有512个条目 - 512个3级页表因为4级页表可以有512个条目
- 512*512个2级页表因为每个3级页表可以有512个条目 - 512*512个2级页表因为每个3级页表可以有512个条目
- 512*512*512个1级页表因为每个2级页表可以有512个条目 - 512\*512\*512个1级页表因为每个2级页表可以有512个条目
### 页表格式 ### 页表格式
@@ -225,7 +225,7 @@ pub struct PageTable {
| 63 | no execute | 禁止在该页中运行代码EFER寄存器中的NXE比特位必须一同被设置 | | 63 | no execute | 禁止在该页中运行代码EFER寄存器中的NXE比特位必须一同被设置 |
我们可以看到仅1251位会用于存储页帧地址或页表地址其余比特都用于存储标志位或由操作系统自由使用。 我们可以看到仅1251位会用于存储页帧地址或页表地址其余比特都用于存储标志位或由操作系统自由使用。
其原因就是该地址总是指向一个4096比特对齐的地址、页表或者页帧的起始地址。 其原因就是该地址总是指向一个4096字节对齐的地址、页表或者页帧的起始地址。
这也就意味着0-11位始终为0没有必要存储这些东西硬件层面在使用该地址之前也会将这12位比特设置为052-63位同理因为x86_64平台仅支持52位物理地址类似于上文中提到的仅支持48位虚拟地址的原因 这也就意味着0-11位始终为0没有必要存储这些东西硬件层面在使用该地址之前也会将这12位比特设置为052-63位同理因为x86_64平台仅支持52位物理地址类似于上文中提到的仅支持48位虚拟地址的原因
进一步说明一下可用的标志位: 进一步说明一下可用的标志位:

View File

@@ -7,7 +7,7 @@ date = 2019-03-14
[extra] [extra]
chapter = "Memory Management" chapter = "Memory Management"
translation_based_on_commit = "27ab4518acbb132e327ed4f4f0508393e9d4d684" translation_based_on_commit = "27ab4518acbb132e327ed4f4f0508393e9d4d684"
translators = ["woodyZootopia", "garasubo"] translators = ["swnakamura", "garasubo"]
+++ +++
この記事では私達のカーネルをページングに対応させる方法についてお伝えします。まずページテーブルの物理フレームにカーネルがアクセスできるようにする様々な方法を示し、それらの利点と欠点について議論します。次にアドレス変換関数を、ついで新しい<ruby>対応付け<rp> (</rp><rt>マッピング</rt><rp>) </rp></ruby>を作るための関数を実装します。 この記事では私達のカーネルをページングに対応させる方法についてお伝えします。まずページテーブルの物理フレームにカーネルがアクセスできるようにする様々な方法を示し、それらの利点と欠点について議論します。次にアドレス変換関数を、ついで新しい<ruby>対応付け<rp> (</rp><rt>マッピング</rt><rp>) </rp></ruby>を作るための関数を実装します。

View File

@@ -9,7 +9,7 @@ chapter = "Memory Management"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "afeed7477bb19a29d94a96b8b0620fd241b0d55f" translation_based_on_commit = "afeed7477bb19a29d94a96b8b0620fd241b0d55f"
# GitHub usernames of the people that translated this post # GitHub usernames of the people that translated this post
translators = ["woodyZootopia", "garasubo"] translators = ["swnakamura", "garasubo"]
+++ +++
この記事では、私たちのカーネルにヒープ<ruby>割り当て<rp> (</rp><rt>アロケーション</rt><rp>) </rp></ruby>の機能を追加します。まず動的メモリの基礎を説明し、どのようにして借用チェッカがありがちなアロケーションエラーを防いでくれるのかを示します。その後Rustの基本的なアロケーションインターフェースを実装し、ヒープメモリ領域を作成し、アロケータクレートを設定します。この記事を終える頃には、Rustに組み込みの`alloc`クレートのすべてのアロケーション・コレクション型が私たちのカーネルで利用可能になっているでしょう。 この記事では、私たちのカーネルにヒープ<ruby>割り当て<rp> (</rp><rt>アロケーション</rt><rp>) </rp></ruby>の機能を追加します。まず動的メモリの基礎を説明し、どのようにして借用チェッカがありがちなアロケーションエラーを防いでくれるのかを示します。その後Rustの基本的なアロケーションインターフェースを実装し、ヒープメモリ領域を作成し、アロケータクレートを設定します。この記事を終える頃には、Rustに組み込みの`alloc`クレートのすべてのアロケーション・コレクション型が私たちのカーネルで利用可能になっているでしょう。

File diff suppressed because it is too large Load Diff

View File

@@ -570,11 +570,26 @@ use super::align_up;
use core::mem; use core::mem;
impl LinkedListAllocator { impl LinkedListAllocator {
/// Aligns a given address up to a multiple of
/// `mem::align_of::<ListNode>, which is 8 bytes
/// for x86_64.
fn align_to_list_node(addr: usize) -> usize {
align_up(addr, mem::align_of::<ListNode>())
}
/// Checks to make sure that alignment and size conditions
/// to store a `ListNode` are guaranteed for a given region
/// [addr, addr + size).
fn is_valid_region(addr: usize, size: usize) -> bool {
addr == Self::align_to_list_node(addr) &&
size >= mem::size_of::<ListNode>()
}
/// Adds the given memory region to the front of the list. /// Adds the given memory region to the front of the list.
unsafe fn add_free_region(&mut self, addr: usize, size: usize) { unsafe fn add_free_region(&mut self, addr: usize, size: usize) {
// ensure that the freed region is capable of holding ListNode // ensure that the region is capable of holding ListNode
assert_eq!(align_up(addr, mem::align_of::<ListNode>()), addr); assert!(Self::is_valid_region(addr, size));
assert!(size >= mem::size_of::<ListNode>());
// create a new list node and append it at the start of the list // create a new list node and append it at the start of the list
let mut node = ListNode::new(size); let mut node = ListNode::new(size);
@@ -664,18 +679,34 @@ impl LinkedListAllocator {
fn alloc_from_region(region: &ListNode, size: usize, align: usize) fn alloc_from_region(region: &ListNode, size: usize, align: usize)
-> Result<usize, ()> -> Result<usize, ()>
{ {
let alloc_start = align_up(region.start_addr(), align); let mut alloc_start = align_up(region.start_addr(), align);
let alloc_end = alloc_start.checked_add(size).ok_or(())?;
if alloc_start != region.start_addr() {
// We have some potential wasted space at the beginning of the region
// that cannot be used due to alignment constraints. We want to be
// able to recycle this space as well in our linked list. Otherwise
// we may never be able to reclaim this space.
// We need to ensure that there is enough space up front for a `ListNode`
// so we need to realign alloc_start after `size_of::<ListNode>` bytes
// from `region.start_addr()`.
// In practice, this can occur in x86_64 only when align is set to 16 bytes.
let pushed_start_addr = region
.start_addr()
.checked_add(mem::size_of::<ListNode>())
.ok_or(())?;
alloc_start = align_up(pushed_start_addr, align);
}
let alloc_end = alloc_start.checked_add(size).ok_or(())?;
if alloc_end > region.end_addr() { if alloc_end > region.end_addr() {
// region too small // region too small
return Err(()); return Err(());
} }
let excess_size = region.end_addr() - alloc_end; let excess_size = region.end_addr() - alloc_end;
if excess_size > 0 && excess_size < mem::size_of::<ListNode>() { if excess_size > 0 && !Self::is_valid_region(alloc_end, excess_size) {
// rest of region too small to hold a ListNode (required because the // Improper alignment or the rest of region too small to hold a ListNode (required
// allocation splits the region in a used and a free part) // because the allocation splits the region into a used and up to two free parts).
return Err(()); return Err(());
} }
@@ -687,7 +718,16 @@ impl LinkedListAllocator {
First, the function calculates the start and end address of a potential allocation, using the `align_up` function we defined earlier and the [`checked_add`] method. If an overflow occurs or if the end address is behind the end address of the region, the allocation doesn't fit in the region and we return an error. First, the function calculates the start and end address of a potential allocation, using the `align_up` function we defined earlier and the [`checked_add`] method. If an overflow occurs or if the end address is behind the end address of the region, the allocation doesn't fit in the region and we return an error.
The function performs a less obvious check after that. This check is necessary because most of the time an allocation does not fit a suitable region perfectly, so that a part of the region remains usable after the allocation. This part of the region must store its own `ListNode` after the allocation, so it must be large enough to do so. The check verifies exactly that: either the allocation fits perfectly (`excess_size == 0`) or the excess size is large enough to store a `ListNode`. The function performs a couple of less obvious checks on top of that. When we first perform `align_up` we may get an `alloc_start` that is not the same as `region.start_addr()`. In this case, there can still be some free memory we need to keep track of between `region.start_addr()` (inclusive) to this initially aligned `alloc_start` (exclusive). We need to ensure that this region is suitable for storing a `ListNode` by performing the alignment and size checks in `is_valid_region`.
As `region.start_addr()` is guaranteed to satisfy the alignment condition of `ListNode`, we technically only need to guarantee that the size is not too small. We try and realign after accounting for this space to store one `ListNode` instance after `region.start_addr()`. This may end up pushing our end address out of our region, in which case this entire region we are checking will not be sufficient.
It is interesting to note that this situation can occur in one edge case in the 64-bit architecture we are targeting, where `align` is set to 16 bytes and `region.start_addr()` happens to be some number `16*n + 8`. `alloc_start` would then be set to `16*(n+1)`, leaving us `head_excess_size` of just 8 bytes, which would be insufficient to store the 16 bytes required for a `ListNode`.
We could also have some free memory between `alloc_end` (inclusive) to `region.end_addr()` (exclusive). Here `alloc_end` (in general) is not guaranteed to satisfy the alignment condition of `ListNode`, nor is there a guarantee that the remaining space is sufficient to store a `ListNode`. This check is necessary because most of the time an allocation does not fit a suitable region perfectly, so that a part of the region remains usable after the allocation. This part of the region must store its own `ListNode` after the allocation, so it must be large enough to do so, and it must satisfy the alignment condition, which is exactly what our `is_valid_region` method performs.
We shall soon see how we will actually modify the requested layout size and alignment in our implementation of `GlobalAlloc::alloc()` for the `LinkedListAllocator` to ensure that it additionally conforms to the alignment requirements for storing a `ListNode`. This is essential to ensure that `GlobalAllocator::dealloc()` can successfully add the region back into our linked list.
#### Implementing `GlobalAlloc` #### Implementing `GlobalAlloc`
@@ -712,10 +752,20 @@ unsafe impl GlobalAlloc for Locked<LinkedListAllocator> {
if let Some((region, alloc_start)) = allocator.find_region(size, align) { if let Some((region, alloc_start)) = allocator.find_region(size, align) {
let alloc_end = alloc_start.checked_add(size).expect("overflow"); let alloc_end = alloc_start.checked_add(size).expect("overflow");
let excess_size = region.end_addr() - alloc_end;
if excess_size > 0 { let start_addr = region.start_addr();
allocator.add_free_region(alloc_end, excess_size); let end_addr = region.end_addr();
let tail_excess_size = end_addr - alloc_end;
if tail_excess_size > 0 {
allocator.add_free_region(alloc_end, tail_excess_size);
} }
let head_excess_size = alloc_start - start_addr;
if head_excess_size > 0 {
allocator.add_free_region(start_addr, head_excess_size);
}
alloc_start as *mut u8 alloc_start as *mut u8
} else { } else {
ptr::null_mut() ptr::null_mut()
@@ -735,7 +785,7 @@ Let's start with the `dealloc` method because it is simpler: First, it performs
The `alloc` method is a bit more complex. It starts with the same layout adjustments and also calls the [`Mutex::lock`] function to receive a mutable allocator reference. Then it uses the `find_region` method to find a suitable memory region for the allocation and remove it from the list. If this doesn't succeed and `None` is returned, it returns `null_mut` to signal an error as there is no suitable memory region. The `alloc` method is a bit more complex. It starts with the same layout adjustments and also calls the [`Mutex::lock`] function to receive a mutable allocator reference. Then it uses the `find_region` method to find a suitable memory region for the allocation and remove it from the list. If this doesn't succeed and `None` is returned, it returns `null_mut` to signal an error as there is no suitable memory region.
In the success case, the `find_region` method returns a tuple of the suitable region (no longer in the list) and the start address of the allocation. Using `alloc_start`, the allocation size, and the end address of the region, it calculates the end address of the allocation and the excess size again. If the excess size is not null, it calls `add_free_region` to add the excess size of the memory region back to the free list. Finally, it returns the `alloc_start` address casted as a `*mut u8` pointer. In the success case, the `find_region` method returns a tuple of the suitable region (no longer in the list) and the start address of the allocation. Using `alloc_start`, the allocation size, and the end address of the region, it calculates the end address of the allocation and the excess free fragments that are usable again. If the excess sizes are not zero, it calls `add_free_region` to add the excess sizes of the memory regions back to the free list. Finally, it returns the `alloc_start` address casted as a `*mut u8` pointer.
#### Layout Adjustments #### Layout Adjustments
@@ -797,6 +847,51 @@ many_boxes_long_lived... [ok]
This shows that our linked list allocator is able to reuse freed memory for subsequent allocations. This shows that our linked list allocator is able to reuse freed memory for subsequent allocations.
Additionally, to test that we are not leaking any excess segments due to `alloc_start` realignment we can add a simple test case:
```rust
// in tests/heap_allocation.rs
#[test_case]
fn head_excess_reuse() {
#[derive(Debug, Clone, PartialEq, Eq)]
#[repr(C, align(8))]
struct A(u128, u64);
assert_eq!(8, align_of::<A>());
assert_eq!(24, size_of::<A>()); // 24 % 16 = 8
#[derive(Debug, Clone, PartialEq, Eq)]
#[repr(C, align(16))]
struct B(u128, u64);
assert_eq!(16, align_of::<B>());
assert_eq!(32, size_of::<B>());
let a1 = Box::new(A(1, 1));
let b1 = Box::new(B(1, 1));
let a2 = Box::new(A(2, 2));
assert_eq!(*a1, A(1, 1));
assert_eq!(*b1, B(1, 1));
assert_eq!(*a2, A(2, 2));
let a1_raw = Box::into_raw(a1) as usize;
let b1_raw = Box::into_raw(b1) as usize;
let a2_raw = Box::into_raw(a2) as usize;
assert_eq!(HEAP_START, a1);
assert_eq!(HEAP_START + 48, b1);
assert_eq!(HEAP_START + 24, a2);
}
```
In this test case we start off with two identical structs `A` and `B`, with different alignment requirements as specified in their struct `#[repr]` attributes. Instances of `A` will have addresses that are a multiple of 8 and those of `B` will have addresses that are a multiple of `16`.
`a1`, an instance of struct `A` on the heap, takes up space from `HEAP_START` to `HEAP_START + 24`, as `HEAP_START` is a multiple of 8 already. `b1` is an instance of struct `B` on the heap, but it needs an address that is a multiple of 16. Therefore, although `HEAP_START + 24` is available, our `alloc_from_region` will first attempt to set `alloc_start = HEAP_START + 32`. However, this will not leave enough room to store a `ListNode` in the 8 bytes between `HEAP_START + 24` and `HEAP_START + 32`. Next, it will attempt to set `alloc_start = HEAP_START + 48` to satisfy both the alignment constraint and to allow a `ListNode` to account for the excess size at the head end of this region.
Because we are adding the `head_excess_size` fragment after `tail_excess_size` fragment in our `alloc` implementation, and because our linked list implementation follows LIFO (Last In First Out) ordering, our linked list will first search the `head_excess_size` region first on a new heap alloc request. We exploit this fact in this test by trying to allocate `a2`, which is an instance of struct `A`, which should fit neatly in the 24 bytes that were recycled from `HEAP_START + 24` to `HEAP_START + 48` as a part of the `head_excess_size` fragment from the previous allocation for `b1`. We can see that in our final lines of this test we are leaking these Boxed pointers and casting them to `usize` to help perform these assertions to ensure that our linked list allocator accounted for all the excess fragments.
### Discussion ### Discussion
In contrast to the bump allocator, the linked list allocator is much more suitable as a general-purpose allocator, mainly because it is able to directly reuse freed memory. However, it also has some drawbacks. Some of them are only caused by our basic implementation, but there are also fundamental drawbacks of the allocator design itself. In contrast to the bump allocator, the linked list allocator is much more suitable as a general-purpose allocator, mainly because it is able to directly reuse freed memory. However, it also has some drawbacks. Some of them are only caused by our basic implementation, but there are also fundamental drawbacks of the allocator design itself.

View File

@@ -9,7 +9,7 @@ chapter = "Multitasking"
# Please update this when updating the translation # Please update this when updating the translation
translation_based_on_commit = "bf4f88107966c7ab1327c3cdc0ebfbd76bad5c5f" translation_based_on_commit = "bf4f88107966c7ab1327c3cdc0ebfbd76bad5c5f"
# GitHub usernames of the authors of this translation # GitHub usernames of the authors of this translation
translators = ["kahirokunn", "garasubo", "sozysozbot", "woodyZootopia"] translators = ["kahirokunn", "garasubo", "sozysozbot", "swnakamura"]
# GitHub usernames of the people that contributed to this translation # GitHub usernames of the people that contributed to this translation
translation_contributors = ["asami-kawasaki", "Foo-x"] translation_contributors = ["asami-kawasaki", "Foo-x"]
+++ +++
@@ -471,7 +471,7 @@ Futureは `Poll::Ready` を返した後、再びポーリングされるべき
コンパイラが生成するステートマシンとその `Future` traitの実装はこのようになっている**かもしれません**。実際には、コンパイラは異なる方法でコードを生成しています。 (一応、現在は[_generators_]をベースにした実装になっていますが、これはあくまでも実装の詳細です。) コンパイラが生成するステートマシンとその `Future` traitの実装はこのようになっている**かもしれません**。実際には、コンパイラは異なる方法でコードを生成しています。 (一応、現在は[_generators_]をベースにした実装になっていますが、これはあくまでも実装の詳細です。)
[_generators_]: https://doc.rust-lang.org/nightly/unstable-book/language-features/generators.html [_generators_]: https://doc.rust-lang.org/stable/unstable-book/language-features/generators.html
パズルの最後のピースは、生成された `example` 関数自体のコードです。関数のヘッダは次のように定義されていたことを思い出してください: パズルの最後のピースは、生成された `example` 関数自体のコードです。関数のヘッダは次のように定義されていたことを思い出してください:

View File

@@ -464,7 +464,7 @@ Futures should not be polled again after they returned `Poll::Ready`, so we pani
We now know what the compiler-generated state machine and its implementation of the `Future` trait _could_ look like. In practice, the compiler generates code in a different way. (In case you're interested, the implementation is currently based on [_generators_], but this is only an implementation detail.) We now know what the compiler-generated state machine and its implementation of the `Future` trait _could_ look like. In practice, the compiler generates code in a different way. (In case you're interested, the implementation is currently based on [_generators_], but this is only an implementation detail.)
[_generators_]: https://doc.rust-lang.org/nightly/unstable-book/language-features/generators.html [_generators_]: https://doc.rust-lang.org/stable/unstable-book/language-features/generators.html
The last piece of the puzzle is the generated code for the `example` function itself. Remember, the function header was defined like this: The last piece of the puzzle is the generated code for the `example` function itself. Remember, the function header was defined like this:

View File

@@ -24,7 +24,7 @@
{% if search_term is number %} {% if search_term is number %}
{% set discussion_url = "https://github.com/phil-opp/blog_os/discussions/" ~ search_term %} {% set discussion_url = "https://github.com/phil-opp/blog_os/discussions/" ~ search_term %}
{% else %} {% else %}
{% set search_term_encoded = `"` ~ search_term ~ `"` | urlencode %} {% set search_term_encoded = `"` ~ search_term ~ `"` ~ ` in:title` | urlencode %}
{% set discussion_url = `https://github.com/phil-opp/blog_os/discussions/categories/` ~ category_path ~ `?discussions_q=` ~ search_term_encoded %} {% set discussion_url = `https://github.com/phil-opp/blog_os/discussions/categories/` ~ category_path ~ `?discussions_q=` ~ search_term_encoded %}
{% endif %} {% endif %}