Lorenzo Stoakes

The Linux Memory Manager

I am writing a book about the Linux memory management subsystem which I anticipate will be ready around mid-2024. The intent is to provide as complete a description of how memory management functions in linux as I can, diving deep into the subject with a primary focus on kernel implementation.

While I had hoped to write the book in the open, I decided on balance it would be better to keep things private. I will however keep updating my development diary as work proceeds. I also maintain a twitter account which has more frequent updates.

Proposed Chapters

  1. Allocators - Describe in detail how dynamic memory allocation works, the trade-offs, the algorithms - essentially the what, why and how of memory allocation.

  2. Physical memory allocation - The buddy allocator, struct page*, struct folio*, the kernel functions that provide memory (e.g. alloc_pages()), nodes, zones (in brief, more detailed in NUMA chapter), watermarks, migrate types, page blocks, GFP flags, movability? And yeah maybe this will be quite a large chapter...

  3. Virtual memory - The why, what and how, page tables, page table flags, page table sizes, virtual memory layout, direct mapping, kernel/userland split, basics of how memory is allocated maybe? Page faults, demand paging, basics of swap + mm I/O.

  4. Swap - How swap is operated, the internals and interactions with other parts of the subsystem, the tuneables and perhaps broaching the deeply controversial topic of how much swap you should set and how this interacts with other aspects of how the mm subsystem operates.

  5. Userland virtual memory - Process VMAs, dynamic stack allocation, how memory is copied between userland/kernel, how malloc() actually feeds into the kernel (e.g. sbrk(), mmap()). Copy-on-write, forking, basics of huge pages (covered in more detail in huge pages chapter).

  6. Kernel virtual memory - Slab allocator, kmalloc()/kfree(), vmalloc(), kernel process address space, GUP, kernel threads, etc.

  7. Memory pressure/reclaim - What memory pressure is, how it fits in with demand paging, higher order page starvation, compaction, reclaim, etc.

  8. NUMA - Nodes in much more detail, NUMA rebalance, internals, etc.

  9. Out of memory killer - How the OOM killer works, how to decode an OOM dmesg report, analysis of code, oom_score_adj, etc.

  10. cgroups - How to use memory cgroups, how they are implemented internally, etc.

  11. The page cache - (perhaps split?) - Discussion of the page cache and how this is maintained and operated (and which tunables impact it), dirty pages, internals and interactions with issues such as memory pressure.

  12. Memory-mapped I/O - (perhaps merge with previous chapter?) Discussion of how files can be mapped into memory, how you interact with them, how this is handled internally, how dirty pages are written out, etc.

  13. Practical guide to linux memory management - The various procfs/sysfs interfaces and how to use and decode them, generally a practical how to for sysadmins/developers. Really quite excited about this bit - should be really quite useful for people.

  14. Practical guide to monitoring memory - Using eBPF, DAMON to monitor mm - could perhaps be rolled into previous chapter?

  15. Practical guide to shared memory - How to use, considerations, internals, different interfaces.

  16. Huge pages - (needs to be a chapter?) Discussion of huge pages, why they're useful, the interfaces for them, the internals and specific considerations introduced into kernel code to consider them, Transparent Huge Pages (THP), tuneables.

  17. Early memory management - memblock, bitmap, ELF sections marked init, transition to core memory manager, algorithms.

  18. Security mitigations - Spectre, meltdown, etc. mitigations and impact on kernel code.

  19. Cache management - (not certain whether this will be a separate chapter or not) TLB flushes, cache structure impact on memory in general.

  20. Device memory - Use of memory by drivers specifically memory-mapped registers via the likes of ioremap() and friends. Some consideration of how a driver might use vmalloc() and general driver topics perhaps?

  21. Userland fault handling - The magic of userfaultfd, how this is implemented internally and how it can be used in practice.