Operating Systems

18.0 What this chapter gives you#

1. You will be able to say exactly what an operating system is, and name the three jobs it exists to do.
2. You will be able to describe what a kernel is, what lives inside it, and why almost every kernel is written in C with a little assembly.
3. You will be able to narrate a computer’s boot, from the power supply’s ready signal to the login prompt, with real timings from a real machine.
4. You will be able to explain a process, a thread, and the difference, and read the output of ps without guessing.
5. You will be able to run a scheduling algorithm by hand on paper and produce the same answer the machine would.
6. You will be able to explain what happens mechanically when your program asks the kernel for something, and read a real strace line by line.
7. You will be able to describe file descriptors, permissions, mounting, pipes, signals and shared memory, and say which one to reach for.
8. You will be able to explain containers, virtual machines and kernel panics in terms of one idea: privilege.
9. You will be able to tell the history of UNIX, Linux, Windows, macOS and Android with correct names and years.
10. You will know what building your own operating system actually involves, and which learning resources to start from.

18.1 What an operating system is and why it must exist#

PLAIN18.1.1 in simple words#

1. An operating system is a program whose job is to run other programs.
2. It is the first big program that starts when you switch the machine on, and the last one running when you switch it off.
3. It has three jobs, and only three. Everything else it does is a detail of one of them.
4. Job one: share the hardware. One processor, one disk, one screen, many programs that all want them. Somebody must decide who gets what and when.
5. Job two: keep programs apart. Your music player must not be able to read or wreck your bank app’s memory, whether by accident or on purpose.
6. Job three: hide the hardware behind something simple. Your program says “write these bytes to this file”. It does not say “spin up the disk, move the head, wait”.
7. Without an operating system you can still run one program. That program must then do every one of those three jobs itself, for itself.
8. The operating system is not the desktop, not the windows, not the icons. Those sit on top. The operating system is the part underneath that decides.

PLAIN18.1.2 a picture in your head#

1. Think of a shared workshop with one lathe, one drill and one bench.
2. Twenty people want to build things. Left alone they would fight over the lathe, put each other’s parts in the wrong bin and ruin each other’s work.
3. So the workshop hires a manager.
4. The manager keeps a list of who wants the lathe and gives each person a turn of a few minutes. That is sharing the hardware.
5. The manager gives each person a locked drawer and refuses to let anyone open somebody else’s drawer. That is keeping people apart.
6. The manager takes requests in plain words. You say “cut me a 40 mm disc”. You do not set the lathe’s speeds and feeds yourself. That is hiding the hardware behind something simple.
7. The manager also produces nothing. No customer ever pays for the manager’s own work. The manager exists purely so that twenty people can share.

Where this comparison breaks: the workshop manager is a person with judgement, who can be argued with. The operating system is code, and it decides by fixed rules written years earlier by strangers. It also switches between jobs thousands of times a second, far faster than a human could hand over a lathe. And a real workshop manager cannot physically stop you opening a drawer. The operating system can, because the processor itself refuses the instruction.

PLAIN18.1.3 a worked example#

1. You double-click a music player. Here is what the operating system does that you never see.
2. It finds the program file on the disk, reads its header, and works out how much memory it needs.
3. It sets aside memory for it, and arranges that the player can only see its own memory and nothing else.
4. It creates a process: a record saying “this program is running, here is its memory, here are its open files, here is its identity”.
5. It starts the program’s first instruction, and hands the processor to it.
6. A few milliseconds later it takes the processor away again and gives it to your browser, then to a background updater, then back to the player.
7. The player asks to open a sound file. It does not know which disk, which cable or which controller. It says a name. The operating system does the rest.
8. The player asks to send sound to the speakers. The operating system mixes it with the alert sound from your mail app so both are heard.
9. You close the player. The operating system reclaims the memory, closes the files, and removes the record.
10. Count the operating system’s work in that story: sharing, separating and simplifying. There is nothing else.

PLAIN18.1.4 what is really happening inside#

1. The trick that makes all of this possible is a feature of the processor itself, not of the software.
2. The processor can run in at least two modes. In one mode every instruction is allowed. In the other, dangerous instructions are refused.
3. The operating system’s core runs in the all-allowed mode. Your programs run in the restricted mode.
4. In the restricted mode a program cannot change which memory it can see, cannot talk to devices directly, and cannot switch off interrupts.
5. So when a program needs any of those things it must ask. The asking mechanism is a special instruction that jumps into the operating system at a fixed, pre-arranged address.
6. The program does not choose where it lands. That is the whole point. It lands where the operating system decided, running the operating system’s code.
7. A second hardware feature makes sharing possible: the timer interrupt. A chip pokes the processor at a fixed rate, say 250 times a second.
8. Each poke forces the processor to stop what it is doing and run operating system code. That is how a program that never gives up control is taken off the processor anyway.
9. Take away those two hardware features and no operating system can enforce anything. It becomes a library of helpful routines that programs may ignore.
10. That is exactly what life is like on a small embedded chip with no protection hardware, and it is why such systems trust their code completely.

TECHNICAL18.1.5 the engineer’s version#

1. Formally, an operating system is a resource manager and a virtual machine provider. Both descriptions appear in the standard textbooks: Silberschatz, Galvin and Gagne’s Operating System Concepts, and Tanenbaum and Bos’s Modern Operating Systems.
2. The resource manager view: it multiplexes CPU time, physical memory, storage bandwidth, network bandwidth and device access among competing requesters.
3. The virtual machine view: it presents each process with the illusion of a private processor, a private flat address space and a private set of files.
4. The enforcement primitives are hardware. On x86-64 these are the four privilege rings (only 0 and 3 are used in practice), the paging unit driven by CR3, and the syscall and sysret instruction pair.
5. On ARMv8-A the equivalents are exception levels EL0 (application), EL1 (kernel), EL2 (hypervisor) and EL3 (secure monitor), plus the svc instruction for supervisor calls.
6. Without an OS you are on bare metal. Real categories of bare-metal software: bootloaders, PC firmware, and the tightest embedded control loops.
7. A microcontroller such as an ATmega328P (used in the Arduino Uno) has 2 KB of SRAM and no memory management unit. There is no meaningful isolation to enforce, so the usual answer is a main with an infinite loop.
8. Between bare metal and a full OS sits the real-time operating system (RTOS). FreeRTOS is roughly 6,000 to 9,000 lines of C and gives you tasks, a scheduler, queues and semaphores, with no memory protection by default.

9. The Linux source tree line count is the whole tree including every driver and every architecture. Any single running kernel compiles a small fraction of it. Quoting “Linux is 30 million lines” as the size of a running kernel is wrong.
10. Tools that show you the boundary in action: strace on Linux, dtruss on macOS, Process Monitor on Windows, and perf trace on Linux.

WORDS18.1.6 remember these#

1. Operating system — the program that runs other programs — the resource manager and abstraction layer between hardware and applications.
2. Bare metal — no operating system at all — code executing directly on hardware with no supervisor and no runtime services.
3. Kernel mode — the mode where everything is allowed — the privileged execution state, ring 0 on x86-64 or EL1 on ARMv8-A.
4. User mode — the mode where programs are restricted — the unprivileged execution state, ring 3 or EL0.
5. Timer interrupt — a regular poke from a chip — a periodic hardware interrupt used to force scheduler entry and enable preemption.
6. Multiplexing — sharing one thing among many users — time-division or space-division allocation of a single resource among several requesters.

18.2 The kernel: the part that is really the operating system#

PLAIN18.2.1 in simple words#

1. The kernel is the core of the operating system. It is the part that runs in the all-allowed mode of the processor.
2. It is a single program file on disk. On Linux it is often called vmlinuz. On Windows it is ntoskrnl.exe. On macOS it is the kernel file inside a kernel collection.
3. When people say “Linux” strictly, they mean only this file and its source code. Everything else you use is separate software.
4. Things that are inside the kernel: the scheduler, memory management, the file systems, the network stack, and most device drivers.
5. Things that are outside the kernel: the shell, the window system, the web browser, the compiler, the login screen, the package manager.
6. The dividing line is privilege, not importance. Your window system is very important and still runs as an ordinary program.
7. Code inside the kernel can do anything, so a mistake there can destroy the whole machine. Code outside can only wreck itself.
8. Designers therefore argue endlessly about how much should be inside.

PLAIN18.2.2 a picture in your head#

1. Think of a hospital. The kernel is the operating theatre and the staff allowed inside it.
2. Inside the theatre you may do anything to the patient. Nothing stops you. That power is needed, and it is dangerous.
3. Everything else in the hospital — reception, the cafe, the pharmacy counter, the ward — happens outside, under normal rules.
4. A monolithic kernel is a hospital where the theatre is huge: pharmacy, imaging, pathology and the blood bank are all inside the sterile zone. Fast, because nobody walks anywhere. Risky, because one contaminated glove contaminates everything.
5. A microkernel is a hospital where the theatre is a tiny room containing only the surgeon. Pharmacy, imaging and pathology are separate departments you telephone. Safer. Slower, because of all the telephoning.
6. A hybrid is a hospital that says “tiny theatre” in the brochure, and in practice keeps imaging and pathology inside anyway.

Where this comparison breaks: hospital departments genuinely cannot infect each other from down the corridor, whereas a microkernel’s separated services still share the same physical memory chips and the same processor. The isolation is enforced by the memory management unit, not by distance. And a phone call between hospital departments costs seconds; a message between microkernel services costs about a microsecond. The relative costs are completely different.

PLAIN18.2.3 a worked example#

1. Ask: where does the code that understands the ext4 filesystem live?
2. In Linux, inside the kernel. It is a module, ext4.ko, loaded into kernel memory and running with full privilege.
3. In MINIX 3, a filesystem server is an ordinary user-mode process. If it crashes, MINIX restarts it and the machine keeps running.
4. In Windows, NTFS is ntfs.sys, a kernel-mode driver. Full privilege.
5. Now ask: where does the code that draws your windows live?
6. On Linux, outside the kernel entirely, in the X server or a Wayland compositor. The kernel only provides the low-level graphics device interface.
7. On Windows since NT 4.0 in 1996, a large part of the window manager and the graphics engine was moved into kernel mode for speed. That decision was controversial and caused years of graphics-driver blue screens.
8. Same job, three different sides of the line, for reasons of speed and history rather than principle.

PLAIN18.2.4 what is really happening inside#

1. A kernel is not a process. Nothing schedules it as a job. It is a body of code that gets entered when something happens.
2. There are exactly three ways in. Remember these three and kernel behaviour stops being mysterious.
3. Way one: a system call. A program deliberately executes the special instruction that jumps into the kernel.
4. Way two: an interrupt. A device raises a line, the processor stops the current instruction stream and runs a kernel handler.
5. Way three: an exception or fault. The program did something the processor could not complete: touched an unmapped page, divided by zero, executed an illegal instruction.
6. In all three cases the processor switches mode, switches to a kernel stack, and jumps to an address the kernel published in advance.
7. The kernel does the work, then returns, and the processor drops back to the restricted mode at the exact instruction that follows.
8. Between those events the kernel is doing nothing at all. An idle Linux machine really is idle. The kernel is a set of reactions, not a running loop.
9. There are exceptions to that: kernels also run their own threads for background work. On Linux you can see them in ps with names in square brackets, like [kworker/0:1].

TECHNICAL18.2.5 the engineer’s version#

1. Monolithic: all OS services execute in a single kernel address space in supervisor mode. Communication is a plain function call. Examples: Linux, FreeBSD, all classic UNIX.
2. Microkernel: the kernel provides only address spaces, threads and inter-process communication. Drivers, filesystems and network stacks run as user-mode servers. Examples: MINIX 3, QNX Neutrino, seL4, GNU Hurd.
3. Hybrid: a microkernel-derived structure with performance-critical services linked into kernel space. Examples: Windows NT, XNU. Some engineers reject “hybrid” as a marketing word for “monolithic with a Mach heritage”. Say that both readings are defensible.
4. Exokernel: the kernel only multiplexes and protects hardware, exposing it almost raw; libraries in user space implement the abstractions. From MIT, Dawson Engler and Frans Kaashoek, 1995. Nearly all influence, almost no deployment. The idea resurfaced in unikernels and in Linux io_uring.
5. The Tanenbaum-Torvalds debate began on 29 January 1992 in the comp.os.minix newsgroup, in a post by Andrew S. Tanenbaum, author of MINIX and of the standard textbook, titled “LINUX is obsolete”.
6. Tanenbaum’s two claims: monolithic kernels are the wrong design, and writing one in 1991 was “a giant step back into the 1970s”; and Linux was too tied to the Intel 386 to survive changes in hardware.
7. Torvalds’s replies: microkernels are theoretically nicer but MINIX had real design faults, targeting the 386 was a deliberate choice for a learning project, and application-level portability is what users actually feel.
8. How it turned out, honestly: Tanenbaum was right about portability mattering and Linux was later ported to more than 20 architectures. He was wrong about the market outcome. Torvalds was right that a working monolithic kernel beat an elegant unfinished one. Neither side won the design argument outright, and seL4’s formal proof of correctness in 2009 showed the microkernel case is still very much alive.

9. Why C: it compiles to predictable machine code with no hidden runtime, no garbage collector, no exceptions and no implicit allocation. A kernel cannot depend on services that only exist once the kernel is running.
10. Why assembly is still needed: the very first instructions after reset, context switch, interrupt entry and exit, setting control registers, and atomic primitives. In Linux 6.x the arch/ directory holds this, a few tens of thousands of assembly lines against millions of lines of C.
11. Why C++ appears in places: XNU’s IOKit driver framework uses a restricted subset of Embedded C++ with no exceptions, no templates and no multiple inheritance. Windows drivers use C++ widely.
12. Rust in Linux: Miguel Ojeda’s Rust for Linux work was merged for Linux 6.1, released 11 December 2022. The first production Rust drivers landed in Linux 6.8 in March 2024, including the Android Binder driver and two Ethernet PHY drivers. As of December 2025 Rust is an official kernel language alongside C and assembly, not an experiment.
13. Why Rust: about two-thirds of serious kernel security bugs historically come from memory-safety errors — use after free, buffer overflow, data races. Rust’s borrow checker rejects most of those at compile time with no runtime cost. This is established, not marketing: Microsoft and Google have both published the roughly 70 percent figure for their own codebases.
14. Windows also ships Rust in the kernel: Microsoft stated in 2023 that parts of the Windows kernel, including a rewritten win32kbase font parser and DWM core components, are now Rust.

WORDS18.2.6 remember these#

1. Kernel — the core that has full power — the privileged portion of the OS that executes in supervisor mode.
2. Monolithic kernel — one big privileged block — all services share a single kernel address space and call each other directly.
3. Microkernel — a tiny privileged core — only address spaces, threads and IPC are privileged; other services run as user-mode servers.
4. Kernel module — a piece of kernel you can add later — dynamically loadable object linked into the running kernel’s address space.
5. Interrupt — a device raising its hand — an asynchronous hardware signal that diverts the processor into a registered handler.
6. Exception — the processor could not continue — a synchronous fault such as a page fault, divide by zero or invalid opcode.

18.3 The boot sequence, step by step#

PLAIN18.3.1 in simple words#

1. Booting is the process of getting from “no power” to “a working machine”, using nothing but what is already stored in the hardware.
2. The word comes from “pulling yourself up by your own bootstraps”, which is the impossible thing this process appears to do.
3. It is a chain. Each stage is small and stupid, and its only real job is to find and start the next, slightly cleverer stage.
4. Stage 0: the power supply tells the rest of the machine that the voltages are stable and it is safe to start.
5. Stage 1: the processor starts executing at one fixed address that is burned into the chip’s design. It always starts there. It cannot start anywhere else.
6. Stage 2: at that address is firmware, a small program in a chip on the board. It tests the machine and finds something to boot from.
7. Stage 3: the firmware loads a bootloader from disk. The bootloader knows how to find the operating system.
8. Stage 4: the bootloader loads the kernel into memory and jumps into it.
9. Stage 5: the kernel sets itself up, finds the real disk, and starts the very first ordinary program.
10. Stage 6: that first program starts everything else, and eventually you get a login prompt.

PLAIN18.3.2 a picture in your head#

1. Think of arriving at a large office building at night, with only a note in your pocket saying “go to the front desk”.
2. At the front desk is a laminated card. It says: check the lights work, check the lifts work, then take the lift to floor 3.
3. On floor 3 is another card: it says which of several offices you want, and what the key code is.
4. In that office is a thick manual. Reading it takes a while. It tells you how to run the whole building.
5. Once you have read the manual, you telephone the staff and they arrive one by one and start their own jobs.
6. Finally the reception desk opens and members of the public can come in. That is your login prompt.
7. Notice that no stage had to know everything. Each stage only had to know enough to find the next.

Where this comparison breaks: you are one person walking through a building, whereas the real sequence involves several processors. On a modern x86 machine the main processor is not even the first thing that runs; a separate management engine inside the chipset starts first and holds the main processor in reset. And the “cards” are not just instructions, they are complete programs with drivers, network stacks and graphics, sometimes larger than early operating systems.

PLAIN18.3.3 a worked example#

1. Here is a real boot, measured on the Linux machine this chapter was written on. The times are seconds since the kernel took control, printed by dmesg.

0.000000  Linux version 6.18.5, gcc 15.2.0
0.000064  tsc: Detected 2100.000 MHz processor
0.165101  Booting paravirtualized kernel on KVM
0.173953  random: crng init done
0.475547  smpboot: CPU0: Intel(R) Xeon(R) @ 2.10GHz
0.494363  smpboot: Total of 2 processors activated
0.633029  Memory: 8203096K/8388216K available
0.636116  devtmpfs: initialized
0.927746  Unpacking initramfs...
1.039733  virtio_blk virtio1: [vda] 536870912 blocks
1.162061  Freeing unused kernel image memory: 2892K
1.163192  Write protecting kernel read-only data: 26624k
1.174639  Run /process_api as init process
2.551159  EXT4-fs (vda): mounted filesystem r/w

2. Read that from the top. For the first 0.165 seconds the kernel is doing nothing but working out what machine it is on.
3. At 0.475 the first processor is identified, and by 0.494 the second processor has been woken up and put to work. Before that moment the machine had one working core no matter how many it owns.
4. At 0.633 the memory manager reports what it has: 8,203,096 KB usable out of 8,388,216 KB total. The missing 185 MB is the kernel itself plus reserved regions.
5. At 0.927 the kernel unpacks the initial ramdisk, a small filesystem held in memory that contains the drivers needed to reach the real disk.
6. At 1.039 the block device driver finds the disk. Only now does a disk exist as far as the kernel is concerned.
7. At 1.162 the kernel frees its own setup code, which will never run again. 2,892 KB returned to the pool.
8. At 1.174 the kernel starts the first user-mode program. On this machine that is /process_api. On a normal Linux desktop the line reads Run /sbin/init as init process.
9. At 2.551 the real root filesystem is mounted. Everything after this point is ordinary software.
10. The whole kernel phase took 2.55 seconds, and 1.37 of those seconds were spent after the first user program had already started.

PLAIN18.3.4 what is really happening inside#

1. Power good. The power supply raises a signal when its outputs are within tolerance. Until then the whole board is held in reset. On a desktop this takes roughly 100 to 500 milliseconds after you press the button.
2. Reset vector. The processor comes out of reset with its registers at fixed values and starts fetching from one address chosen by the chip designers. Nothing is loaded from disk yet, because nothing can read a disk.
3. That address is wired so that it lands in the firmware chip, not in RAM. RAM is not even usable yet, because the memory controller has not been set up.
4. Firmware. The firmware runs. It configures the memory controller, tests the machine, and builds tables describing what hardware exists.
5. The self test at this stage is called POST, power-on self test. If it fails before the screen works, the machine reports the failure by beeping in a pattern or flashing a light, because that is all it has.
6. Boot device selection. The firmware works down a list of places to look: internal disk, USB, network. You can edit this list in the firmware settings.
7. Bootloader. The firmware loads a small program from the chosen device and jumps to it. This program’s whole job is to find the kernel.
8. Why a separate bootloader exists at all: the firmware does not know what a Linux kernel is, and the kernel is too big and too clever to be started directly by dumb firmware.
9. Loading the kernel. The kernel file on disk is usually compressed. The bootloader loads it, and a small decompressor stub at its front expands the real kernel into memory and jumps to it.
10. Initial ramdisk. The bootloader also loads a second file: a compressed archive of a tiny root filesystem. The kernel unpacks it into memory.
11. Why: the real disk might need a driver, and that driver might live on the real disk. The ramdisk breaks the circle by carrying the driver with it.
12. Kernel initialization. The kernel sets up its page tables, starts the other processors, initializes the scheduler and memory manager, and probes for devices.
13. Mounting the root filesystem. With a working disk driver, the kernel mounts the real root and switches to it, then throws away the ramdisk.
14. The first user process. The kernel executes one program in user mode and gives it process ID 1. If that program ever exits, the kernel panics, because there is nothing left to run.
15. Services. Process 1 reads its configuration and starts everything else: logging, networking, the sound daemon, the display manager.
16. Login. The display manager or the terminal getty draws a prompt. The boot is over.

TECHNICAL18.3.5 the engineer’s version#

1. On x86-64 the reset vector is physical address 0xFFFFFFF0, sixteen bytes below the top of the 4 GiB space, executed in 16-bit real mode with CS base set so the first fetch lands in the firmware flash. This is fixed by Intel and AMD, not by software.
2. The processor comes out of reset in real mode even on a 2026 machine, and the firmware must walk it through protected mode to long mode. That is 1978 compatibility still costing time in every boot.
3. On ARMv8-A the reset address is implementation defined and supplied by the SoC; the boot core starts at EL3 and steps down to EL2 and EL1.
4. BIOS is the legacy firmware interface, from the IBM PC of 1981. It reads the first 512-byte sector, the master boot record, checks for the signature 0x55AA at offset 510, loads it at 0x7C00 and jumps there. 446 bytes of code is all you get.
5. UEFI replaced it. Intel began the work in 1998 for Itanium as the Intel Boot Initiative, later EFI; Intel stopped at version 1.10 and handed it to the Unified EFI Forum, which published UEFI 2.0 in 2006. The current specification is UEFI 2.11, published in December 2024.
6. UEFI boots differently: it reads a GPT-partitioned disk, mounts the EFI System Partition, which the specification requires to be FAT12, FAT16 or FAT32, and executes a PE-format .efi executable from it.
7. Standard ESP paths: \EFI\BOOT\BOOTX64.EFI is the fallback the specification mandates; \EFI\Microsoft\Boot\bootmgfw.efi is Windows Boot Manager; \EFI\ubuntu\grubx64.efi is a distribution’s GRUB. The first is a standard. The others are conventions.
8. Secure Boot is a UEFI feature: firmware verifies the signature on the loaded image against keys in the db variable. Most distributions use a small Microsoft-signed shim that then verifies the distribution’s own key.
9. GRUB 2 is the common Linux bootloader. It reads the filesystem, presents a menu, and uses the Linux boot protocol: it loads vmlinuz and initrd, fills in a boot_params structure, and jumps to the kernel entry point.
10. The kernel command line is passed as a string. On the machine used above it contained rdinit=/process_api, which is what caused PID 1 to be that program rather than /sbin/init.
11. The initial ramdisk on modern Linux is an initramfs: a cpio archive, usually compressed with gzip, zstd or lz4, unpacked into a tmpfs. The older initrd was a block-device image. Both names are still used loosely.
12. PID 1 in practice: systemd on most Linux distributions since Fedora 15 in May 2011, Arch in October 2012, and Debian and Ubuntu from 2015; launchd on macOS since Mac OS X 10.4 Tiger in 2005, written by Dave Zarzycki; wininit.exe and smss.exe on Windows; init from BusyBox or OpenRC on smaller Linux systems.
13. Typical timings on a real x86-64 laptop, measured with systemd-analyze. These are approximate and vary widely by machine.

14. Commands that observe this: dmesg for kernel timestamps, systemd-analyze for the phase split, systemd-analyze blame for the slowest services, systemd-analyze critical-chain for the dependency path, bootctl for UEFI state, and efibootmgr -v for the boot entry list.
15. The honest version: the sequence described here starts at the main processor’s reset vector, and on modern x86 that is not the true beginning. Intel’s Management Engine and AMD’s Platform Security Processor run first, verify the firmware, and release the main cores. You cannot observe or disable that from the operating system.

power good
    |
    v
CPU reset vector  0xFFFFFFF0  (real mode)
    |
    v
firmware: POST, memory init, ACPI tables
    |
    v
boot device list -> ESP -> BOOTX64.EFI / GRUB
    |
    v
load vmlinuz + initramfs, fill boot_params
    |
    v
kernel: decompress, page tables, SMP, drivers
    |
    v
unpack initramfs -> find real root -> switch_root
    |
    v
exec PID 1 (systemd / launchd / init)
    |
    v
services -> display manager -> login prompt

WORDS18.3.6 remember these#

1. Bootstrapping — starting from nothing by stages — a chain loader sequence in which each stage loads and transfers control to the next.
2. Reset vector — the one address a CPU starts at — the architecturally fixed fetch address after reset, 0xFFFFFFF0 on x86-64.
3. Firmware — the program in a chip on the board — non-volatile platform code implementing BIOS or UEFI services.
4. POST — the switch-on self test — power-on self test, reporting faults by beep codes or diagnostic LEDs before video exists.
5. Bootloader — the program that finds the kernel — a chain-loading stage such as GRUB 2, Windows Boot Manager or boot.efi.
6. Initramfs — a tiny filesystem carried in memory — a compressed cpio archive unpacked into tmpfs, holding the drivers needed to mount the real root.
7. PID 1 — the first ordinary program — the init process, ancestor of all user processes; its exit causes a kernel panic.

18.4 Processes#

PLAIN18.4.1 in simple words#

1. A program is a file on disk. It does nothing. It is just bytes.
2. A process is that program while it is running: the code, plus its memory, plus its current position, plus everything the system is holding for it.
3. The same program can be running as many processes at once. Ten terminal windows are ten processes from one program file.
4. Each process gets a number, the process ID, usually written PID. It is how everything else refers to it.
5. Every process except the very first has a parent: the process that created it. That gives you a family tree with PID 1 at the root.
6. A process is not always running. Most of the time most processes are asleep, waiting for a key press, a disk read or a network packet.
7. When a process finishes, it does not vanish immediately. It leaves behind a small note saying how it ended, and waits for its parent to read it.
8. If the parent never reads that note, the dead process stays in the table as a zombie: dead, but still listed.
9. If a parent dies before its child, the child is adopted by PID 1. Such a child is called an orphan.

PLAIN18.4.2 a picture in your head#

1. Think of a cookery book on a shelf. That is the program.
2. Now think of you, in a kitchen, halfway through a recipe from it. That is the process.
3. The process is not the recipe. It is the recipe plus the ingredients on the bench, the pan on the heat, your finger on the current line, and the oven timer you set.
4. Your sister can cook the same recipe in a different kitchen at the same time. Same program, two processes, completely separate benches.
5. The doorbell rings. You mark your place, wipe your hands and answer it. When you come back you read your mark and continue. That is a context switch.
6. If you leave the kitchen for good, someone must still come and clear up and note whether the dish worked. Until they do, the mess stays on the bench. That is a zombie.

Where this comparison breaks: you have one pair of hands, so you genuinely stop cooking when you answer the door. A modern processor has several cores and can truly cook several dishes at once. And your memory of where you were is in your head; a process’s memory of where it was is written into a data structure that another program could, with enough privilege, read or alter.

PLAIN18.4.3 a worked example#

1. Here is a real program that creates a child and replaces the child with a different program. This is how every UNIX shell starts every command.

#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>
int main(void) {
    printf("before fork: pid=%d ppid=%d\n",
           getpid(), getppid());
    pid_t p = fork();
    if (p == 0) {
        printf("child : pid=%d ppid=%d fork gave %d\n",
               getpid(), getppid(), p);
        execl("/bin/echo", "echo", "child is now echo",
              NULL);
        _exit(127);
    }
    printf("parent: pid=%d fork gave %d\n", getpid(), p);
    int st; waitpid(p, &st, 0);
    printf("parent: child %d exited with %d\n",
           p, WEXITSTATUS(st));
    return 0;
}

2. Compiled with gcc and run, it printed this, exactly:

before fork: pid=5344 ppid=4984
child : pid=5346 ppid=5344 fork gave 0
child is now echo
parent: pid=5344 fork gave 5346
parent: child 5346 exited with 0

3. Read it carefully. fork was called once and returned twice, in two different processes.
4. In the child it returned 0. In the parent it returned 5346, the child’s PID. That is the only way each side can tell which one it is.
5. The child’s own PID is 5346 and its parent PID is 5344, which is the parent. The family link is real and visible.
6. Then execl replaced the child completely. Same PID 5346, entirely different program. The line “child is now echo” was printed by /bin/echo.
7. The parent waited, and collected the exit status 0.
8. One honest detail. When the same program’s output was sent into a pipe instead of a screen, the line child : pid=... never appeared at all. The C library buffers output when it is not a terminal, the text was still sitting in the buffer, and execl threw the whole buffer away when it replaced the process image. This is a real bug people hit; the fix is fflush before exec.

PLAIN18.4.4 what is really happening inside#

1. The kernel keeps one record per process. Textbooks call it the process control block. Linux calls it struct task_struct.
2. That record holds, at minimum: the process ID, the parent’s ID, the state, the saved register values, a pointer to the memory map, the table of open files, the owning user, the scheduling priority and the accounting counters.
3. On Linux 6.x struct task_struct is a few kilobytes and has well over a hundred fields. All processes live in a linked structure the kernel walks.
4. A process moves between a small number of states. There are five that matter.
5. New: the record exists but the process has not started running.
6. Ready: it could run right now, it just does not have a processor.
7. Running: it is executing on a processor this instant.
8. Blocked or waiting: it asked for something that has not arrived, such as a disk read, and cannot proceed.
9. Terminated: it has finished but the record has not yet been removed.
10. Only three transitions are interesting. Ready to running is the scheduler choosing it. Running to ready is the scheduler taking the processor away. Running to blocked is the process itself asking for something.
11. Note the asymmetry: blocked goes to ready, never straight to running. When your disk read finishes you become eligible; you do not jump the queue.

        admitted            scheduler picks it
 [New] ---------> [Ready] -------------------> [Running]
                     ^                            |  |
                     |  time slice ends           |  |
                     +----------------------------+  |
                     |                               |
                     |  event arrives                | exit
                [Blocked] <-- waits for I/O ---------+
                                                     |
                                                     v
                                              [Terminated]

12. Creating a process on UNIX is two steps, deliberately. fork makes a copy of the calling process. exec throws away the copy’s program and loads a new one in its place.
13. That looks wasteful and is not, because fork does not really copy the memory. It marks every page as shared and read-only, and only makes a real copy of a page when one side writes to it. This is copy on write.
14. Windows does it in one step instead. CreateProcess takes the program name and a pile of options and makes a new process running that program. There is no fork.
15. The two-step design is why UNIX shells are simple: between fork and exec the child can quietly rearrange its own file descriptors, and the new program inherits the arrangement without knowing anything about it.

TECHNICAL18.4.5 the engineer’s version#

1. Linux does not really have separate fork, vfork and thread creation. All three are clone() with different flag sets. fork is clone with SIGCHLD and nothing shared.
2. This trace, captured with strace on the shell command sh -c 'echo hello world | tr a-z A-Z', shows a real fork:

clone(child_stack=NULL,
      flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID
            |SIGCHLD,
      child_tidptr=0x7f26e1971a10) = 18232

3. Process states in ps on Linux, from the STAT column: R runnable or running, S interruptible sleep, D uninterruptible sleep (usually disk), T stopped, Z zombie, I idle kernel thread. Suffixes: s session leader, l multi-threaded, + foreground, < high priority, N low.
4. D state deserves special mention. A process in uninterruptible sleep cannot be killed, not even with signal 9, because it is inside a kernel path that cannot safely unwind. Persistent D state means broken storage or a hung network filesystem.
5. A real zombie, produced deliberately by forking a child that exits while the parent does not call wait:

  PID  PPID STAT COMMAND
20066 20065 Z    zombie

6. A real orphan, produced by a parent that exits before its child. The child printed its parent ID twice, one second apart:

parent 23414: exiting immediately
child 23415: ppid is 23414
child 23415: ppid is now 1

7. Reparenting to PID 1 is the kernel’s rule. On systemd systems a process can instead register itself as a subreaper with prctl(PR_SET_CHILD_SUBREAPER), and orphans go to it rather than to PID 1.
8. PID limits: the default maximum on 64-bit Linux is 4,194,304, set in /proc/sys/kernel/pid_max. The historical default of 32,768 came from a 16-bit signed field. PIDs wrap around and are reused, which is a real source of race conditions in scripts that kill by PID.
9. Per-process limits are visible in /proc/PID/limits. On the machine used here:

10. /proc/PID/stat is the raw record, 52 space-separated fields. The first seven are pid, comm, state, ppid, pgrp, session, tty. Field 14 and 15 are user and system jiffies consumed. ps and top read exactly this file.
11. Windows: CreateProcessW takes 10 parameters including STARTUPINFO and PROCESS_INFORMATION. A Windows process starts with one thread and a PEB (process environment block). Handles are inherited only if marked inheritable and bInheritHandles is true.
12. Observation tools by platform: ps aux, ps -ef, pstree, top, htop and /proc on Linux; ps, top and Activity Monitor on macOS; dtruss and sample on macOS; Task Manager, Process Explorer, Process Monitor and Get-Process on Windows.
13. Useful ps incantations. ps -eo pid,ppid,stat,pri,ni,etime,comm prints the fields this section talks about. ps -eLf shows one line per thread. ps -eo pid,rss,vsz,comm --sort=-rss finds memory hogs.

WORDS18.4.6 remember these#

1. Program — a file of instructions — a passive executable image on storage.
2. Process — a program while running — an execution instance with its own address space, descriptor table, credentials and scheduling state.
3. Process control block — the kernel’s record of a process — task_struct on Linux, EPROCESS on Windows.
4. PID — a process’s number — a reusable integer identifier, capped by /proc/sys/kernel/pid_max.
5. fork — make a copy of yourself — clone() creating a new address space via copy-on-write page sharing.
6. exec — become a different program — replace the current process image while keeping the PID and inherited descriptors.
7. Zombie — dead but still listed — a terminated process whose exit status has not yet been reaped by wait.
8. Orphan — a child whose parent died — a process reparented to PID 1 or to the nearest subreaper.
9. Copy on write — pretend to copy, copy only if changed — shared read-only page mappings duplicated lazily on the first write fault.

18.5 Threads#

PLAIN18.5.1 in simple words#

1. A thread is one flow of execution: one finger moving down the list of instructions.
2. A process has at least one thread. It may have many.
3. All the threads in one process share the same memory. If one writes to a variable, the others see the change instantly.
4. Each thread has its own stack, because each thread is in a different place in the program and has its own local variables and its own chain of calls.
5. That is the entire difference. Shared memory, separate position and stack.
6. Threads are used when several parts of one job must happen at once: one thread drawing the screen, another loading a file, another talking to the network.
7. Because they share memory, threads are fast to create and cheap to talk between.
8. Because they share memory, one bad thread can corrupt the data of all the others, and there is no protection at all.
9. That trade is the whole story of threads: speed and sharing, bought with danger.

PLAIN18.5.2 a picture in your head#

1. A process is a flat with a locked front door. A thread is a person living in it.
2. Two flats cannot see into each other. That is process isolation, enforced by the lock.
3. Two flatmates share the kitchen, the fridge and every cupboard. Neither needs permission to open anything. That is thread sharing.
4. Each flatmate still has their own bedroom with their own things. That is the per-thread stack.
5. If one flatmate takes the milk while the other is pouring it, you get a mess. Nobody broke a rule. They just both used one thing at once. That is a race condition.
6. Moving into a spare bedroom is quick. Getting a whole new flat takes longer and costs more. That is why threads are cheaper than processes.

Where this comparison breaks: flatmates can talk and agree who uses the kitchen. Threads cannot agree about anything unless you write the agreement yourself, in the form of locks. And a flat has one kitchen; a multi-core processor gives each thread its own set of caches, so two threads can genuinely believe different things about the same variable until the caches are made to agree.

PLAIN18.5.3 a worked example#

1. Measured on the machine used for this chapter, a 2.10 GHz Intel Xeon with two cores, Linux 6.18, glibc on Ubuntu 24.04.
2. One thousand threads created, then all joined. One thousand processes forked, then all reaped.

pthread_create only        :    32.8 us
fork only (small process)  :   182.5 us
fork only (256 MB dirty)   :  2620.1 us

3. Creating a thread cost 32.8 microseconds. Creating a process cost 182.5 microseconds. The thread is about 5.6 times cheaper.
4. Now the third line. The same fork, after the parent had allocated and written to 256 MB of memory, cost 2,620 microseconds. Fourteen times more.
5. Why: copy-on-write does not copy the data, but it must still copy the page tables that describe it. 256 MB is 65,536 pages of 4 KB, and every entry must be duplicated and marked read-only.
6. The honest version: “a thread is far cheaper than a process” is only true sometimes. Fork’s cost depends on how much memory the parent has mapped. Thread creation does not depend on that at all.
7. A separate measurement, thread create and join one at a time, gave 80.2 microseconds against 134.3 for fork and wait. The gap narrows because the join and the wait dominate.
8. Take the lesson, not the numbers: threads are cheaper, but on Linux the gap is a factor of a few, not a factor of a thousand. On Windows the gap is larger, because Windows process creation is genuinely heavier.

PLAIN18.5.4 what is really happening inside#

1. Here is exactly what is shared between two threads of one process, and what is not.
2. Shared: the code, the global variables, the heap, the open file table, the current directory, the user identity, the signal handlers.
3. Not shared: the stack, the register values including the program counter, the thread ID, the signal mask, and any thread-local storage.
4. When the kernel switches from thread A to thread B in the same process, it saves A’s registers and loads B’s, and that is nearly all.
5. When it switches between threads of different processes, it must also switch the memory map, which means loading a new page table root and, on older processors, flushing the address translation cache.
6. That flush is why cross-process switches cost more than same-process ones. Modern processors avoid most of it with tagged translation caches: PCID on x86-64 since Westmere in 2010, ASID on ARM.
7. There are two places threads can be implemented.
8. Kernel-level threads: the kernel knows about each one and schedules them individually. If one blocks on disk, the others keep running. This is what Linux, Windows and macOS do.
9. User-level threads: a library inside your process switches between them, and the kernel sees only one thread. Switching is very fast because no mode change happens. But if one blocks in the kernel, all of them stop.
10. Modern languages have revived the user-level idea under new names: goroutines in Go, virtual threads in Java 21, async tasks in Rust and Python. They pair it with a runtime that never blocks the underlying kernel thread, which removes the old flaw.
11. A thread pool exists because creating a thread costs tens of microseconds and most tasks are shorter than that. You create a fixed set of threads once and feed them work from a queue for the life of the program.

TECHNICAL18.5.5 the engineer’s version#

1. The POSIX threads API is IEEE Std 1003.1c-1995, universally called pthreads. The core calls are pthread_create, pthread_join, pthread_detach, pthread_mutex_lock and pthread_cond_wait.
2. On Linux a pthread is created by clone() with CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SETTLS. The kernel calls the result a task; there is no separate thread object.
3. Because of CLONE_THREAD, all threads of a process share a thread group ID, which is the value getpid() returns. The per-thread identifier is the TID, returned by gettid(). ps -eLf shows both as PID and LWP.
4. Default thread stack size on glibc is the process stack rlimit, typically 8 MiB, reserved as virtual address space and committed lazily. On musl it is 128 KiB. On Windows the default reserve is 1 MiB with 4 KiB committed.
5. That 8 MiB reservation is why you cannot have a million pthreads on a 64-bit system with default settings and a modest vm.max_map_count, even though the memory is never touched.
6. Threading models, in the classic taxonomy: 1:1 (one kernel thread per user thread; Linux NPTL, Windows), N:1 (all user threads on one kernel thread; green threads), M:N (a runtime multiplexes M user threads onto N kernel threads; Go, Erlang, Java 21 virtual threads).
7. Linux abandoned M:N deliberately. NPTL, the Native POSIX Thread Library by Ulrich Drepper and Ingo Molnár, shipped in glibc 2.3.2 in 2003 and replaced LinuxThreads with a strict 1:1 model, because the kernel scheduler was made good enough that a userspace one added nothing.

8. Thread pool sizing rules of thumb. CPU-bound work: pool size equal to the core count. I/O-bound work: cores multiplied by one plus the ratio of wait time to compute time. Both are heuristics, not laws.
9. Real pool defaults: Java’s ForkJoinPool.commonPool uses cores minus one; .NET’s ThreadPool starts at the core count and grows by one thread every 500 ms under starvation; nginx uses one worker process per core with an event loop rather than a thread pool.
10. Observation: ps -eLf, top -H, htop with thread view on Linux; /proc/PID/task/ lists one directory per thread; perf sched shows switch events; Windows Performance Analyzer and Process Explorer on Windows.

WORDS18.5.6 remember these#

1. Thread — one flow through the code — a schedulable execution context with its own stack and register set.
2. Thread group — the threads of one process — tasks sharing a TGID, which is what getpid() returns on Linux.
3. Race condition — two threads touching one thing at once — an outcome that depends on unsynchronized interleaving of accesses.
4. Thread pool — reuse threads instead of making new ones — a fixed set of worker threads fed by a task queue.
5. 1:1 model — one kernel thread per program thread — the NPTL and Windows model, where the kernel schedules every thread directly.
6. M:N model — a runtime shuffles many onto few — user-space scheduling of green threads onto a smaller pool of kernel carriers.

18.6 Scheduling#

PLAIN18.6.1 in simple words#

1. Your machine seems to run fifty programs at once. If it has eight cores, at most eight instructions are actually being executed at any instant.
2. The trick is speed. The system gives each program a tiny slice of time, then switches to the next, and goes round and round.
3. Switch fast enough and it looks continuous, in the same way that separate photographs shown quickly look like a moving picture.
4. The part of the operating system that decides who goes next is the scheduler.
5. Its job is to answer one question, over and over, thousands of times a second: of everything that could run right now, which one runs next, and for how long.
6. There is no perfect answer, because the goals fight each other.
7. You want the machine to be busy. You want short jobs to finish quickly. You want the mouse pointer to move the instant you move the mouse. You want no program to be forgotten forever.
8. A scheduler that is good at one of those is usually worse at another. Every real scheduler is a compromise that someone chose.

PLAIN18.6.2 a picture in your head#

1. Picture a doctor’s surgery with one doctor and a waiting room.
2. First come first served: strictly in arrival order. Perfectly fair, and one person with a forty-minute problem makes everyone else late.
3. Shortest job first: call in whoever will be quickest. The waiting room empties fastest. Someone with a long problem may sit there all day.
4. Round robin: everyone gets five minutes, then goes to the back of the queue if they are not finished. Nobody waits forever. Everyone comes back several times, and each return wastes a minute settling in.
5. Priority: emergencies first. Right, and if emergencies keep arriving the ordinary patient never gets seen. That is starvation.
6. Multilevel feedback: several waiting rooms. Everyone starts in the fast one. If you use your whole slot without finishing you are moved to a slower room with longer slots. Short visits stay fast, long ones do not clog things.

Where this comparison breaks: the receptionist knows roughly how long each appointment will take. A scheduler does not, and cannot: predicting how long a program will run is equivalent to solving the halting problem. Every real scheduler that claims to run the shortest job first is in fact guessing from recent history. And the doctor takes a minute to switch patients; a processor switches in about a microsecond, which changes every calculation.

PLAIN18.6.3 a worked example#

1. Four jobs arrive at a single core. Times are in arbitrary equal units.

2. Total work is 16 units, so whatever we do, the last job finishes at time 16. Only the waiting changes.
3. First come first served. A runs 0 to 7, B 7 to 11, C 11 to 12, D 12 to 16.
4. Waiting times: A 0, B 5, C 7, D 7. Average wait 4.75.
5. Shortest job first, without interrupting. A runs 0 to 7 because it is the only one there. At 7 the choices are B (4), C (1), D (4). C is shortest, so C runs 7 to 8, then B 8 to 12, then D 12 to 16.
6. Waiting times: A 0, B 6, C 3, D 7. Average wait 4.00.
7. Shortest remaining time first, which is the same idea but allowed to interrupt. A starts. At time 2 B arrives needing 4, A has 5 left, so B takes over. At 4 C arrives needing 1, B has 2 left, so C takes over and finishes at
   5. B then finishes 5 to 7. D runs 7 to 11. A finally finishes 11 to 16.
8. Waiting times: A 9, B 1, C 0, D 2. Average wait 3.00. Best of the four, and A paid for it.
9. Round robin with a slice of 2 units. New arrivals join the back of the queue before a job that has just been interrupted.

10. Waiting times: A 9, B 3, C 2, D 6. Average wait 5.00.
11. Put the four side by side.

12. Round robin came last on both averages. It is still what real interactive systems use, because averages are not what a human notices.
13. What a human notices is that under round robin, C started 2 units after it arrived and B started immediately. Under first come first served, C sat still for 7 units. Responsiveness is not in the average.

PLAIN18.6.4 what is really happening inside#

1. A context switch is the act of taking the processor away from one thread and giving it to another.
2. Here is exactly what gets saved, in order.
3. The general-purpose registers. On x86-64 that is 16 of them, 8 bytes each.
4. The program counter, so the thread resumes at the right instruction. On x86-64 this is already on the kernel stack, pushed by the interrupt.
5. The flags register, which holds the results of the last comparison.
6. The stack pointer.
7. The floating point and vector register state. On x86-64 with AVX-512 this is the big one: over 2,500 bytes. Modern kernels defer this and only save it if the new thread actually uses those registers.
8. Then, if the new thread belongs to a different process, the page table root register is reloaded, which changes what memory is visible.
9. The direct cost of all that is roughly one to three microseconds.
10. The indirect cost is larger and invisible: the new thread finds the caches full of the old thread’s data, and runs slowly until it has refilled them. Measured cache-warming costs of 10 to 100 microseconds are common.
11. This is why a scheduler that switches too often makes a machine slower even though it looks fairer.
12. The time slice or quantum is how long a thread may run before the scheduler is asked again. Typical values are 1 to 10 milliseconds, which is a thousand to ten thousand times the switch cost.
13. Preemptive multitasking means the system can take the processor away whether or not the program agrees. This requires the timer interrupt.
14. Cooperative multitasking means a program keeps the processor until it chooses to give it up. One program with an infinite loop freezes everything.
15. Cooperative was not a theory. Classic Mac OS up to version 9 and Windows 3.x both worked that way, and both froze exactly as described.

TECHNICAL18.6.5 the engineer’s version#

1. Scheduling metrics, defined precisely. Turnaround time is completion minus arrival. Waiting time is turnaround minus service time. Response time is first-run minus arrival. Throughput is completed jobs per unit time.
2. Linux CFS, the Completely Fair Scheduler, was written by Ingo Molnár, inspired by Con Kolivas’s Rotating Staircase Deadline scheduler, and merged in Linux 2.6.23 in October 2007.
3. CFS abolished fixed time slices. It tracked vruntime, virtual runtime, per task, scaled by the task’s weight, and always ran the task with the smallest vruntime. The ready set was a red-black tree, so picking the next task was O(log n).
4. Nice values map to weights by a table where each step of 1 changes CPU share by about 10 percent, and nice 0 has weight 1024. Nice ranges from -20 to +19.
5. EEVDF replaced CFS in Linux 6.6, released 29 October 2023. The name is Earliest Eligible Virtual Deadline First, from a 1995 paper by Ion Stoica and Hussein Abdel-Wahab.
6. EEVDF gives each task a virtual deadline computed from its requested slice and its lag, and runs the eligible task with the earliest deadline. It handles latency-sensitive tasks properly, which CFS could only do through a growing pile of heuristics and out-of-tree patches.
7. Linux scheduling classes, in strict priority order: SCHED_DEADLINE, then the real-time classes SCHED_FIFO and SCHED_RR, then SCHED_OTHER and SCHED_BATCH, then SCHED_IDLE. Verified on the machine used here with chrt -m:

SCHED_OTHER min/max priority  : 0/0
SCHED_FIFO  min/max priority  : 1/99
SCHED_RR    min/max priority  : 1/99
SCHED_BATCH min/max priority  : 0/0
SCHED_IDLE  min/max priority  : 0/0
SCHED_DEADLINE min/max        : 0/0

8. A SCHED_FIFO task at priority 99 that never blocks will hang the machine, or would if not for the throttle: sched_rt_runtime_us defaults to 950000 out of sched_rt_period_us of 1000000, so real-time tasks are capped at 95 percent of each second.
9. SCHED_RR round-robin slice on this machine, from /proc/sys/kernel/sched_rr_timeslice_ms, is 100 ms.
10. Windows uses 32 priority levels: 0 is the zero-page thread, 1 to 15 are dynamic, 16 to 31 are real-time. The level is computed from a process priority class and a thread priority offset.

11. Windows boosts a thread’s priority when its wait completes, and boosts the foreground window’s thread, then decays the boost by one level per quantum. Quantum on Windows client is 2 clock intervals, roughly 20 to 30 ms; on Server it is 12 intervals, roughly 120 to 180 ms, favouring throughput.
12. macOS uses a multilevel feedback queue with four bands: normal, system high priority, kernel mode only, and real-time. It also exposes Grand Central Dispatch quality-of-service classes, from QOS_CLASS_USER_INTERACTIVE down to QOS_CLASS_BACKGROUND, which map onto scheduler parameters and, on Apple silicon, onto the choice of performance or efficiency core.
13. Real context switch counts. On the machine used here, /proc/stat reported ctxt 4153017 after 95 minutes of light use, roughly 730 switches per second across two cores.
14. Per-process counters live in /proc/PID/status as voluntary_ctxt_switches (the task blocked) and nonvoluntary_ctxt_switches (the scheduler preempted it). A high nonvoluntary count means CPU contention; a high voluntary count means I/O.
15. Observation and control: chrt to read and set policy, nice and renice for weights, taskset for CPU affinity, perf sched latency and perf sched map for switch-level traces, pidstat -w for switch rates, vmstat 1 for the cs column, and schedtool for everything at once.
16. Where experts disagree: whether a general-purpose kernel should ever adopt a pluggable scheduler interface. Linux merged sched_ext, which allows BPF programs to implement schedulers, in Linux 6.12 in November 2024, after years of objection that it would fragment behaviour. Both sides had a real point.

WORDS18.6.6 remember these#

1. Scheduler — the chooser of who runs next — the kernel component implementing a policy over the set of runnable tasks.
2. Context switch — swapping one thread for another — saving and restoring architectural state, and possibly the address space.
3. Quantum — how long you get before being asked to stop — the scheduling time slice, typically 1 to 10 ms on desktop systems.
4. Preemption — being interrupted whether you like it or not — involuntary removal of a running task, driven by the timer interrupt.
5. Starvation — never getting a turn — indefinite postponement of a low-priority task by a stream of higher-priority ones.
6. Priority inversion — a low task blocking a high one — a high-priority task waiting on a lock held by a low-priority task; fixed by priority inheritance.
7. CFS — Linux’s fair scheduler from 2007 — weight-proportional virtual runtime scheduling over a red-black tree.
8. EEVDF — its 2023 replacement — earliest eligible virtual deadline first, merged in Linux 6.6.

18.7 System calls#

PLAIN18.7.1 in simple words#

1. Your program cannot open a file. It genuinely cannot. The instruction that would talk to the disk is refused in user mode.
2. So it asks the operating system to do it. That request is a system call.
3. A system call is not a function call into a library. It is a controlled jump across the boundary between your program and the kernel.
4. There are only a few hundred of them, and they are the complete list of everything your program can ask the machine to do.
5. Everything else your program does — arithmetic, loops, string handling — needs no permission and never crosses the boundary.
6. So the full power of a program is: any computation it likes, plus this fixed menu of requests.
7. When people say a language “can do system programming”, they mean it can make these calls directly rather than through several layers.
8. Every file you open, every byte you send over the network, every window you draw and every key you read passes through this menu.

PLAIN18.7.2 a picture in your head#

1. Think of a bank counter with thick glass and a small hatch.
2. You are outside. The money is inside. You cannot reach it.
3. There is a printed list of things you may ask for: withdraw, deposit, balance, transfer. Numbered 1 to 40.
4. You fill in a slip: request number, then the details in fixed boxes. You push it through the hatch.
5. The clerk checks the slip, does the work inside where you cannot see, and pushes back a result: either what you asked for, or a refusal code.
6. You cannot ask for anything not on the list. You cannot go behind the counter. You cannot see how the vault works.
7. Filling in a slip and waiting takes longer than counting your own coins. So you learn to ask for a lot at once rather than making forty small requests.

Where this comparison breaks: the clerk is a separate person, whereas the kernel runs on the very same processor as your program, just in a different mode. And the bank clerk cannot look inside your wallet; the kernel can read and write every byte your program owns, and does, in order to fill in your answers. The boundary protects the kernel from you. It does not protect you from the kernel.

PLAIN18.7.3 a worked example#

1. Here is a complete C program:

#include <stdio.h>
int main(void) { printf("hello\n"); return 0; }

2. Run under strace, which prints every system call, it made exactly these, with the boring middle removed:

execve("./hello", ["./hello"], 0x7ffd... ) = 0
brk(NULL)                               = 0x5587a3d30000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=53283}) = 0
mmap(NULL, 53283, PROT_READ, MAP_PRIVATE, 3, 0) = ...
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6",
       O_RDONLY|O_CLOEXEC)              = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0..."..., 832) = 832
mmap(NULL, 2170256, PROT_READ, MAP_PRIVATE, 3, 0) = ...
mmap(..., PROT_READ|PROT_EXEC, MAP_FIXED, 3, 0x28000)
close(3)                                = 0
arch_prctl(ARCH_SET_FS, 0x7f22105ce740) = 0
set_tid_address(0x7f22105cea10)         = 18134
mprotect(0x7f22103ff000, 16384, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024})
fstat(1, {st_mode=S_IFIFO|0600, st_size=0}) = 0
getrandom("\x8a\x08\xd7\x55...", 8, GRND_NONBLOCK) = 8
brk(0x5587a3d51000)                     = 0x5587a3d51000
write(1, "hello\n", 6)                  = 6
exit_group(0)                           = ?

3. Now every line, in order.
4. execve is the call that loaded this program. It is the first line of every trace because the trace starts in the shell, before the program exists.
5. brk(NULL) asks where the heap currently ends. It is a query, not a change.
6. access("/etc/ld.so.preload") returned -1 ENOENT, meaning the file does not exist. That is not an error; the dynamic linker is checking for an optional override and finding none.
7. openat on /etc/ld.so.cache returned 3. That number is a file descriptor: the handle you use for every later operation on that file. 0, 1 and 2 are already taken by standard input, output and error.
8. fstat(3, ...) asks how big it is: 53,283 bytes.
9. mmap maps those bytes into memory rather than reading them, so the library cache can be looked at as an array.
10. close(3) releases the descriptor. Descriptor 3 is now free again, which is why the very next openat also returns 3.
11. The second openat finds the C library itself, all 2,125,328 bytes of it.
12. read(3, "\177ELF...", 832) reads the first 832 bytes: the ELF header. The first four bytes are 0x7F then ELF, the magic number that identifies the file format.
13. Then four mmap calls place the library into memory with different permissions: read-only for constants, read plus execute for code, read plus write for data.
14. arch_prctl(ARCH_SET_FS, ...) sets the FS segment base, which is how thread-local storage is found on x86-64.
15. set_tid_address tells the kernel where to clear a word when this thread dies, which is how pthread_join learns the news.
16. mprotect(..., PROT_READ) makes the relocation table read-only after it has been filled in. That is a hardening measure called RELRO.
17. prlimit64 reads the stack limit: 8,192 times 1,024 bytes, that is 8 MiB.
18. fstat(1, ...) checks what standard output is. Here it reported S_IFIFO, a pipe. That is why the C library chose full buffering rather than line buffering.
19. getrandom(..., 8, ...) fetches 8 random bytes for the stack canary, the guard value that detects buffer overflows.
20. brk grows the heap by 132 KiB for the output buffer.
21. write(1, "hello\n", 6) is the one call the program actually meant to make. Six bytes to descriptor 1. It returned 6, meaning all six were accepted.
22. exit_group(0) ends every thread of the process with status 0.
23. Count them: about twenty-five system calls to print one word, and only one of them was the point. The rest is the cost of dynamic linking.

PLAIN18.7.4 what is really happening inside#

1. Mechanically, on a 64-bit Intel or AMD processor, a system call is this.
2. The program puts a number in the register rax. That number says which service it wants. write is 1. read is 0. openat is 257.
3. It puts the arguments in registers, in a fixed order: rdi, rsi, rdx, r10, r8, r9. Up to six.
4. It executes one instruction: syscall.
5. The processor then does several things at once, in hardware. It saves the return address into rcx and the flags into r11. It switches to privilege ring 0. It loads the instruction pointer from a control register that only the kernel could write.
6. That control register, MSR_LSTAR, was set during boot to the address of the kernel’s entry stub. So the program lands exactly where the kernel decided, and nowhere else.
7. The kernel stub switches to a kernel stack, saves the rest of the registers, checks that the number in rax is within range, and calls the function in the system call table at that index.
8. The function does the work. It may block, in which case the scheduler picks somebody else and this thread resumes here later.
9. The result goes into rax. Errors are returned as small negative numbers.
10. The kernel restores registers and executes sysret, which puts the processor back into ring 3 at the address in rcx.
11. The C library then looks at rax. If it is between -1 and -4095, the library stores its negation in errno and returns -1 to you.
12. That is why C code checks for -1 and then reads errno. The kernel never set errno; the library did.
13. Cost, measured on the machine used here: a bare getpid system call took 0.089 microseconds, that is 89 nanoseconds, about 187 cycles at 2.10 GHz.
14. That is genuinely expensive next to an ordinary function call of one or two nanoseconds, and cheap next to a disk read of 100 microseconds.
15. Some calls are so common that the kernel arranges for them to need no boundary crossing at all. It maps a small shared page called the vDSO into every process, containing code and data that answer them in user mode.
16. Measured on this machine: clock_gettime through the vDSO took 0.030 microseconds; forced through the real system call it took 0.144 microseconds. Nearly five times cheaper for doing the same thing.

TECHNICAL18.7.5 the engineer’s version#

1. The x86-64 Linux calling convention for system calls: number in rax, args in rdi, rsi, rdx, r10, r8, r9, return in rax. Note r10 rather than rcx, because syscall overwrites rcx with the return address.
2. On AArch64 Linux: number in x8, args in x0 to x5, instruction svc #0, return in x0. On RISC-V: number in a7, args a0 to a5, instruction ecall.
3. Older x86 mechanisms, in historical order: int 0x80 software interrupt (slow, about 1,000 cycles), sysenter and sysexit from Pentium II, syscall and sysret from AMD64. Windows used int 0x2E before switching to sysenter.
4. Linux has 300 to 460 system calls depending on architecture and version. Verified numbers from asm/unistd_64.h on this machine:

5. Grouped by purpose, the ones worth memorizing:

6. ioctl is the escape hatch: a single call with a device-specific command number, used for everything that does not fit the read and write model. It is widely regarded as an ugly but necessary design.
7. Counting calls with strace -c on ls /tmp on this machine gave 76 calls total, 5 of which returned errors. The largest single group was 17 mmap calls, all from dynamic linking.
8. Windows equivalents are in ntdll.dll as Nt or Zw functions: NtCreateFile, NtReadFile, NtWriteFile, NtAllocateVirtualMemory. The documented API you are meant to call is Win32, in kernel32.dll, which calls these. Microsoft does not guarantee the numbers, and they change between builds, which is why malware that hardcodes them breaks after updates.
9. macOS syscall numbers are split into classes by the top byte: class 1 is Mach traps, class 2 is BSD calls. write is 0x2000004.
10. Reducing the cost: io_uring, merged in Linux 5.1 in May 2019, replaces per operation calls with two shared ring buffers, allowing batched submission and completion with, in the best case, zero system calls per operation.
11. The security cost: Meltdown and Spectre, disclosed 3 January 2018, forced kernel page-table isolation, which roughly doubled or tripled system call cost on affected processors. Measured overheads of 5 to 30 percent on system call heavy workloads were common in 2018.
12. Tools: strace and ltrace on Linux, dtruss and dtrace on macOS (which require disabling System Integrity Protection for many targets), Process Monitor and Event Tracing for Windows on Windows, and bpftrace on modern Linux for anything the others cannot do.

WORDS18.7.6 remember these#

1. System call — asking the kernel for something — a controlled mode transition into a numbered kernel service routine.
2. File descriptor — a small number naming an open thing — an index into the per-process open file descriptor table.
3. errno — the reason the last request failed — a thread-local integer set by the C library from the kernel’s negative return value.
4. vDSO — a shortcut that skips the kernel — a kernel-provided shared object mapped into every process, serving calls like clock_gettime in user mode.
5. ioctl — the request that covers everything else — a device-specific control call carrying an opaque command number and argument.
6. io_uring — batching requests instead of asking one at a time — shared submission and completion ring buffers, in Linux since 5.1.

18.8 Memory management from the operating system’s side#

PLAIN18.8.1 in simple words#

1. Chapter 11 covered memory in depth. One line to bring it back: every program sees its own private set of addresses starting at zero, and the hardware translates those into real addresses in the physical chips.
2. That translation is done by a unit in the processor using tables the kernel builds. The kernel owns those tables. No program can edit its own.
3. Here we care about the part the kernel does, which is three jobs.
4. Job one: hand out page frames. Physical memory is cut into fixed blocks, usually 4 KB. The kernel keeps track of which are used and which are free.
5. Job two: decide what to do when there is not enough. Either free something, or write something to disk, or kill a program.
6. Job three: enforce limits, so one program cannot take everything.
7. A surprising fact: the kernel routinely promises more memory than it has.
8. It does that because programs ask for far more than they use, and most of the promise is never called in.
9. When the promise is called in and the memory does not exist, something has to die. The kernel picks a victim and kills it.

PLAIN18.8.2 a picture in your head#

1. Think of an airline selling seats on a plane with 180 seats.
2. Experience says about 5 percent of people never turn up. So the airline sells 189 tickets. That is overcommit.
3. Almost always it works and the plane goes out full. Occasionally everyone turns up, and someone gets removed from the flight.
4. The airline does not remove people at random. It has a rule: latest booking, cheapest fare, travelling alone.
5. The kernel does the same. It has a scoring rule and it kills the highest scorer. That code is called the out of memory killer.
6. Memory the program asked for but never touched is a ticket that was never used. The kernel only allocates a real page frame the first time you actually write to the address.

Where this comparison breaks: the airline can offer you money to take a later flight. The kernel cannot negotiate. It sends signal 9 and the process ends instantly with no chance to save anything. And an airline’s overbooking is calculated from real statistics; Linux’s default overcommit setting is a rough heuristic that guesses, and it can be switched off entirely.

PLAIN18.8.3 a worked example#

1. A program calls malloc(1024 * 1024 * 1024), asking for one gigabyte, on a machine with 8 GB of RAM and 6.7 GB free.
2. The C library sees a large request and calls mmap rather than brk.
3. The kernel checks its overcommit policy. The default on this machine, from /proc/sys/vm/overcommit_memory, is 0: the heuristic mode. It agrees.
4. The kernel records a mapping one gigabyte long in the process’s list of memory regions and returns a pointer. Not one byte of physical memory has been allocated.
5. free reports no change. The program’s virtual size, VSZ in ps, jumps by 1 GB. Its resident size, RSS, does not move.
6. The program writes one byte at the start. The processor tries to translate that address, finds no page table entry, and raises a page fault.
7. The kernel handles the fault, takes one free 4 KB frame, zeroes it, points the table entry at it, and restarts the instruction. RSS goes up by 4 KB.
8. Only when the program walks the whole gigabyte does the kernel hand over 262,144 frames and the memory is really consumed.
9. Now suppose it does that when only 200 MB is free. The kernel first tries to reclaim: it drops cached file pages, and swaps out unused anonymous pages.
10. If that is not enough, the out-of-memory killer runs. It reads a score for every process and kills the highest.
11. On this machine the score for a small shell process was 666 out of 1000, from /proc/self/oom_score, and the tunable /proc/self/oom_score_adj was 0. Setting the adjustment to -1000 makes a process immune.
12. In the kernel log you would then see a line beginning Out of memory: Killed process 1234 (chrome), followed by a full report of who had what.

PLAIN18.8.4 what is really happening inside#

1. The kernel divides physical memory into page frames and keeps one small record per frame. On Linux that is struct page, about 64 bytes each.
2. For 8 GB of RAM with 4 KB pages, that is 2,097,152 frames and about 128 MB of records. The kernel’s own bookkeeping costs about 1.6 percent of memory.
3. Free frames are managed by the buddy allocator: free memory is held in lists of blocks of 1, 2, 4, 8 up to 1024 pages. Splitting and merging is done by halving and pairing, which keeps fragmentation manageable.
4. Small kernel objects do not each take a page. A second allocator, the slab allocator, carves pages into same-sized objects with free lists. You can see it in /proc/slabinfo.
5. Memory that holds file contents is not wasted. The page cache keeps recently read file data in otherwise free memory and hands it back instantly when reclaimed. That is why “free memory” on a healthy Linux box is near zero and this is correct, not a problem.
6. When memory runs short the kernel walks its lists of pages and decides what to evict, approximately least-recently-used. Clean file pages are cheapest to drop, because the copy on disk is still valid. Dirty pages must be written first. Anonymous pages, which have no file, must go to swap.
7. If there is no swap and nothing left to reclaim, the only remaining action is to kill something.
8. The out-of-memory score is roughly proportional to the process’s resident memory plus its swap use, expressed out of 1000, then adjusted by oom_score_adj. The biggest user usually dies. That is deliberate: killing the biggest recovers the most memory for the fewest deaths.

TECHNICAL18.8.5 the engineer’s version#

1. Linux overcommit modes, in /proc/sys/vm/overcommit_memory:

2. In mode 2, the limit is swap plus overcommit_ratio percent of RAM. The default ratio is 50, verified on this machine. With 8 GB and no swap, that allows only 4 GB of committed address space in total, which is why mode 2 is rarely used without tuning.
3. Windows does not overcommit in the Linux sense. It maintains a commit charge against a commit limit equal to RAM plus the pagefile, and VirtualAlloc with MEM_COMMIT fails when the limit is reached. The failure is returned to the program rather than delivered later as a kill.
4. That is a genuine design difference with consequences. Linux gives you better memory utilization and a risk of sudden death; Windows gives you honest allocation failures and requires a large pagefile.
5. Page sizes: 4 KiB base on x86-64 and on Linux ARM64 by default; 2 MiB and 1 GiB huge pages on x86-64; 16 KiB base on Apple silicon macOS and on iOS. Transparent Huge Pages on Linux promotes eligible 2 MiB regions automatically and can be controlled per process with madvise.
6. Per-process limits come from setrlimit and are visible in /proc/PID/limits. RLIMIT_AS caps total address space, RLIMIT_DATA the heap, RLIMIT_STACK the main stack (8 MiB soft on this machine), RLIMIT_MEMLOCK how much may be pinned.
7. Cgroups give stronger limits than rlimits, because they apply to a group and are enforced by the reclaim path. In cgroup v2, memory.max is a hard cap that triggers cgroup-local OOM kill, memory.high is a throttling threshold, and memory.low protects a floor during reclaim.
8. Useful counters, all real files: /proc/meminfo for the global picture, /proc/PID/status for VmRSS, VmSize and VmSwap, /proc/PID/smaps for per-mapping detail, and /proc/pressure/memory for PSI stall percentages, which is the single best early warning of memory trouble.
9. Reading /proc/meminfo on this machine gave MemTotal: 8216168 kB, MemFree: 6709900 kB, MemAvailable: 7407208 kB and Cached: 847024 kB. MemAvailable is the number to trust; MemFree ignores reclaimable cache.
10. Tools: free -m, vmstat 1, smem, ps -eo pid,rss,vsz --sort=-rss, pmap, slabtop, and dmesg | grep -i "killed process" to find OOM events after the fact. On macOS, vm_stat, footprint and Activity Monitor’s memory pressure graph.

WORDS18.8.6 remember these#

1. Page frame — one fixed block of real memory — a physical page, usually 4 KiB on x86-64.
2. Page fault — the translation was missing — a trap raised when a virtual address has no valid mapping, handled by the kernel.
3. Overcommit — promising more than you have — allowing total mapped address space to exceed RAM plus swap.
4. OOM killer — the code that picks who dies — the out-of-memory handler that selects and SIGKILLs the highest scoring process.
5. Page cache — free memory holding file data — the unified cache of file-backed pages, reclaimable without I/O when clean.
6. Buddy allocator — splitting and pairing free blocks — the power-of-two physical page allocator underneath the kernel’s memory subsystem.

18.9 The filesystem from the operating system’s side#

PLAIN18.9.1 in simple words#

1. A disk stores numbered blocks. It knows nothing about files, names or folders. Those are entirely the operating system’s invention.
2. The filesystem is the code that turns “a file called notes.txt in a folder called work” into “blocks 41,922 to 41,930 on this disk”.
3. There are dozens of filesystems: ext4, NTFS, APFS, XFS, Btrfs, FAT32, ZFS. They all store data differently.
4. Your programs do not know which one they are using, and should not care.
5. That works because the kernel has a layer in the middle that presents one uniform set of operations: open, read, write, close, list.
6. Every filesystem implements that same set. The layer picks the right one based on where the file lives.
7. Mounting is how a filesystem gets attached to the tree of names. You say “this disk appears at /media/photos” and from then on it does.
8. When you open a file, you get back a small number. Every later operation uses that number. The kernel remembers everything else.
9. Permissions decide who may do what. On UNIX they are simple and were designed in 1971. On Windows they are elaborate lists.

PLAIN18.9.2 a picture in your head#

1. Think of a large library.
2. The shelves hold numbered boxes. Box 41,922 contains some pages. Boxes know nothing about titles. That is the disk.
3. The card catalogue maps a title to a list of box numbers. That is the filesystem’s index, called the inode table on UNIX.
4. The front desk is where you ask for a book. You give a title; they hand you a ticket with a number on it, say ticket 3. That is your file descriptor.
5. From then on you say “more from ticket 3” and they know exactly which book and which page you were on. You never say the title again.
6. Two people can hold two tickets for the same book, each on a different page. Two descriptors, one file, two positions.
7. Adding a new wing to the library and declaring that its shelves count as part of aisle F is mounting.

Where this comparison breaks: a library has one physical copy of a book, so two readers cannot both have it. A filesystem hands out as many descriptors as are asked for, and if two writers write at once without coordinating, the result is a mixture of both. The filesystem does not lock anything for you unless you ask.

PLAIN18.9.3 a worked example#

1. Real permission bits, produced on the machine used here by creating files and running ls -l:

-rw-r--r-- 1 root root    0 a.txt      644
-rwxr-x--- 1 root root    0 b.sh       750
drwxr-sr-x 2 root root 4096 d          2755
-rwsr-xr-x 1 root root    0 s          4755
-rwxrwxrwt 1 root root    0 t          1777

2. Read the first column in four pieces. First character: file type. - is an ordinary file, d a directory, l a symbolic link, c a character device, b a block device, s a socket, p a pipe.
3. Then three groups of three: what the owner may do, what the group may do, what everyone else may do.
4. In each group: r read, w write, x execute. A dash means not allowed.
5. So -rw-r--r-- is: owner may read and write; group may read; others may read. Nobody may execute.
6. The numeric form packs each group into one octal digit, with read worth 4, write worth 2, execute worth 1. Owner 4+2=6, group 4, others 4, giving 644.
7. b.sh at 750 is owner read, write and execute (7), group read and execute (5), others nothing (0).
8. Now the odd ones. s shows -rwsr-xr-x, mode 4755. The s where the owner’s x should be is the setuid bit. It means: when this program runs, it runs as the file’s owner, not as you.
9. That is how /usr/bin/passwd works. Verified on this machine, it really is -rwsr-xr-x 1 root root 64152. You need to change a root-owned file, so the program briefly becomes root.
10. Setuid is the single most dangerous bit in UNIX. A bug in a setuid-root program is a way for any user to become root. Modern systems replace it with capabilities wherever they can.
11. d shows drwxr-sr-x, mode 2755. The s in the group position on a directory is setgid: new files created inside inherit the directory’s group. That is how shared project folders work.
12. t shows -rwxrwxrwt, mode 1777. The final t is the sticky bit. On /tmp it means: anyone may create files here, but you may only delete your own. Without it, any user could delete anyone’s temporary files.
13. On a directory the bits mean something different from a file: r lists the names, w creates and deletes entries, x allows you to traverse into it.
14. A directory with x but not r is real and useful: you can open a known path inside it but cannot see what is there.

PLAIN18.9.4 what is really happening inside#

1. The virtual filesystem layer, VFS, is a set of function pointers. Each filesystem registers its own implementations of open, read, write, lookup and about forty others.
2. When your program calls read, the kernel finds the file’s inode, follows its pointer to the operations table, and calls whichever function is there. ext4’s read or NFS’s read or procfs’s read.
3. That indirection is why /proc/self/status can be read like a file even though there is no such file on any disk. Reading it runs a kernel function that generates the text on the spot.
4. There are three tables involved in open files, and confusing them is the most common source of misunderstanding.
5. Table one: the file descriptor table, one per process. It maps small integers to entries in table two.
6. Table two: the open file table, system-wide. Each entry holds the current position, the access mode, and a pointer into table three.
7. Table three: the inode table. One entry per actual file, holding size, owner, permissions, timestamps and the block map.
8. Now the behaviours make sense. dup copies a descriptor entry, so both point at the same open file entry, so both share one position. open on the same file twice makes two open file entries, so two independent positions.
9. And after fork, parent and child have separate descriptor tables that point at the same open file entries, so their positions move together.
10. Real descriptors, from /proc/self/fd on this machine:

0 -> /dev/null
1 -> /tmp/.../output
2 -> /tmp/.../output
3 -> /proc/6536/fd

11. Note that 1 and 2 point at the same file: standard output and standard error were both redirected to one place.
12. open does this: parse the path one component at a time, checking traverse permission on each directory; find the final inode; check the requested access against the permission bits; allocate an open file entry; find the lowest free descriptor number; return it. The rule that it is always the lowest free number is a standard, and shell redirection depends on it.
13. read does this: look up the descriptor, take the position, ask the filesystem for those bytes, which usually come from the page cache, copy them into the program’s buffer, advance the position, return the count.
14. write usually does not touch the disk at all. It copies into the page cache and marks it dirty. A kernel thread writes it out later. fsync is what forces it out now.
15. close releases the descriptor and decrements a reference count on the open file entry. Only when that count reaches zero is the entry freed.

TECHNICAL18.9.5 the engineer’s version#

1. Linux VFS core objects: struct super_block for a mounted filesystem, struct inode for a file, struct dentry for a name-to-inode cache entry, and struct file for an open instance.
2. The dentry cache is the reason repeated path lookups are fast. Without it, opening /usr/share/doc/x/y would require reading four directories from disk every time.
3. Mounting attaches a superblock at a mountpoint. On this machine /proc/mounts showed real entries:

proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,relatime 0 0
devtmpfs /dev devtmpfs rw,size=4102960k,mode=755 0 0
tmpfs /dev/shm tmpfs rw,size=8216168k 0 0
devpts /dev/pts devpts rw,mode=600 0 0

4. Note that proc, sysfs, devtmpfs and tmpfs have no disk at all. They are kernel interfaces wearing a filesystem’s clothes. That is the VFS abstraction paying off.
5. UNIX permission checking order is strict: if you are the owner, only the owner bits apply, even if the group bits are more permissive. Then group, then other. First match wins, and it is not a union.
6. umask masks off bits from newly created files. The default 022 verified here turns a requested 666 into 644 and a requested 777 into 755.
7. POSIX access control lists, from the withdrawn POSIX 1003.1e draft 17 but implemented anyway, add named users and groups: setfacl -m u:alice:rw file, read back with getfacl. A file with an ACL shows a + after its mode in ls -l.
8. Windows uses a completely different model. Every securable object has a security descriptor containing an owner SID, a group SID, a DACL (discretionary access control list) and a SACL (system ACL, for auditing).
9. A DACL is an ordered list of access control entries, each granting or denying specific rights to a SID. There are far more than three rights: FILE_READ_DATA, FILE_WRITE_ATTRIBUTES, DELETE, WRITE_DAC, WRITE_OWNER, SYNCHRONIZE and more.
10. Evaluation order matters: deny entries are processed before allow entries, and inherited entries after explicit ones. Tools: icacls on the command line, the Security tab in Explorer.
11. Filesystem comparison, all figures from the filesystems’ own documentation:

12. FAT32’s 4 GiB limit is the reason a single large video file cannot be copied to many USB sticks as sold. It is a 32-bit size field, not a bug.
13. Tools: lsof and fuser to see who has a file open, df and du for space, stat for one file’s metadata, mount and findmnt for the mount tree, tune2fs -l for ext4 superblock details, debugfs for raw inspection, fsck for repair.

WORDS18.9.6 remember these#

1. Filesystem — the scheme that turns names into blocks — the on-disk format plus the driver implementing it.
2. VFS — one interface over many filesystems — the virtual filesystem switch, dispatching operations through per-filesystem function tables.
3. Inode — the record describing one file — metadata and block map, identified by number rather than by name.
4. File descriptor — the ticket you get from open — a per-process index into the descriptor table, always the lowest free number.
5. Mount — attaching a filesystem into the name tree — binding a superblock to a directory so paths below it resolve there.
6. setuid — the program runs as its owner, not you — mode bit 4000, causing the effective UID to be set from the file’s owner on exec.
7. Sticky bit — you may only delete your own files here — mode bit 1000 on a directory, restricting deletion to owners.
8. Page cache write-back — write returns before the disk is touched — buffered writes marked dirty and flushed later, forced by fsync.

18.10 Inter-process communication#

PLAIN18.10.1 in simple words#

1. Processes are deliberately kept apart. That is the whole point of the design.
2. But they often need to work together, so the operating system provides controlled ways to pass information across the wall.
3. There are six that matter, and they differ mainly in three questions: how fast, how much can you send, and can the two sides be on different machines.
4. Pipes: a one-way stream of bytes from one process to another. Made by the shell every time you use the | symbol.
5. Named pipes: the same thing, but with a name in the filesystem, so unrelated processes can find it.
6. Signals: a single number sent to a process to interrupt it. Almost no information, but it arrives even if the target is stuck.
7. Message queues: a post box holding whole messages with boundaries preserved, so a 20-byte message arrives as 20 bytes and not as part of a stream.
8. Shared memory: two processes agree to see the same physical memory. The fastest possible, because nothing is copied.
9. Sockets: a two-way connection, which works between processes on one machine and between machines across a network, with the same code.
10. Remote procedure call: a layer on top that makes a call to another machine look like an ordinary function call in your program.

PLAIN18.10.2 a picture in your head#

1. Two offices in the same building need to exchange work.
2. A pipe is a chute in the wall. Papers go in one end and come out the other, in order, one way only. Simple and fast.
3. A named pipe is the same chute with a label on it, so anyone in the building can find it rather than only the two who installed it.
4. A signal is a fire alarm. It carries one bit of meaning, you cannot send data with it, and it interrupts whatever the other office was doing.
5. A message queue is a set of pigeonholes. Each item stays a separate item. Nothing merges.
6. Shared memory is knocking a hole in the wall and putting one desk half in each room. Both sides see the same paper immediately. Nobody carries anything. And both sides can grab the same sheet at once, so you need a rule about whose turn it is.
7. A socket is a telephone line, which happens to work identically whether the other office is next door or in another country.

Where this comparison breaks: a chute holds an unlimited pile of paper. A real pipe holds a fixed amount, 64 KiB by default on Linux, and when it is full the writer simply stops until the reader catches up. That blocking behaviour is not a flaw, it is the flow control that makes pipelines work on files larger than memory.

PLAIN18.10.3 a worked example#

1. This shell command:

sh -c 'echo hello world | tr a-z A-Z'

2. Traced with strace -f, the kernel calls were exactly these, with process IDs at the left:

18231 pipe2([3, 4], 0)                  = 0
18231 clone(...SIGCHLD...)              = 18232
18231 close(4)                          = 0
18232 close(3)                          = 0
18232 dup2(4, 1)                        = 1
18232 close(4)                          = 0
18232 write(1, "hello world\n", 12)     = 12
18232 +++ exited with 0 +++
18231 clone(...SIGCHLD...)              = 18233
18231 close(3)                          = 0
18233 dup2(3, 0)                        = 0
18233 close(3)                          = 0
18233 execve("/usr/bin/tr", ["tr","a-z","A-Z"]) = 0
18233 read(0, "hello world\n", 8192)    = 12
18233 read(0, "", 8192)                 = 0
18233 write(1, "HELLO WORLD\n", 12)     = 12
18231 wait4(-1, [exited 0], 0, NULL)    = 18232
18231 wait4(-1, [exited 0], 0, NULL)    = 18233

3. Line by line. pipe2 creates the pipe and returns two descriptors at once: 3 is the read end, 4 is the write end. One object, two handles.
4. clone makes the first child, PID 18232, which will run echo. It inherits both descriptors.
5. The parent closes 4, its copy of the write end, because it will not write.
6. The child closes 3, its copy of the read end, because it will not read.
7. dup2(4, 1) is the key move. It makes descriptor 1, standard output, point at the same open file entry as descriptor 4. Now anything written to standard output goes into the pipe.
8. The child then closes 4, because 1 already refers to the pipe. Two handles to the same thing are no longer needed.
9. write(1, "hello world\n", 12) puts 12 bytes into the pipe buffer.
10. The second child, 18233, does the mirror image: dup2(3, 0) makes standard input read from the pipe, then execve replaces it with /usr/bin/tr.
11. Notice that tr does none of this. By the time tr starts, the plumbing is already done and it just reads standard input like any program. That is why every UNIX tool composes.
12. read(0, "hello world\n", 8192) gets the 12 bytes. The next read returns 0.
13. That zero is end-of-file, and it happens only because every copy of the write end has been closed. If any process had kept one open, tr would have waited forever. Forgetting to close the write end is the classic pipe bug.
14. Finally write(1, "HELLO WORLD\n", 12) sends the result to the terminal, and the shell reaps both children with wait4.

PLAIN18.10.4 what is really happening inside#

1. A pipe is a fixed-size circular buffer in kernel memory with two ends and no name. 64 KiB on Linux by default, adjustable with fcntl and F_SETPIPE_SZ.
2. Writing to a full pipe blocks. Reading an empty pipe with writers still open blocks. Reading an empty pipe with no writers returns 0. Writing to a pipe with no readers delivers SIGPIPE, which kills the writer by default.
3. That last rule is why head on a huge file stops quickly rather than reading everything: head exits, the pipe loses its reader, the producer is killed.
4. A signal is a bit set in the target’s task structure. Nothing is delivered immediately. The next time that process is about to return to user mode, the kernel notices the bit and diverts it to the handler.
5. That is why a process stuck in uninterruptible kernel sleep does not respond to signals: it is never about to return to user mode.
6. Signal 9, SIGKILL, and signal 19, SIGSTOP, cannot be caught or ignored, because they are handled by the kernel itself rather than delivered.
7. Shared memory is the only mechanism with no copying. The kernel maps the same physical page frames into two processes’ page tables. After setup, the kernel is not involved at all.
8. That is also its danger. With no kernel in the loop there is no ordering, so the two sides must agree on synchronization themselves, usually with a semaphore or an atomic flag plus memory barriers.
9. Sockets carry a protocol stack even when both ends are on one machine. A UNIX domain socket skips the network stack and is roughly twice as fast as a loopback TCP socket for the same data.

TECHNICAL18.10.5 the engineer’s version#

1. Two copies means user to kernel and kernel to user. splice and vmsplice on Linux can reduce pipe transfers to one copy or zero by moving page references instead of bytes.
2. Signals: 31 standard signals plus 32 real-time signals on Linux. Real-time signals queue and carry a small value; standard signals do not queue, so two SIGUSR1 sent quickly may be delivered once.
3. A demonstration on this machine sent SIGUSR1 to itself three times in a tight loop and the handler ran three times, because each was delivered before the next was raised. Send them faster than delivery and you would lose some.
4. The first six signals as printed by kill -l here: 1 SIGHUP, 2 SIGINT, 3 SIGQUIT, 4 SIGILL, 5 SIGTRAP, 6 SIGABRT. Also worth knowing: 9 SIGKILL, 11 SIGSEGV, 13 SIGPIPE, 15 SIGTERM, 17 SIGCHLD, 19 SIGSTOP.
5. SIGTERM asks politely and can be caught for cleanup. SIGKILL cannot. A shutdown script that sends only SIGKILL loses data; systemd sends SIGTERM, waits DefaultTimeoutStopSec (90 s by default), then SIGKILL.
6. System V IPC, from AT&T System V in 1983: msgget, semget, shmget, with keys instead of file descriptors and no automatic cleanup. Limits on this machine from ipcs -l: 32,000 message queues maximum, 8,192-byte maximum message size, 4,096 shared memory segments.
7. POSIX IPC, IEEE 1003.1b-1993, is the modern replacement: mq_open, sem_open, shm_open, all using file-descriptor semantics and names under /dev/shm and /dev/mqueue.
8. Modern Linux additions: eventfd, signalfd and timerfd turn events, signals and timers into readable descriptors, so they can be waited on by epoll alongside sockets. memfd_create gives an anonymous file for sharing.
9. Higher-level buses: D-Bus on Linux desktops, Mach ports on macOS, Binder on Android, ALPC on Windows. All are message passing with identity and permissions on top.
10. Remote procedure call history: Birrell and Nelson’s 1984 paper Implementing Remote Procedure Calls set the model. Sun RPC, RFC 1057 in 1988, powered NFS. Modern systems use gRPC over HTTP/2 with Protocol Buffers, or JSON over HTTP.
11. The honest version, from Waldo, Wyant, Wollrath and Kendall’s 1994 Sun paper A Note on Distributed Computing: a remote call is not a local call. It can be slow, it can fail halfway, and it can succeed while the reply is lost. Any RPC layer that hides this will eventually mislead you.
12. Tools: lsof -p PID to see pipes and sockets, ss -x for UNIX sockets, ipcs and ipcrm for System V objects, strace -e trace=ipc, and dbus-monitor for D-Bus traffic.

WORDS18.10.6 remember these#

1. Pipe — a one-way chute between processes — an anonymous kernel ring buffer with a read end and a write end, 64 KiB by default on Linux.
2. FIFO — a pipe with a name — a filesystem object that unrelated processes can open to reach the same buffer.
3. Signal — a numbered interruption — an asynchronous notification delivered on return to user mode; SIGKILL and SIGSTOP cannot be caught.
4. Shared memory — the same physical pages in two processes — zero-copy IPC requiring caller-supplied synchronization.
5. Socket — a connection endpoint — a descriptor supporting stream or datagram communication, local via AF_UNIX or remote via AF_INET.
6. RPC — calling a function on another machine — a stub-and-skeleton layer that marshals arguments over a transport, with failure modes a local call lacks.

18.11 Device management and drivers#

PLAIN18.11.1 in simple words#

1. Chapter 13 covered devices and buses. One line to bring it back: a device is a lump of hardware that the processor talks to through registers, memory regions and interrupts.
2. A driver is the piece of software that knows how to talk to one specific device. It is the translator between the general and the particular.
3. The operating system’s job here is to make every disk look like a disk, so that the filesystem code does not need to know which brand it is.
4. It does that by defining an interface, and requiring every driver to implement it: open, close, read, write, control.
5. On UNIX systems, devices then appear as files in /dev, so programs can talk to hardware using the same calls they use for files.
6. Hot plug is the ability to add and remove hardware while running. The kernel notices, loads a driver, and tells userspace.
7. The I/O scheduler decides in what order pending disk requests are sent, because order matters enormously for speed on a spinning disk and matters less on a solid-state one.

PLAIN18.11.2 a picture in your head#

1. Think of an electrical socket standard.
2. A kettle, a lamp and a laptop charger are completely different inside. They all end in the same plug.
3. The socket is the interface. The plug is the driver. The building’s wiring does not know or care what is on the other end.
4. A new appliance arrives, you plug it in, and the building needs no changes. That is a loadable driver.
5. If a badly made appliance shorts out, it can trip the whole building, because it is connected directly to the mains. That is why a bad driver crashes the kernel.

Where this comparison breaks: an appliance can only draw power, whereas a driver runs inside the kernel with the same privileges as the kernel itself. A closer comparison would be an appliance that could rewire the building. This is exactly why driver signing exists, and why microkernels move drivers out of the kernel.

PLAIN18.11.3 a worked example#

1. Real device files from /dev on the machine used here:

crw-rw-rw- 1 root root   1, 3 /dev/null
crw-rw-rw- 1 root root   1, 5 /dev/zero
crw-rw-rw- 1 root root   1, 8 /dev/random
brw------- 1 root root 254, 0 /dev/vda

2. The first character is the device class. c means character device: a stream of bytes, read one after another, no seeking. b means block device: a fixed-size array of blocks you can jump around in.
3. Where a normal file would show its size, a device file shows two numbers. /dev/null is 1, 3.
4. The first is the major number: which driver handles this. Major 1 is the kernel’s memory device driver. Major 254 here is virtio block.
5. The second is the minor number: which device that driver should use. Minor 3 is null, minor 5 is zero, minor 8 is random. Same driver, three behaviours.
6. So /dev/null contains no data at all. It is a name, a type and two integers. Everything it does is code in the kernel.
7. Writing to /dev/null calls a function that discards the bytes and reports success. Reading from it returns end of file immediately.
8. Permissions on device files are ordinary UNIX permissions, and they are the access control. /dev/vda, the raw disk, is brw-------: only root may open it. /dev/null is crw-rw-rw-: anyone.

PLAIN18.11.4 what is really happening inside#

1. When you open /dev/null, the VFS sees the inode is a character device, reads major 1 minor 3 from it, and looks up major 1 in the character device table.
2. It replaces the file’s operations table with that driver’s, so every later read and write goes to driver code.
3. The device file is therefore not a link to hardware. It is a name that carries two numbers, and the numbers select code.
4. Drivers do their work in two halves, because interrupts must be short.
5. The top half runs when the interrupt arrives. It does the minimum: acknowledge the device, grab the data, and schedule the rest.
6. The bottom half runs slightly later with interrupts enabled, and does the real work. Linux calls these softirqs, tasklets and workqueues.
7. Hot plug on Linux works like this: the bus controller raises an interrupt, the bus driver reads the new device’s identity numbers, the kernel matches those numbers against every driver’s table of supported devices, and loads the one that claims it.
8. It then sends an event up to userspace over a netlink socket, where udev receives it and creates the device file, sets permissions and may run rules.
9. The I/O scheduler sits between the filesystem and the driver. It holds requests briefly so it can merge adjacent ones and reorder them.
10. On a spinning disk that reordering is worth an enormous amount, because a head seek costs about 5 to 10 milliseconds and a reordering that avoids one saves more time than all the software involved.
11. On an SSD there is no head, so reordering saves much less, and the modern default is a scheduler that mostly stays out of the way.

TECHNICAL18.11.5 the engineer’s version#

1. Linux device model: struct bus_type, struct device, struct device_driver and struct class, all exposed under /sys. /sys/bus/pci/devices lists every PCI device; /sys/class/net lists every network interface.
2. Driver binding is by ID table. A PCI driver declares a struct pci_device_id[] of vendor and device IDs; the core matches and calls the driver’s probe. Modules carry these tables in a .modalias section so udev can load them on demand.
3. Character devices are registered with alloc_chrdev_region and cdev_add; block devices go through the block layer, which today is blk-mq, the multi-queue block layer merged in Linux 3.13 in 2014 and made the only path in Linux 5.0 in 2019.
4. Linux I/O schedulers available in the multi-queue layer:

5. Read and set it per device: cat /sys/block/sda/queue/scheduler shows the list with the active one in brackets; writing a name changes it.
6. Windows uses the Windows Driver Model and, since Vista, the Windows Driver Frameworks. KMDF drivers run in kernel mode; UMDF drivers run in user mode, which is Windows quietly adopting the microkernel argument for classes of device where the performance cost is acceptable.
7. Driver signing: 64-bit Windows has required kernel-mode driver signatures since Vista in 2006, and since Windows 10 version 1607 in 2016 new kernel drivers must be signed through the Microsoft hardware developer portal.
8. macOS deprecated kernel extensions and replaced them with DriverKit and System Extensions from macOS 10.15 Catalina in 2019. DriverKit drivers run in user space. On Apple silicon, loading a kext requires lowering the security policy and rebooting.
9. Real interrupt counts from /proc/interrupts on the machine used here: IRQ 26 is ttyS0, the serial port, with 12,584 interrupts on CPU0 and 0 on CPU1. Interrupt affinity, not chance, put them all on one core.
10. Tools: lspci -v, lsusb, lsblk, lsmod, modinfo, udevadm monitor, dmesg on Linux; system_profiler and ioreg on macOS; Device Manager, pnputil and driverquery on Windows.

WORDS18.11.6 remember these#

1. Driver — code that knows one specific device — a kernel or user-mode module implementing a standard interface for particular hardware.
2. Device file — a name that selects driver code — a filesystem node carrying a type plus major and minor numbers.
3. Major number — which driver — the index selecting a registered driver in the character or block device table.
4. Minor number — which instance — the value the driver interprets to pick a device or behaviour.
5. Top half and bottom half — do the urgent bit now, the rest later — interrupt handler versus deferred processing in softirq or workqueue context.
6. I/O scheduler — deciding the order of disk requests — the block layer elevator merging and reordering requests, chosen per device.

18.12 Protection and isolation#

PLAIN18.12.1 in simple words#

1. Everything in this chapter that stops one program hurting another comes from one idea: some code is trusted and some is not, and the hardware knows which.
2. The processor runs in user mode or kernel mode. In user mode the dangerous instructions simply fail.
3. A program cannot reach another program’s memory because the address it uses is not a real address. It is translated, and the translation table only contains its own memory.
4. If it tries an address that is not in its table, the processor raises a fault, and the kernel usually kills it. That is the segmentation fault.
5. On top of that base, systems add finer tools.
6. Capabilities cut root’s absolute power into pieces, so a program can be allowed to bind a low port without being allowed to do everything.
7. Sandboxes shrink what a program may ask for at all, by filtering the system call list.
8. Containers give a process a private view of the filesystem, the process list, the network and the users, while sharing one kernel.
9. Virtual machines go further and give a whole fake computer, with its own kernel, on top of a program called a hypervisor.
10. When the trusted code itself fails, there is nothing above it to catch the error, so the machine stops. That is a kernel panic or a blue screen.

PLAIN18.12.2 a picture in your head#

1. Think of a building with staff passes.
2. Kernel mode is the master key. User mode is a pass that opens only your own office.
3. Capabilities are what happens when you cut the master key into forty separate keys and hand out only the ones each person needs.
4. A sandbox is an office where most of the doors have been bricked up, so even a valid key opens nothing.
5. A container is a floor of the building with its own numbering, its own directory board and its own staff list. It feels like a whole building. Walk down the fire stairs far enough and it is still the same building, with one boiler serving everything. That boiler is the shared kernel.
6. A virtual machine is a separate building, with its own boiler, built inside a warehouse. Much stronger separation, much more concrete.

Where this comparison breaks: a wall in a building is physical, and no clever argument gets you through it. Processor isolation is enforced by logic that can have design flaws, which is exactly what Spectre and Meltdown were in 2018: correct permission checks defeated by timing side effects of speculation. The walls were real. Information still leaked around them.

PLAIN18.12.3 a worked example#

1. Namespaces are the mechanism containers are built from. Every process on Linux belongs to one of each kind. Real listing from /proc/self/ns on this machine:

cgroup -> cgroup:[4026531835]
ipc    -> ipc:[4026531839]
mnt    -> mnt:[4026531832]
net    -> net:[4026531833]
pid    -> pid:[4026531836]
time   -> time:[4026531834]
user   -> user:[4026531837]
uts    -> uts:[4026531838]

2. Those numbers are namespace identifiers. Two processes with the same number share that view of the world.
3. Run a container and it gets different numbers for some of these. Same kernel, different view.
4. A process in a new PID namespace sees itself as PID 1 and cannot see any process outside. ps inside a container genuinely shows only the container’s processes, because the kernel is answering a different question.
5. A process in a new mount namespace has its own filesystem tree. That is how a container’s / can be an Ubuntu image on a Fedora host.
6. A process in a new network namespace has its own interfaces, its own routing table and its own port numbers. Two containers can both listen on port 80.
7. Namespaces control what you can see. Cgroups control how much you can use. Real controllers mounted on this machine include cpu, cpuacct, cpuset, memory, blkio, pids, devices and freezer.
8. Put the two together and you have a container: a normal process, with a private view from namespaces and a resource cap from cgroups. There is no “container” object in the Linux kernel at all.

PLAIN18.12.4 what is really happening inside#

1. Privilege rings on x86 were designed with four levels, 0 to 3. Every general purpose operating system uses exactly two: 0 for the kernel, 3 for programs.
2. Rings 1 and 2 were meant for drivers and services. They were abandoned because they do not exist on other architectures, and portable kernels could not rely on them.
3. Virtualization added a level below 0. On Intel it is VMX root mode, on AMD it is SVM host mode, on ARM it is EL2. The hypervisor runs there, so a guest kernel can believe it is in ring 0 while actually being supervised.
4. That hardware support arrived in 2005 and 2006, with Intel VT-x and AMD-V. Before it, virtualization required rewriting guest instructions on the fly.
5. Capabilities on Linux split root into about 40 separate powers, such as CAP_NET_BIND_SERVICE for low ports, CAP_SYS_ADMIN for a great deal, and CAP_DAC_OVERRIDE for ignoring file permissions.
6. Sandboxing with seccomp works by giving the kernel a filter program that is run on every system call, deciding allow, deny, kill or trap. Once installed it cannot be removed, and the process cannot regain the lost ability.
7. A kernel panic happens when kernel code hits a condition it cannot recover from: a null pointer dereference in kernel space, a corrupted internal structure, or an unhandled exception in an interrupt handler.
8. There is no higher authority to catch the error and no isolated context to kill, so the only safe action is to stop and print what happened.
9. Windows does exactly the same thing and calls it a bug check. The blue screen shows a stop code and, on modern versions, writes a crash dump to disk before restarting.

TECHNICAL18.12.5 the engineer’s version#

1. Linux namespace types and the clone flags that create them:

2. Dates: the mount namespace arrived in Linux 2.4.19 in 2002, the rest through 2.6.x, user namespaces completed in 3.8 in 2013, and the time namespace in 5.6 in 2020.
3. Cgroups came from Google, written by Paul Menage and Rohit Seth, merged as “control groups” in Linux 2.6.24 in January 2008. Cgroup v2 with a unified hierarchy became the default in most distributions from around 2021.
4. Container runtimes are userspace assemblers of these primitives: Docker from 2013, then the OCI runtime specification, runc, containerd, podman, and Kubernetes as the orchestrator above them.
5. The honest version: a container shares the host kernel, so a kernel privilege escalation escapes it. A virtual machine shares only the hypervisor, which has a much smaller attack surface. Anyone who tells you containers are as isolating as VMs is selling something.
6. The middle ground is real: Firecracker, released by AWS in 2018, is a minimal virtual machine monitor of roughly 50,000 lines of Rust, with a device model of five devices, booting in about 125 milliseconds. gVisor intercepts system calls in a userspace kernel instead. Kata Containers puts a real VM under a container interface.
7. Mandatory access control layers sit above discretionary permissions: SELinux (originating from the United States National Security Agency, merged in Linux 2.6 in 2003), AppArmor, and on macOS the Sandbox subsystem plus System Integrity Protection, introduced in OS X 10.11 El Capitan in 2015.
8. Hypervisor types: type 1 runs on the hardware directly (Xen from 2003, VMware ESXi, Microsoft Hyper-V, KVM once you accept that Linux becomes the hypervisor); type 2 runs as an application (VirtualBox, VMware Workstation, older Parallels). The line is blurry and the classification is a convention, not a standard.
9. Panic and bug check diagnostics: kdump and crash on Linux, /var/crash dumps, panic=1 on the kernel command line to reboot instead of hanging (which was set on the machine used here); WinDbg with !analyze -v on Windows minidumps in C:\Windows\Minidump.
10. Common Windows stop codes worth knowing: IRQL_NOT_LESS_OR_EQUAL usually a driver touching paged memory at high interrupt level; PAGE_FAULT_IN_NONPAGED_AREA bad pointer or failing RAM; SYSTEM_SERVICE_EXCEPTION a fault in a system call path. All three most often mean a driver, not Windows itself.

WORDS18.12.6 remember these#

1. Privilege ring — how much the processor will let you do — hardware protection level, 0 for kernel and 3 for user on x86-64.
2. Capability — one slice of root’s power — a per-thread bit such as CAP_NET_BIND_SERVICE, granted independently of UID 0.
3. Seccomp — a filter on which system calls you may make — a BPF program applied to every syscall, irreversible once installed.
4. Namespace — a private view of one kind of resource — the kernel mechanism giving a process its own PIDs, mounts, network or users.
5. Cgroup — a cap on how much you may use — hierarchical resource accounting and limiting for CPU, memory, I/O and process count.
6. Hypervisor — the thing that runs whole operating systems — software at a privilege level below the guest kernel, using VT-x, AMD-V or EL2.
7. Kernel panic — the trusted code failed — an unrecoverable kernel fault with no higher authority to handle it; a bug check or blue screen on Windows.

18.13 UNIX: the full story#

PLAIN18.13.1 in simple words#

1. Nearly every operating system you will use descends from one project, or was built to imitate it.
2. It began as a failure. In the mid-1960s MIT, General Electric and Bell Labs tried to build Multics, an ambitious time-sharing system for the GE-645.
3. Multics was late, huge and complicated. Bell Labs pulled out in 1969.
4. Two of the Bell Labs people, Ken Thompson and Dennis Ritchie, missed the pleasant working environment they had lost.
5. Thompson found a little-used PDP-7 in a corner and, in 1969, wrote a small system for it, helped by Ritchie and Rudd Canaday.
6. It was a joke on the name: Multics was big and did everything at once, so theirs was Unics, later spelled UNIX.
7. In 1970 the group got a PDP-11, and UNIX became a real working system.
8. In 1973 they rewrote almost the whole thing in C, a language Ritchie had made for the purpose. Almost nobody wrote operating systems in a high-level language then.
9. That single decision is why UNIX spread. Move the C compiler to a new machine and most of the operating system comes with it.

PLAIN18.13.2 a picture in your head#

1. Think of a kitchen with a drawer full of single-purpose tools: a peeler, a grater, a sieve, a whisk. Each does one thing well.
2. The alternative is one huge machine with forty buttons that peels, grates, sieves and whisks, badly, and cannot be repaired.
3. UNIX chose the drawer. Small programs, each doing one job, that can be connected end to end.
4. The connection is the pipe: the output of one becomes the input of the next, without either knowing the other exists.
5. To make that work, everything must be the same shape, so the shape chosen was plain lines of text.
6. That is why ls | grep report | wc -l works, and why you can invent a combination nobody has ever tried and it will still work.

Where this comparison breaks: kitchen tools are physical and cannot be combined into new tools. UNIX programs can be composed into scripts that become new programs, so the drawer grows. And the text-stream convention has real costs: programs must parse each other’s output, which is fragile, and this is exactly the argument PowerShell makes by passing objects instead.

PLAIN18.13.3 a worked example#

1. The UNIX philosophy, as Doug McIlroy, the head of the Bell Labs computing research department and the inventor of the pipe, stated it in the 1978 Bell System Technical Journal:
2. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.
3. Expect the output of every program to become the input to another, as yet unknown, program. Do not clutter output with extraneous information.
4. Design and build software, even operating systems, to be tried early, ideally within weeks. Do not hesitate to throw away the clumsy parts and rebuild them.
5. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some out after you have finished using them.
6. Two further principles are usually added, from Rob Pike and others: everything is a file, and text is the universal interface.
7. The first is why a disk, a terminal, a random number source and a running process all appear as things you can open and read.

PLAIN18.13.4 what is really happening inside#

1. AT&T could not sell UNIX. A 1956 consent decree, settling an antitrust case, forbade the Bell System from entering businesses outside common carrier communications.
2. So AT&T licensed UNIX to universities for a nominal fee, with source code. That accident is why an entire generation of computer scientists learned operating systems by reading a real one.
3. At Berkeley, students and staff extended their copy heavily: the C shell, the vi editor, virtual memory, the fast filesystem, job control, and the sockets API that the whole internet uses.
4. Their distribution was the Berkeley Software Distribution, BSD, from 1978.
5. The 1984 breakup of AT&T removed the restriction, and AT&T began selling UNIX commercially as System V. Suddenly the free university tradition and a commercial product were the same code.
6. Then came the UNIX wars: through the late 1980s, vendors split into rival camps, each with an incompatible UNIX, each claiming to be the standard.
7. Customers responded by demanding a written specification that any vendor could implement, which produced POSIX.
8. In 1992 AT&T’s Unix System Laboratories sued Berkeley Software Design over BSD code. The case settled in February 1994, requiring a handful of files to be removed, and 4.4BSD-Lite was released free of AT&T code.
9. The delay mattered. During the two years that BSD’s legal status was unclear, a student in Helsinki released a kernel with no legal cloud over it at all.

TECHNICAL18.13.5 the engineer’s version#

1. Key dates, all verifiable: Multics development from 1964 at MIT Project MAC with Bell Labs and General Electric; Bell Labs withdrew in 1969; Thompson’s PDP-7 work began in 1969; the PDP-11/20 arrived in 1970 and the name UNIX was adopted; Version 4 was rewritten in C in 1973.
2. Ritchie and Thompson’s paper The UNIX Time-Sharing System appeared in Communications of the ACM, volume 17 number 7, July 1974, and is the document that made the outside world take notice.
3. Thompson and Ritchie received the ACM Turing Award in 1983 for UNIX and for the theory of operating systems.
4. POSIX is IEEE Std 1003.1, first published in 1988. The name was suggested by Richard Stallman when the committee wanted something pronounceable. It standardizes the C API, the shell and utilities, not the kernel design.
5. The Single UNIX Specification, maintained by The Open Group, is what confers the right to use the UNIX trademark. macOS is certified UNIX. Linux is not, and has never applied. AIX, HP-UX and Solaris are.
6. The family tree, with founding dates:

7. The split that matters technically: System V gave us init runlevels, System V IPC and the /etc/init.d layout; BSD gave us sockets, the fast filesystem, rc scripts and much of the networking code that every system, including Windows, borrowed.
8. Both branches are still running in production. FreeBSD powers Netflix’s content delivery servers and is the base of the PlayStation 4 and 5 system software; OpenBSD produces OpenSSH, which almost every server on earth runs.

WORDS18.13.6 remember these#

1. Multics — the ambitious project UNIX reacted against — the 1964 MIT, GE and Bell Labs time-sharing system that Bell Labs abandoned in 1969.
2. UNIX philosophy — small tools, joined together — composition of single-purpose programs over a universal text interface.
3. BSD — the Berkeley branch of UNIX — the distribution that produced sockets, the fast filesystem and today’s FreeBSD, NetBSD and OpenBSD.
4. System V — the AT&T commercial branch — the source of runlevels, System V IPC and much enterprise UNIX practice.
5. POSIX — the written rules any UNIX-like system can follow — IEEE Std 1003.1, first published in 1988, defining API, shell and utilities.
6. Single UNIX Specification — what lets you call it UNIX — The Open Group’s certification, held by macOS, AIX, HP-UX and Solaris, not by Linux.

18.14 Linux#

PLAIN18.14.1 in simple words#

1. In 1991 a 21-year-old student in Helsinki, Linus Torvalds, wanted a UNIX-like system for his new 386 PC. The teaching system MINIX was restricted and he could not change it freely.
2. On 25 August 1991 he posted to the newsgroup comp.os.minix. The message is worth quoting exactly, because it is famous for being wrong:

Hello everybody out there using minix -

I'm doing a (free) operating system (just a hobby, won't be
big and professional like gnu) for 386(486) AT clones.

3. The post continued that he had ported bash 1.08 and gcc 1.40, asked what features people wanted, and added a postscript: the system was free of any MINIX code, had a multi-threaded filesystem, and was “NOT protable” because it used 386 task switching.
4. Version 0.01 was released on 17 September 1991.
5. He wrote only the kernel. Everything else that made it usable — the shell, the compiler, the text tools, the C library — already existed, from the GNU project.
6. The GNU project had been started by Richard Stallman in 1983 to build a complete free UNIX-like system. By 1991 GNU had almost everything except a working kernel.
7. So the two halves fitted together exactly. That is not luck; it is what happens when both sides target the same POSIX interface.
8. In January 1992, with version 0.12, Torvalds put Linux under the GNU General Public License. He has repeatedly called this the single best decision he made.

PLAIN18.14.2 a picture in your head#

1. Think of an engine and a car.
2. The Linux kernel is the engine. On its own it moves nothing and has no seats.
3. GNU supplied the gearbox, the wheels, the steering and the dashboard.
4. A distribution is the finished car: someone chose an engine version, bolted on a particular set of parts, painted it, tested it and put a badge on it.
5. Ubuntu, Debian, Fedora, Arch and Android are all cars built around the same family of engines, and they do not look or drive alike.
6. This is why “which is better, Linux or Ubuntu” is a confused question. One is an engine, the other is a car containing it.

Where this comparison breaks: a car engine is useless outside a car, whereas the Linux kernel really does run alone in embedded systems with a single program on top. And a car maker builds one car; a distribution assembles tens of thousands of independently developed pieces it did not write and cannot fully test.

PLAIN18.14.3 a worked example#

1. What a distribution actually adds to the kernel, concretely:
2. A C library, usually glibc, sometimes musl. The kernel does not provide one.
3. A userland: GNU coreutils, or BusyBox on small systems, or Toybox on Android.
4. An init system and service manager, usually systemd since about 2015.
5. A package manager and a repository of tested, signed, compiled packages: apt with .deb, dnf with .rpm, pacman, apk.
6. A default configuration for thousands of programs, which is most of the real work and almost none of the credit.
7. A desktop environment, if any: GNOME, KDE Plasma, Xfce.
8. A release policy: Debian stable freezes and supports for years; Arch ships updates continuously; Ubuntu LTS releases every two years in April with five years of support.
9. Security updates, which is the service people actually pay for.
10. Numbers for scale: Debian 12, released June 2023, contains over 64,000 binary packages. The kernel is one of them.

PLAIN18.14.4 what is really happening inside#

1. How a change gets into Linux, in order:
2. You write a patch and send it as plain text email to the mailing list for the relevant subsystem, with a Signed-off-by line certifying its origin.
3. The subsystem maintainer reviews it. Most patches are rejected or sent back several times. Review is public and often blunt.
4. If accepted, it goes into the maintainer’s tree, then into linux-next, where it is built and tested against everything else nightly.
5. When the merge window opens, the maintainer sends a pull request to Torvalds, who merges it.
6. The merge window is two weeks. Then come seven or eight weekly release candidates, and then the release. The whole cycle is nine to ten weeks and has been remarkably regular since 2005.
7. Torvalds wrote Git in 2005 in about ten days, specifically because this workflow needed a tool that did not exist. The tool exists because of the process, not the other way round.
8. Version numbers carry no meaning beyond ordering. Torvalds increments the first digit when the second gets “too big”, by his own description. That is why 2.6.39 became 3.0, 3.19 became 4.0, and 6.17 became 7.0 in April 2026.
9. Long-term support kernels are chosen once a year and maintained for two to six years, or longer under the Civil Infrastructure Platform.

TECHNICAL18.14.5 the engineer’s version#

1. Verified version history and current state, from kernel.org as of 9 August 2026: mainline 7.2-rc7, stable 7.1.8, longterm 6.18.44, 6.12.103, 6.6.151, 6.1.182, 5.15.215 and 5.10.264.

2. Licensing: the kernel is GPL version 2 only, not “version 2 or later”. That deliberate choice means it can never be relicensed to GPLv3 without the agreement of thousands of copyright holders.
3. The user-space boundary is covered by a syscall exception note, which is why proprietary applications on Linux are legal while proprietary in-kernel drivers are legally contested.
4. The GNU/Linux naming argument, stated fairly. The Free Software Foundation’s position: the system is mostly GNU with a Linux kernel, so calling the whole thing Linux erases the project that supplied most of it and the freedom philosophy that motivated it. The common counter-position: the name refers to the kernel and to common usage, most modern systems contain far less GNU code than in 1993, and Android contains none. Both sides are factually correct about different things; the disagreement is about naming, not about facts.
5. Where Linux runs, with real figures. Supercomputers: on the June 2026 TOP500 list all 500 systems run a Linux-based operating system, as they have since November 2017. The list is led by LineShine at the National Supercomputing Centre in Shenzhen at 2,198.40 PFlop/s, ahead of El Capitan at Lawrence Livermore at 1,809.00 PFlop/s and Frontier at Oak Ridge at 1,353.00 PFlop/s.
6. Mobile: Android, which is a Linux kernel with a completely different userland, accounts for roughly 70 percent of smartphone operating system share worldwide, with iOS most of the rest. Share figures move and vary by source, so treat the exact number as approximate.
7. Servers and cloud: Linux is the majority operating system on public cloud instances and on web-facing servers, by a wide margin in every survey; exact percentages vary by methodology and are worth checking before quoting.
8. Embedded: routers, televisions, cars, cameras, industrial controllers, spacecraft. The Mars helicopter Ingenuity ran Linux on a Snapdragon 801 with the F Prime framework, the first Linux flight on another planet, in 2021.
9. Desktop: the smallest share, in the low single digits of percent, and rising slowly. Steam’s hardware survey and Statcounter both place it around 2 to 5 percent in 2025 and 2026 depending on method.
10. Contribution scale: roughly 1,500 to 2,000 developers from over 200 companies contribute to each release. The largest corporate contributors in recent years include Intel, Red Hat, Google, AMD, Linaro and Huawei. The image of Linux as hobbyist code has been wrong for over fifteen years.

WORDS18.14.6 remember these#

1. Kernel versus distribution — the engine versus the finished car — the Linux kernel is one component; a distribution assembles it with libc, userland, init, packaging and configuration.
2. GPL — the licence that requires sharing changes — the GNU General Public License version 2 for the kernel, requiring derived works to be distributed under the same terms with source.
3. Merge window — the two weeks when new features are accepted — the first phase of each nine to ten week Linux release cycle.
4. LTS kernel — the version kept alive for years — a long-term support release chosen annually and maintained with backported fixes.
5. Signed-off-by — the line certifying where a patch came from — the Developer Certificate of Origin attestation required on every kernel patch.
6. Userland — everything above the kernel — the libraries, shells and utilities that make a kernel into a usable system.

18.15 Windows#

PLAIN18.15.1 in simple words#

1. Windows has two completely separate ancestries that were merged only in 2001.
2. The first is MS-DOS. Tim Paterson at Seattle Computer Products wrote 86-DOS in about six weeks in 1980, as a port of the ideas in Digital Research’s CP/M to the 8086 processor.
3. Microsoft licensed it, hired Paterson in May 1981, bought the rights that July for $25,000, and licensed it to IBM, which shipped it as PC DOS 1.0 in August 1981.
4. MS-DOS had no protection, no multitasking and no memory management. One program ran and owned the machine.
5. Windows 1.0, in November 1985, was not an operating system. It was a program you ran from DOS that drew overlapping panels and let you switch between applications. Windows 2.0, 3.0 in 1990 and 3.1 in 1992 were the same idea, much improved.
6. The second ancestry is Windows NT, started from nothing in 1988 by a team under Dave Cutler, whom Microsoft had recruited from Digital Equipment Corporation.
7. Cutler had designed the RSX-11M and VMS operating systems at Digital. NT is full of VMS ideas, and the resemblance was noticed immediately.
8. Windows NT 3.1 shipped on 27 July 1993. It had protection, preemptive multitasking, proper security and portability across processors.
9. For eight years Microsoft sold two families: the NT line for business, and Windows 95, 98 and Me for home users, which were still DOS underneath.
10. Windows XP in 2001 put everyone on the NT kernel. Every Windows since is NT.

PLAIN18.15.2 a picture in your head#

1. Think of a town with an old wooden high street and, on the edge, a new engineered town centre built to modern codes.
2. The wooden high street is the DOS line. Everything is convenient, nothing is fireproof, and one careless shop burns the row down.
3. The new centre is NT: fire doors, separate services, inspections. Slower to build, much heavier, and it does not burn down.
4. For years people kept shopping on the wooden street because that is where all the shops they liked were.
5. Eventually the new centre was made to look exactly like the old street, so nobody had to change their habits, and the wooden buildings were removed.
6. That disguise is the Win32 API, and it is the real reason the migration worked.

Where this comparison breaks: the disguise is not cosmetic. Win32 is a genuine programming interface with defined semantics, and NT implements it as one of several personalities over a lower interface. And the wooden street was never fully removed: enough compatibility behaviour survives in Windows 11 that programs from 1995 often still run.

PLAIN18.15.3 a worked example#

1. The registry is the part of Windows most often described wrongly, so here it is properly.
2. It is a hierarchical database of settings, stored in a small number of binary files called hives, and exposed as a tree of keys and values.
3. Before it, settings lived in hundreds of .ini text files scattered everywhere. There was no way to set permissions on them, no transactions, and no way to find them all.
4. The registry gave one namespace, per-value access control, and a defined API. That was a real engineering improvement, not a bureaucratic one.
5. The visible top-level keys and what they actually are:

6. Only HKEY_LOCAL_MACHINE and HKEY_USERS hold real data. The others are views and links, which is why the same value appears in two places.
7. On disk the hives are C:\Windows\System32\config\SYSTEM, SOFTWARE, SAM and SECURITY, plus NTUSER.DAT in each user’s profile.
8. Value types include REG_SZ (text), REG_DWORD (32-bit number), REG_BINARY and REG_MULTI_SZ.
9. Registry cleaners are, with rare exceptions, useless. A registry with 200,000 unused keys costs a modern machine no measurable time, because lookups are indexed. This is settled, not controversial.

PLAIN18.15.4 what is really happening inside#

1. The NT kernel is layered, and the layers have names worth knowing.
2. The HAL, hardware abstraction layer, is hal.dll. It hides differences between motherboards: interrupt controllers, timers, bus access. Above it, the rest of the kernel sees one abstract machine.
3. The kernel proper, sometimes called the microkernel, handles scheduling, interrupts, synchronization primitives and multiprocessor coordination. It is small and never paged out.
4. The executive sits above it and contains the real subsystems: the object manager, process manager, memory manager, I/O manager, cache manager, security reference monitor and configuration manager. All of these live in ntoskrnl.exe.
5. The object manager is NT’s most distinctive idea. Processes, threads, files, mutexes, registry keys and devices are all objects in one namespace with one set of rules for naming, referencing and securing.
6. Above the executive, in user mode, are subsystems: personalities that present a particular API. Win32 was one, and OS/2 and POSIX subsystems existed early on and were later removed.
7. That structure is why the Windows Subsystem for Linux was possible. WSL 1, in 2016, was a new subsystem translating Linux system calls to NT ones. WSL 2, from 2019, gave up and shipped a real Linux kernel in a lightweight Hyper-V virtual machine instead, because full syscall compatibility proved harder than virtualization.

TECHNICAL18.15.5 the engineer’s version#

1. NT 3.1 shipped for four architectures in 1993: Intel x86, DEC Alpha, MIPS and later PowerPC. That portability was designed in from the start, in contrast to early Linux.
2. NT’s design lineage from VMS is direct: asynchronous I/O request packets, the interrupt request level scheme, the object and handle model, and the layered driver stack all have VMS counterparts.
3. The version numbering is a source of confusion. NT 3.1, 3.5, 3.51, 4.0, then 5.0 was Windows 2000, 5.1 was XP, 6.0 Vista, 6.1 Windows 7, 6.2 Windows 8, 6.3 Windows 8.1, then 10.0 for both Windows 10 and Windows 11.
4. Current state as of August 2026: Windows 11 version 26H1, build 10.0.28000, released 10 February 2026, a platform release focused on ARM devices including Snapdragon X2. Version 25H2, build 26200, shipped 30 September 2025.
5. NTFS, shipped with NT 3.1 in 1993, is a journaling filesystem built around a Master File Table in which every file, including the MFT itself, is a record with a set of attributes. Small files live entirely inside their MFT record. It supports hard links, alternate data streams, compression, encryption, quotas and sparse files.
6. Alternate data streams are worth knowing about: file.txt:hidden is a second stream on the same file, invisible to dir, and historically a malware hiding place.
7. Win32 is the documented API, in kernel32.dll, user32.dll and gdi32.dll. It calls the undocumented native API in ntdll.dll. Microsoft’s compatibility promise attaches to Win32, not to the native layer.
8. Driver signing timeline: 64-bit editions have required signed kernel-mode drivers since Vista in 2006; since Windows 10 version 1607 in 2016, new kernel drivers must be submitted to and signed by the Microsoft hardware developer portal. Attestation signing covers most, EV certificates are required to submit.
9. Modern additions: virtualization-based security uses Hyper-V to isolate credentials in a separate virtual trust level; Windows 11 requires TPM 2.0 and UEFI Secure Boot; the July 2024 CrowdStrike incident, in which a faulty configuration update to a kernel-mode security driver bug-checked roughly 8.5 million Windows machines worldwide, restarted the industry argument about how much security software belongs in the kernel at all.
10. Tools: Sysinternals suite (Process Explorer, Process Monitor, Autoruns, WinObj, Handle), WinDbg with the public symbol server, Performance Monitor, Event Viewer, wevtutil, and PowerShell’s Get-Process and Get-Service.

WORDS18.15.6 remember these#

1. MS-DOS — the single-tasking ancestor — the 16-bit real-mode system bought from Seattle Computer Products in 1981 and shipped as PC DOS 1.0.
2. Windows NT — the engineered line that survived — Dave Cutler’s 1993 kernel with protection, portability and preemptive multitasking.
3. HAL — the layer that hides the motherboard — hal.dll, abstracting interrupt controllers, timers and bus access.
4. Executive — the main body of kernel services — object, process, memory, I/O, cache, security and configuration managers inside ntoskrnl.exe.
5. Subsystem — a personality presenting one API — user-mode environment such as Win32, and formerly OS/2 and POSIX.
6. Registry — one database instead of thousands of ini files — hierarchical configuration store in binary hives with per-value access control.
7. NTFS — the Windows filesystem since 1993 — journaling filesystem organized around a Master File Table of attributed records.

18.16 macOS, iOS, Android and the rest#

PLAIN18.16.1 in simple words#

1. Modern Apple systems come from a company Apple did not own in 1996.
2. Steve Jobs left Apple in 1985 and founded NeXT, which built expensive workstations and an operating system called NeXTSTEP.
3. NeXTSTEP combined the Mach kernel from Carnegie Mellon with BSD UNIX code and a set of programming frameworks written in Objective-C.
4. Apple, meanwhile, had spent most of the 1990s failing to replace the ageing classic Mac OS, which had no memory protection and cooperative multitasking.
5. Apple announced it would buy NeXT on 20 December 1996, and closed the deal on 7 February 1997 for about $427 million. Jobs returned with it.
6. NeXTSTEP became the base of Mac OS X, released for desktops on 24 March 2001.
7. The same foundation, with a different interface layer, is iOS, iPadOS, watchOS, tvOS and visionOS. One core, many products.
8. Android took a different route entirely: the Linux kernel, with none of the GNU userland, and an application layer designed from scratch by Google.

PLAIN18.16.2 a picture in your head#

1. Think of two ways of putting a new engine into an old car brand.
2. Apple bought a whole different car, rebadged it, and gradually made it look like the old one. The dashboard was familiar, everything under it was new.
3. Google took a well-known engine, threw away every other part of the donor car, and built a completely new vehicle around it.
4. So an Android phone runs Linux, and almost nothing you know about a Linux desktop applies to it: different C library, different init, different graphics, different IPC, different package format.
5. Calling Android “Linux” is true about the engine and misleading about the car.

Where this comparison breaks: Apple did not simply rebadge. XNU is a genuine hybrid whose Mach layer and BSD layer are welded together in ways neither original design intended, and it has been rewritten heavily over 25 years.

PLAIN18.16.3 a worked example#

1. XNU is the Apple kernel. The name stands for “X is Not Unix”, a joke from the NeXT era, and it is made of three parts.
2. Part one: Mach, from Carnegie Mellon University, originally a microkernel research project by Rick Rashid and Avie Tevanian. Apple uses a heavily modified OSF MK 7.3. It provides virtual memory, tasks, threads, scheduling and Mach ports for messaging.
3. Part two: BSD, derived from 4.3BSD and refreshed from FreeBSD. It provides the POSIX API, the process model, users and permissions, the network stack and the VFS.
4. Part three: IOKit, the driver framework, written in a restricted subset of Embedded C++ with no exceptions, no templates and no multiple inheritance.
5. All three run in one address space in kernel mode. Mach’s separation is a structure, not an isolation boundary. That is why XNU is called hybrid.
6. Darwin is the open-source part: XNU plus the BSD userland plus the core libraries. Apple publishes it. The frameworks, the interface and the applications on top are not open.
7. Above Darwin sit the frameworks: Cocoa and AppKit on macOS, UIKit on iOS, written in Objective-C, and SwiftUI, from 2019, written in Swift.
8. Objective-C came from NeXT. Swift was announced in June 2014 and is now the default for new Apple development, with Objective-C still fully supported.

PLAIN18.16.4 what is really happening inside#

1. System Integrity Protection, introduced in OS X 10.11 El Capitan in 2015, is a rule enforced by the kernel that even root may not modify protected system locations, load unsigned kernel extensions or attach a debugger to Apple binaries.
2. It is configured from the recovery environment, not from the running system, which is the point: a compromised running system cannot turn it off.
3. Since macOS 10.15 Catalina in 2019, the system volume is a separate read-only volume, and since Big Sur in 2020 it is a cryptographically sealed snapshot.
4. The Apple silicon transition was announced at WWDC on 22 June 2020. The first M1 Macs shipped in November 2020, and the transition finished with the Mac Pro in June 2023.
5. Two translation layers made it survivable: Rosetta 2, which translates x86-64 binaries ahead of time on install, and universal binaries containing both architectures in one file.
6. Android is a Linux kernel plus an entirely separate userland.
7. Its C library is Bionic, written by Google, not glibc. Its init is Android’s own. Its IPC is Binder, a kernel driver originally from Be Incorporated’s OpenBinder, doing object-oriented calls between processes.
8. Its application runtime is ART, the Android Runtime, introduced as a preview in Android 4.4 KitKat in 2013 and made the only runtime in Android 5.0 Lollipop in 2014, replacing the older Dalvik virtual machine.
9. ART started as purely ahead-of-time compilation. Since Android 7.0 Nougat it is a hybrid: new apps run interpreted and JIT-compiled while a profile is collected, and the hot code is compiled ahead of time when the device is idle and charging.
10. Between the kernel and the framework sits Android’s HAL, a set of defined interfaces every vendor must implement, so that Android can be ported without Google seeing the vendor’s driver source.
11. Project Treble, from Android 8.0 in 2017, split the vendor implementation from the framework so the framework can be updated independently. It was a response to the update problem, and it helped without solving it.

TECHNICAL18.16.5 the engineer’s version#

1. Current versions as of August 2026: macOS 26 Tahoe, released 15 September 2025, introducing the Liquid Glass design across Apple platforms.

2. Apple filesystem history: HFS from 1985, HFS+ from 1998, and APFS announced in 2016, shipped on iOS 10.3 in March 2017 and macOS 10.13 High Sierra in September 2017. APFS is copy-on-write with cheap snapshots, native encryption and space sharing across volumes in a container.
3. Security hardware: the Secure Enclave, a separate co-processor with its own boot ROM, present since the A7 in 2013, holding keys and biometric templates that the main processor never sees.
4. On Apple silicon, memory pages are 16 KiB rather than 4 KiB, which changes memory footprint measurements noticeably compared with x86 Macs.
5. Android’s kernel is a fork with substantial out-of-tree patches, though the Generic Kernel Image programme since Android 11 has pushed vendors towards a common core. Android security patches are shipped monthly on a published bulletin schedule.
6. ChromeOS is Gentoo-derived Linux with a hardened design: verified boot with a read-only rootfs, automatic background updates to an alternate partition, and Linux applications run inside a virtual machine using crosvm and Termina. Announced 7 July 2009, first Chromebooks June 2011.
7. Real-time operating systems are a different category with a different promise: not speed, but a bounded worst-case response time. Names worth knowing: VxWorks (which ran the Mars rovers Spirit, Opportunity, Curiosity and Perseverance), QNX Neutrino (a true microkernel, dominant in automotive infotainment), FreeRTOS, Zephyr and RTEMS.
8. The distinction that matters: hard real-time guarantees a deadline and treats a miss as a failure; soft real-time treats a miss as degraded quality. Linux with the PREEMPT_RT patch set, merged into mainline in Linux 6.12 in November 2024 after roughly twenty years of development, offers strong soft real-time and, with care, hard real-time on suitable hardware.

WORDS18.16.6 remember these#

1. XNU — Apple’s kernel — a hybrid of Mach for memory and messaging, BSD for POSIX and networking, and IOKit for drivers.
2. Darwin — the open part of macOS — XNU plus BSD userland and core libraries, published by Apple without the frameworks or interface.
3. Mach port — Apple’s basic IPC handle — a capability referring to a message queue, underlying most macOS inter-process communication.
4. SIP — root is not allowed either — System Integrity Protection, kernel enforced since OS X 10.11 in 2015, configurable only from recovery.
5. Binder — Android’s IPC — a kernel driver providing object-oriented calls between processes, derived from OpenBinder.
6. ART — Android’s application runtime — profile-guided hybrid of interpretation, JIT and ahead-of-time compilation, replacing Dalvik in Android 5.0.
7. RTOS — an operating system that promises deadlines — bounded worst-case latency rather than best average throughput.

18.17 How you would actually write one#

PLAIN18.17.1 in simple words#

1. Writing an operating system is not mysterious. It is a long series of small, well-documented steps, each of which is achievable in an evening.
2. The reason few people finish is not difficulty. It is that there are hundreds of steps and no user to please until quite far in.
3. The minimum thing that deserves the name is small: it must start on its own, take control of the processor, put something on the screen, and respond to a key press.
4. That is achievable in a weekend, in about 300 lines, and it is a real operating system in the same sense that a paper aeroplane is a real aircraft.
5. After that the work has a natural order, because each part needs the one before it.
6. You need interrupts before you can have a keyboard. You need a memory allocator before you can have processes. You need processes before a scheduler. You need a disk driver before a filesystem. You need a filesystem before a shell.
7. You do this on a simulated machine, not real hardware, because a simulated machine restarts in a second and lets you inspect every register.
8. Everything you need is documented, free, and has been done by thousands of people who wrote down what went wrong.

PLAIN18.17.2 a picture in your head#

1. Think of building a house on empty ground where nothing exists, not even roads.
2. First you need a track wide enough to bring in a digger. That is the bootloader: just enough to get the next thing in.
3. Then foundations: level ground, marked out, load bearing. That is switching the processor into its proper mode and setting up the tables it needs.
4. Then the wiring loom, before any room is finished, because everything depends on it. That is the interrupt table.
5. Then plumbing: a system for allocating what is scarce. That is the memory manager.
6. Only then do rooms appear, and people move between them. That is processes and the scheduler.
7. The kitchen and the front door come last, and they are the only parts a visitor ever notices. That is the filesystem and the shell.

Where this comparison breaks: a builder can inspect a half-built house by walking through it. When your kernel fails, the machine simply stops, with no message, because the code that would print a message is the code that broke. Building a way to see inside your own system is a real early task, not a luxury.

PLAIN18.17.3 a worked example#

1. The smallest complete thing: a boot sector that prints a character. This is real, assembles with NASM, and runs in QEMU.

    bits 16
    org 0x7c00

start:
    mov ah, 0x0e        ; BIOS teletype function
    mov al, 'K'
    int 0x10            ; call BIOS video service
    jmp $               ; loop here forever

    times 510-($-$$) db 0
    dw 0xaa55           ; boot signature

2. Line by line. bits 16 says generate 16-bit code, because the processor starts in real mode.
3. org 0x7c00 tells the assembler that this code will live at address 0x7C00, which is where BIOS loads a boot sector. That is fixed, not a choice.
4. mov ah, 0x0e selects the BIOS teletype output service, mov al, 'K' is the character, and int 0x10 calls the BIOS video interrupt.
5. jmp $ jumps to itself forever, because there is nothing to return to.
6. times 510-($-$$) db 0 pads with zeros up to byte 510.
7. dw 0xaa55 writes the two-byte signature that BIOS checks before it will treat the sector as bootable.
8. Build and run it with two commands:

nasm -f bin boot.asm -o boot.bin
qemu-system-i386 -drive format=raw,file=boot.bin

9. A window opens and shows the letter K. Nothing else on that machine exists. No operating system, no library, no memory manager, no C runtime. You wrote every instruction that ran.
10. That is 512 bytes and it is the honest starting point of every hobby OS.

PLAIN18.17.4 what is really happening inside#

1. The realistic order of work, with what each stage costs an average learner working evenings.
2. Boot and print — one weekend. The example above.
3. Protected mode or long mode — one to two weeks. Build a global descriptor table, set a bit in a control register, and jump. Getting the jump right is the classic first wall.
4. A C environment — one week. Write a linker script, a small startup stub, and build with a cross-compiler so you are not accidentally linking against your host’s C library.
5. Interrupts — two to three weeks. An interrupt descriptor table, entry stubs that save registers, remapping the interrupt controller, a timer and a keyboard.
6. Physical and virtual memory — one month. A page frame allocator, page tables, and a heap. This is where most projects stall, because bugs here produce silent corruption rather than error messages.
7. Processes and a scheduler — one month. Save and restore context, a task list, round robin. The first successful switch between two tasks is the most satisfying moment in the project.
8. User mode — two to four weeks. Separate address spaces, a system call entry, and a separate stack per privilege level.
9. A disk driver and a filesystem — one to two months. ATA PIO is simplest. FAT16 is the easiest real filesystem to read, and writing is much harder than reading.
10. A shell — two weeks, and it feels like finishing, because for the first time you can type at your own system and it answers.
11. Total, for one person working evenings: roughly six months to a year to reach a usable shell. Two weekends to reach something that boots and responds.

TECHNICAL18.17.5 the engineer’s version#

1. Tooling, all free. A cross-compiler built as i686-elf-gcc or x86_64-elf-gcc, so headers and libraries from your host cannot leak in. NASM or GAS for assembly. GNU ld with a custom linker script. GNU make.
2. Emulators: QEMU for speed and GDB support, Bochs for its built-in debugger which can single-step at the hardware level, and VirtualBox for a final check against something closer to real firmware.
3. Debugging technique that changes everything: run QEMU with -s -S and attach GDB to port 1234. You can then single-step your kernel from the very first instruction, with symbols.
4. Bootloader choices: write your own for learning, use GRUB with the Multiboot2 specification to skip straight to a 32-bit or 64-bit C environment, or use Limine for a modern UEFI-capable option.
5. Learning resources, by name.

6. Notes on those. The OSDev wiki is the single most useful page collection in this field, and its “Beginner Mistakes” page will save you weeks. Operating Systems: Three Easy Pieces, by Remzi and Andrea Arpaci-Dusseau at Wisconsin, is free online and is the best modern textbook. The little book about OS development, by Erik Helin and Adam Renberg, is the shortest complete path from nothing to a small x86 kernel. xv6 is MIT’s teaching reimplementation of Sixth Edition UNIX in ANSI C for RISC-V, with a line-by-line commentary book. Philipp Oppermann’s Writing an OS in Rust is the best written tutorial series of the last decade. SerenityOS was started by Andreas Kling in 2018 and is now a complete graphical UNIX-like system written from nothing including its own browser engine.
7. Specifications you will actually read: the Intel Software Developer’s Manual volume 3 for system programming, the AMD64 Architecture Programmer’s Manual volume 2, the Multiboot2 specification, the UEFI specification, the ATA and NVMe specifications, and the OSDev wiki’s summaries of all of them.
8. The honest scope warning. A hobby kernel that boots, schedules and runs a shell is perhaps 5,000 to 20,000 lines. A system you could daily-drive needs drivers for real hardware, a network stack, a graphics stack, a browser and a toolchain, and that is where the number reaches millions. SerenityOS took a full-time effort and many contributors over years to reach a usable desktop.
9. Why do it anyway: after building a scheduler you will never again be confused about what a context switch is, and after building a page allocator you will never again be confused about virtual memory. The understanding is disproportionate to the code.

WORDS18.17.6 remember these#

1. Cross-compiler — a compiler that targets a different system — a toolchain such as i686-elf-gcc that cannot accidentally use the host’s libraries.
2. Boot sector — the 512 bytes BIOS will run — loaded at 0x7C00 and required to end with the signature 0xAA55.
3. Global descriptor table — the table that describes memory regions — the x86 structure that must exist before entering protected mode.
4. Interrupt descriptor table — the list of handler addresses — the table mapping interrupt and exception numbers to entry points.
5. Multiboot2 — a standard way for a bootloader to hand over — the specification GRUB implements, delivering a kernel into a known state with a memory map.
6. Linker script — the file that decides where code lands — the ld input that defines sections and load addresses for a freestanding binary.

18.98 Common wrong ideas#

1. Wrong: the operating system is the desktop, the icons and the windows. Right: those are ordinary programs. The operating system is the kernel plus the services underneath, and a machine with no graphics at all still has a complete operating system.

2. Wrong: Linux is an operating system. Right: Linux is a kernel. What you install is a distribution, which adds a C library, a userland, an init system and tens of thousands of packages. Android proves the point: same kernel, none of the rest.

3. Wrong: a process and a program are the same thing. Right: a program is a file that does nothing. A process is that program while running, with its own memory, descriptors and scheduling state. One program can be many processes.

4. Wrong: multitasking means the processor runs several programs at once. Right: on one core it runs one thing at a time and switches quickly. Real simultaneity requires several cores. On this machine the kernel had performed 4,153,017 context switches in 95 minutes to produce the illusion.

5. Wrong: threads are always much cheaper than processes. Right: measured here, creating a thread cost 32.8 microseconds against 182.5 for a fork, about 5.6 times. But forking after dirtying 256 MB cost 2,620 microseconds, because page tables must still be copied. The ratio depends on what the process holds.

6. Wrong: malloc gives you memory. Right: it gives you addresses. Physical memory is allocated one 4 KiB page at a time, on the first write to each page, through a page fault. Asking for a gigabyte moves VSZ and leaves RSS unchanged.

7. Wrong: the out-of-memory killer means your machine ran out of RAM through bad management. Right: it means the kernel’s earlier promises could not all be kept at once. Overcommit is deliberate, because programs reserve far more than they touch.

8. Wrong: a system call is just a function call into a library. Right: it is a hardware mode transition. The syscall instruction changes privilege level and jumps to an address only the kernel could set. It cost 89 nanoseconds here against one or two for a normal call.

9. Wrong: write returning success means the data is on the disk. Right: it means the data is in the page cache and marked dirty. Only fsync, fdatasync or O_DIRECT involve the storage device. Databases care about this difference more than anything else.

10. Wrong: containers are lightweight virtual machines. Right: containers are ordinary processes with a restricted view, built from namespaces and cgroups, sharing the host kernel. A kernel vulnerability escapes a container and does not escape a virtual machine.

11. Wrong: a blue screen or kernel panic means the operating system is badly written. Right: it usually means a driver failed. Drivers run with full privilege, there is no higher authority to catch their errors, and stopping is safer than continuing with corrupted state.

12. Wrong: kill -9 is the proper way to stop a program. Right: SIGKILL cannot be caught, so the program cannot flush buffers, release locks or write its state. SIGTERM first, wait, then SIGKILL. A process stuck in D state ignores both.

18.99 Chapter summary in 20 lines#

1. An operating system exists to do three things: share the hardware, keep programs apart, and hide devices behind a simple interface.
2. All of its power comes from two hardware features: a privileged processor mode, and a timer interrupt that forces control back to the kernel.
3. The kernel is the privileged part. It is entered only three ways: a system call, an interrupt, or a fault. Between those it does nothing.
4. Kernel designs run from monolithic (Linux) through hybrid (Windows NT, XNU) to microkernel (MINIX 3, QNX, seL4); the 1992 Tanenbaum-Torvalds debate settled nothing and both approaches ship today.
5. Kernels are written in C for predictable code with no hidden runtime, plus assembly for entry paths, and since Linux 6.1 in December 2022, Rust.
6. Booting is a chain: power good, reset vector at 0xFFFFFFF0, firmware and POST, boot device, bootloader, kernel, initramfs, root filesystem, PID 1.
7. On the machine used here the kernel phase took 2.55 seconds, and PID 1 started at 1.17 seconds, before the real root was mounted at 2.55.
8. A process is a running program with its own address space, descriptors and state; UNIX creates one with fork then exec, Windows with CreateProcess.
9. Threads share everything except stack and registers, which makes them cheap and dangerous; the kernel schedules them individually in the 1:1 model.
10. The scheduler answers one question thousands of times a second, and every algorithm trades throughput against responsiveness; round robin loses on averages and wins on how the machine feels.
11. Linux used CFS from 2.6.23 in 2007 and replaced it with EEVDF in 6.6 in October 2023; Windows uses 32 priority levels with dynamic boosts.
12. A system call puts a number in rax, arguments in six registers, executes syscall, and lands where MSR_LSTAR points; printing “hello” took about twenty-five of them, of which one was the point.
13. The kernel allocates physical page frames lazily on first touch, overcommits deliberately, and kills the highest-scoring process when the promise fails.
14. The virtual filesystem layer dispatches through per-filesystem function tables, which is why /proc can be read like a file that does not exist.
15. Three tables sit behind every open file: the per-process descriptor table, the system-wide open file table, and the inode table.
16. UNIX permissions are owner, group, other with read, write, execute, plus setuid, setgid and sticky; Windows uses ordered access control lists.
17. Pipes, FIFOs, signals, message queues, shared memory, sockets and RPC differ in copies, message boundaries and whether they cross machines.
18. Containers are namespaces plus cgroups on a shared kernel; virtual machines add a hypervisor below the guest kernel and isolate far more strongly.
19. The lineage is one family: Multics failed, UNIX began on a PDP-7 in 1969, was rewritten in C in 1973, split into BSD and System V, was standardized as POSIX in 1988, and produced Linux in 1991, macOS via NeXT in 2001, and Android on a Linux kernel in 2008.
20. Every one of the 500 machines on the June 2026 TOP500 list runs Linux, as has been true since November 2017, while Windows NT’s 1993 design still runs most desktops and XNU runs every Apple device.