Symmetric multiprocessing turns “the scheduler” into a collection of schedulers that must cooperate without quietly becoming one scheduler behind a giant lock. Yggdrasil’s answer is local by default: local state, local queues, local interrupts—and explicit protocols wherever cores meet.
Give every core a private world.
Each CPU has its own current process, preemption flag, run queue, scheduler context,
allocation hint, GDT, TSS, and interrupt stacks. A gs:[0] accessor makes
the current core’s state cheap to reach without pretending it is global.
That split forced a useful audit. Anything that remained global had to be deliberately shared, synchronized, and assigned a lifetime. Everything else became CPU-local or process-owned.
Local state is not merely faster. It makes ownership visible—and visible ownership is how concurrency stays debuggable.
Copy the message, never borrow the heap.
On one core, it is tempting to copy a term directly into the receiver’s heap. On four, that becomes a cross-core write into mutable process state. Heap fragments remove the temptation: senders build self-contained messages, and receivers copy them into their own heaps only when they receive.
The fragment is more than transport storage. It is a precise ownership handoff. A mailbox can move between scheduling contexts without making one process’s allocator a shared synchronization point.
Let idle cores steal work.
Every core schedules from its own queue. When it runs dry, it scans its siblings and steals a runnable process. Wake IPIs nudge idle cores; TLB-shootdown IPIs make recycled stack slots safe; panic IPIs stop the machine as one.
The scheduler could no longer use “not currently running here” as a synonym for “not running.” Publication, teardown, and reaping all needed cross-core protocols. Five bugs marked the places where the old single-core meaning leaked through.
- 01
Cross-core reap waited for the victim’s next safepoint.
- 02
Publish-before-save exposed a stale stack pointer.
- 03
Lock inversion tied process teardown to port cleanup.
- 04
Missing interrupt flags left schedulers spinning with IRQs off.
- 05
Steal-cycle deadlock let two queues wait on each other forever.
Write tests that cannot lie.
A four-core boot proves very little. The acceptance suite runs an unpreemptible spinner that only another core can release. It asserts that every CPU executes stolen work, churns 200 processes across the shootdown path, drives a bytecode echo server, reboots to check disk persistence, and recovers a real UDP payload from the virtual NIC.
$ cargo xtask test
[ok] smp: all 4 APs entered scheduler
[ok] smp: work stealing spread load
[ok] jit: differential suite
[ok] disk: pattern survived reboot
[ok] net: host payload observed in pcap
YGGDRASIL SELFTEST PASS