[JiTianYu391]

Depends on #112 — please merge #112 first. Both PRs rewrite the same lines of mem/alloc.c (the page-tagging loop in page_alloc() and the counter block of heap_get_stats()), so each merged cleanly into develop on its own but conflicted with the other. This branch is now rebased onto #112: the first commit is #112 unchanged, the second is this PR. After #112 lands, only the second commit remains in the diff.

Problem

Every large malloc(), every slab growth/shrink, and (via pointer_owner()) every free() take heap.page_lock, the one lock of the kernel heap's buddy allocator. It does not scale: with the in-kernel benchmark from #110 (kbench, 5 vCPUs, KVM) a 64 KiB malloc()+free() loop costs 144 ns/op on one CPU but 2364 ns/op when every CPU runs it, i.e. ~16x slower. A temporary lock-stat counter (not included) showed ~45% of acquisitions of this lock finding it already held; no other lock (scheduler.lock, frame_allocator.lock, printk_lock, pid_hash_lock) was contended in any kbench test.

Fix (same idea as Linux's per-CPU pagesets)

• Freed blocks of orders 0..5 (up to 128 KiB) are parked in a small per-CPU cache and handed back from there; the cache exchanges blocks with the buddy in batches (half the limit), so page_lock is only taken on a miss or when the cache is full. Limits per order are 8/8/6/4/3/2 blocks.
• A cached block stays allocated in the buddy (it cannot coalesce) but has its tags cleared, so pointer_owner() and the double-free detection reject it exactly like a free block. page_free() claims the head tag with a compare-and-swap, so a repeated or racing free of one block fails instead of being cached twice.
• If the buddy cannot satisfy an allocation, all CPU caches are drained once and the allocation retried, so cached memory cannot cause a spurious out-of-memory. heap_get_stats() counts cached pages as free.
• pointer_owner() reads the page tag without page_lock: the tag of a valid pointer is stable while its block is allocated, and an invalid pointer is still rejected by the owner's magic/cookie checks (the same change as in mm: scale kmalloc with bounded per-CPU magazines #112, which this PR is built on).
• The cache is bypassed until the scheduler has started (CPU topology is not usable before that), like the frame allocator's PCP lists.

Measurement

kbench, 5 vCPUs, A/B alternating, 5 boots per side, 3 rounds per boot, median of per-boot medians (>20% is the only effect I claim). Measured on codex/perf-validation (develop + #108/#109/#111/#112 + #110) with this patch applied, because plain develop misbehaves under kbench without the scheduler fixes.

test	before	after	ratio
malloc/65536/all-cpus (ns/op)	2364.0	780.4	0.33
malloc/65536/solo	143.9	143.3	1.00
malloc/4096/all-cpus	181.6	187.5	1.03
malloc/64/solo, all-cpus	99.7, 105.4	100.4, 104.0	1.01, 0.99
yield, wq-pingpong, pipe-*, kthread/create-exit, wq-prepare-cancel			all within ±10% (noise)

The in-kernel heap self-check used during testing (17 size classes, batch overflow, realloc, exactly one expected rejected double free, heap_validate()) reported 0 failures in all 10 boots.

Why not solo-level yet

With the global statistics atomics (allocation_calls, live_allocations, allocated_bytes, free_calls) temporarily disabled in the large-allocation path, malloc/65536/all-cpus measured 123 ns (= solo). The remaining 780 ns is those 6 shared-cacheline atomic operations per malloc/free pair, not a lock. Making them per-CPU belongs on top of #112 (which already has per-CPU counters for the slab path), so it is not part of this PR.

Notes

• mem/alloc.c only; no Kconfig, sizes the table with mm: scale kmalloc with bounded per-CPU magazines #112's PCP_MAX_CPUS (CONFIG_FRAME_PCP_MAX_CPUS) (~64 KiB of bss at the default 256).
• Not covered by an independent plain-develop link test here: develop currently needs the strcpy LTO workaround (libs: retain strcpy for compiler-generated LTO references #109) to link; the stacked branch (mm: scale kmalloc with bounded per-CPU magazines #112 + this PR, on develop@153578b5) builds without new warnings and boots to the shell with 5 vCPUs under KVM.

🤖 Generated with Claude Code