Linux NUMA series unblocks promotion of hot slow-tier memory
Filters meant to calm socket placement were stranding hot pages on CXL, and Meta patches separate the two jobs.
Gregory Price of Meta has posted a four-patch series that separates memory-tier promotion from ordinary NUMA placement so hot pages can leave slow tiers even when socket-placement filters would have blocked them.
NUMA balancing uses hinting faults both to place tasks near their memory and to promote hot pages from slower memory such as CXL into DRAM. Filters added to skip unproductive placement work, including skips for read-only file mappings, shared copy-on-write folios, and VMAs without recent process activity, also stopped promotion. Hot data in those mappings could remain on a slow tier indefinitely.
The series introduces promotion-only protection walks so the scanner can still raise hint faults for promotion candidates without re-enabling placement sampling. Eligible shared folios may move from a slow tier to a fast one while ordinary placement keeps its existing restrictions. Read-only file mappings and PID-inactive VMAs are scanned for promotion only. Placement scan accounting is tracked separately so promotion work cannot postpone the existing placement-starvation fallback.
On a host with 768 GB of DRAM and 256 GB of CXL running two roughly 430 GB database workloads, Price reported DRAM bandwidth rising from 150 to 200 GB/s to over 250 GB/s, CXL bandwidth falling from a saturated 40 to 45 GB/s to about 10 GB/s, and request latencies dropping from multi-millisecond tails into the 800 us to 2 ms range. A 20 GB hash table that had remained entirely on CXL split evenly between DRAM and CXL afterward and tracked hotness.
Price wrote that the earlier filters "never took both modes (NORMAL, TIERING) into account in the first place" and "optimized for NORMAL and broke TIERING." Reviewer David Hildenbrand pressed for clearer helpers and cleaner mode logic. Peter Zijlstra observed that hint faults are "a big hammer making up for the lack of hardware support for getting this data." The series targets stable kernels as a fix for earlier tiering and VMA-scanning changes.