llama.cpp doubles cold prefill with direct PLE table reads

llama.cpp has landed a direct-read path for lazy PLE tables that more than doubles cold-cache prefill throughput on the hardware the change targets, with measured gains of 91% to 133% on Windows once the cache is genuinely empty.

The work matters most when the PLE table cannot stay resident. Under ordinary mmap, a large varied prefill faults the table into the page cache as a multi-gigabyte transient spike. On unified-memory machines that spike competes with weights and KV cache and can tip the system into pressure or OOM. Direct reads under the existing lazy mode keep the resident footprint nearly flat, so larger backbones and higher-precision PLE tables remain usable.

On an M5 Max with 128 GB of unified memory, the speedup itself was a wash when warm, but the memory behavior was decisive: a representative 105k-token prefill stayed wired with only a few hundred MiB of page-ins and no swap. On Windows, earlier warm-only numbers had hidden the effect; with a cold protocol that flushes the standby list and starts a fresh server process, the same direct path produced the large cold-cache gains on an RTX 3090 system whose heavier weights left too little RAM for the table to stay cached.

The change is therefore both a prefill-speed fix for cold and memory-constrained runs and a headroom feature that lets users keep bigger models and richer PLE tables without being pushed into swap or OOM.