llama.cpp prefill speedups and minor ML library fixes

llama.cpp dominated the day with a high-impact path for lazy PLE table reads that more than doubles cold prefill speed, plus several smaller quantization and MoE proposals. PyTorch and Hugging Face Transformers saw only narrow security and tooling patches.

Direct-read path for lazy PLE tables in llama.cpp

PR 28136 in ggml-org/llama.cpp adds a direct-read path for the lazy PLE table. Benchmarks report up to twice the cold prefill performance and lower memory pressure on Apple, Windows, and Linux. The change matters for anyone serving large models where prefill latency and host memory pressure are the binding constraints.

Heap-buffer-overflow closed in PyTorch quantized max pool

A pull request fixing a heap-buffer-overflow in quantized_max_pool2d for one-dimensional arguments was closed in pytorch/pytorch. Maintainers elected to pursue a broader class-wide fix instead. Users of quantized pooling paths should watch for the follow-on change that will cover the same memory-safety hole.

Retrospective on two-tier MoE expert cache

A single-message thread in ggml-org/llama.cpp revisits an earlier RFC for a two-tier GPU-plus-RAM expert cache with pluggable eviction. The note records that the design later shaped several already-merged pull requests. MoE offload developers still benefit from the documented policy options when tuning memory hierarchy behavior.

Opt-in error on fallback quantization

PR activity in ggml-org/llama.cpp adds a --no-fallback flag to llama-quantize. When set, the tool errors instead of merely warning on required fallback quantization. The switch gives packagers and CI pipelines a deterministic failure mode rather than silent quality loss.

Lazy q8_0 quantization of the KV cache

A pull-request update in ggml-org/llama.cpp continues work on quantizing the KV cache to q8_0 only after the fp16 cache fills. The same change set also validates rotation state as a prerequisite. The approach lets long-context runs stay in higher precision until memory pressure actually appears.