Grouped CPU matmul and speculative decoding drafts
PyTorch developers proposed a oneDNN CPU backend for grouped matrix multiplies aimed at AMD Zen, while a llama.cpp change claims large token-rate gains from a speculative decoding tweak. A separate PyTorch custom-ops registration was sent back to draft over API issues.
CPU grouped matmul proposed for PyTorch
An RFC on the PyTorch repository proposes a CPU backend for torch._grouped_mm based on oneDNN's grouped matmul. It would replace the current per-expert fallback, with AMD Zen CPUs named as the target. The change matters for mixture-of-experts workloads that still fall back to per-expert multiplies on CPU.
llama.cpp skips minimum probability on some drafts
A pull request in llama.cpp proposes skipping the minimum speculative decoding probability for sampled MTP and DFlash2 drafts. The submission includes benchmarks that claim token-rate gains of 20 to 43 percent. The figures are the author's reported results, not an independent measurement.
Custom ops non-differentiable registration back to draft
A PyTorch pull request that would add explicit non-differentiable registration for custom ops was returned to draft. The update cites unresolved dispatch and API issues. The registration path is not ready for review in its current form.