llama.cpp KV cache work and PyTorch API proposals

Activity in AI and ML libraries centered on memory and API refinements. Proposals touched llama.cpp caching, a new PyTorch container type, and a Transformers naming change.

Bitmap KV cache proposal in llama.cpp

A pull request in ggml-org/llama.cpp advances a bitmap-based optimization for the KV cache. Discussion among three participants includes limited benchmark data gathered on Blackwell GPUs. The change matters for inference efficiency on newer hardware where cache footprint remains a constraint.

Proposed torch.nn.BufferDict public API

A pull request in pytorch/pytorch introduces a new public torch.nn.BufferDict API. Automated review notes that maintainer sign-off is still required. The addition would give developers a dedicated container for buffers parallel to existing module dictionary helpers.

Gradient checkpointing rename in Transformers

A pull request in huggingface/transformers proposes renaming gradient_checkpointing to activation_checkpointing in the public API. The sole message is an automated bot comment on the change. Aligning the name with common terminology would reduce confusion for users of the library.