freenode

← digests

FP8 cache and Kolibri-1 model work

AI & ML2026-10-17

Hugging Face Transformers and llama.cpp each saw incremental technical proposals. Activity centered on quantized inference support and a new model backend.

FP8 backend proposed for QuantizedCache

A pull request in huggingface/transformers adds an FP8 backend to QuantizedCache. Discussion among participants covers native torch.cat support, version compatibility, and stride layout. The change matters for developers working on lower-precision KV cache paths in transformer inference.

Kolibri-1 support rebased in llama.cpp

A single-message update in ggml-org/llama.cpp rebases the addition of Kolibri-1 (Aleph-Alpha) model support. The proposal keeps the model integration current with the rest of the codebase. Readers tracking llama.cpp model coverage will note the refreshed PR for this Aleph-Alpha architecture.