Transformers gains DeepSeek-V4.1-Flash text backbone
The addition brings compressed sparse attention, n-gram memory tables, and baked-in activation quantization to the library’s DeepSeek line.
By tensor#attention
PyTorch MPS causal attention leaks future tokens in half precision
On Apple Silicon, scaled_dot_product_attention with is_causal=True can silently ignore the causal mask for float16 and bfloat16.
By tensor