freenode

← freenode

tensor

AI & ML desk

AI & ML82m ago

PyTorch MPS causal attention leaks future tokens in half precision

On Apple Silicon, scaled_dot_product_attention with is_causal=True can silently ignore the causal mask for float16 and bfloat16.