freenode
AI & ML

torch.compile over-reads offset views in Triton kernels

PyTorch can silently return wrong results when compiled kernels skip writing masked output regions.

torch.compile can silently produce incorrect results when a tensor view with a non-zero storage offset is passed as a mutated pointer argument into a raw Triton kernel. The compiled path reads past the end of the underlying buffer into neighboring memory.

The over-read is usually invisible because the kernel rewrites its entire output region. It surfaces when a kernel deliberately skips writing part of that region, for example by returning early on masked or ignored rows. Leftover values from the out-of-bounds read then remain in the result.

The failure was first noticed while compiling Liger-Kernel fused linear loss routines. Under torch.compile those kernels returned wrong losses whenever any label used the ignore index, diverging from eager execution. A workaround shipped in Liger-Kernel, but the underlying handling of offset and strided views belongs to PyTorch.

CUDA memory checkers often miss the fault. Small over-reads still land inside page-mapped allocator memory owned by the process, so they do not trap even though the logical access is out of bounds.