Btrfs folio lock leak hangs KVM guests on file-backed RAM

A regression in the 7.2 release candidates can permanently lock a btrfs large data folio during writeback, deadlocking KVM guests whose RAM is a file on btrfs and stalling host writeback, reclaim, and sync paths behind the same lock.

Christian Borntraeger reported the failure from s390 CI runs: guests with file-backed RAM (zstd compression) hung for hours alongside host flusher workers, khugepaged, and syncfs. Crash dump analysis pointed at a single 256K folio that stayed PG_locked and dirty with an empty btrfs subpage dirty bitmap, no writeback flag, and no I/O in flight. Waiters included vCPU threads and an irqfd worker blocked in btrfs page-mkwrite while holding the mmap lock.

The trigger is a folio marked dirty only at the generic folio level, for example by set_page_dirty_lock after a GUP pin, without any subpage dirty bit or delalloc reservation. btrfs data mappings use the generic dirty-folio helper, so that path never updates the subpage bitmap. When writepage_delalloc later sees an empty submission bitmap it assumes asynchronous submitters already own the unlock and returns early; nothing was submitted, so the folio is never unlocked.

Small single-block folios are insulated because the copy helper always reports bit 0 set. The leak hits subpage configurations since the submission bitmap arrived in 6.12, and is far easier to hit on ordinary 4K systems once btrfs gained large data folios.

Btrfs developer Qu Wenruo questioned whether dirtying a folio without notifying the filesystem via page-mkwrite is valid at all, citing earlier MM guidance that filesystems cannot be expected to handle it. Borntraeger then supplied a pure userspace O_DIRECT reproducer; Qu reproduced the hang on arm64 with 64K pages and called the situation severe, noting that ordered-folio detection code that might have caught it had recently been removed. A fix is under discussion while the wider dirty-without-fs-notify question is raised with the MM side.