btrfs: raid56: fix use-after-free of rbio in bio end_io wakeups
The read/write submission rounds in raid56 count in-flight bios in
rbio->stripes_pending and wait with a bare
wait_event(rbio->io_wait, atomic_read(&rbio->stripes_pending) == 0),
while each bio's end_io does
"if (atomic_dec_and_test(&stripes_pending)) wake_up(&rbio->io_wait)".
Once the final decrement makes the counter visible as zero, the waiter
can pass its condition check without consuming the wakeup -- via
wait_event()'s fast path when all bios complete before the waiter
arrives, or a condition recheck after an earlier wakeup -- and proceed
to free the rbio through rbio_orig_end_io(). The end_io context is
then still inside wake_up() operating on the freed rbio's embedded
waitqueue lock.
This shows up under sustained raid56 RMW load in KVM guests as
recurring "pvqspinlock: lock ... has corrupted value 0x0!" warnings
from __pv_queued_spin_unlock_slowpath with a
__wake_up <- raid_wait_write_end_io <- bio_endio call trace: paravirt
spinlocks detect the unlock of the recycled lock word. On bare metal
the use-after-free is silent and almost always harmless, which is how
it has survived; it is a plausible match for long-standing sporadic
crash reports on busy raid5 filesystems that never reproduce under
sanitizer kernels (the instrumentation widens the dec-to-wake window
so the race is always lost).
Convert the pair to a completion. wait_for_completion()'s fast path
takes the completion's own lock, so it cannot return before complete()
has released it: the completing context is provably finished with the
rbio before the waiter can free it. A submitter-held bias count keeps
one completion per submission round even when a round submits zero
bios or every bio finishes before the submitter starts waiting; the
scrub path only waits when finish_parity_scrub() actually began a
round.
projects / linux / commit