btrfs: stripe_alloc: keep a stripe in the rbio cache until its run has written all of it
A stripe run hands out its sectors in order, so a stripe it allocates is
often written in pieces: a write that ends inside the stripe leaves the
rest to later writes, and writeback allocates a whole range before any of
its bios complete, so even a stripe the frontier has already passed can
still have writes in flight. The later writes cannot pad (the earlier
sectors are live), so unless the stripe is still in the rbio cache each
is a read-modify-write that reads the earlier sectors back from the
devices. Those sectors are not yet covered by the csum tree (their
ordered extents have not finished), so what the devices return goes into
the new parity unverified. A device that returns wrong data there
destroys the earlier write, whose only intact copy at that point may be
the parity: the strip-fault fixture (a device that returns random bytes
for one strip per stripe) lost a file exactly this way, with every RMW of
the run in this same-run class.
The cache already prevents that read when the stripe is still cached: the
later write steals the earlier write's pages (lock_stripe_add()) and reads
nothing. But the cache is an LRU that any other stripe's write can push
such a stripe out of. Pin it instead. Each run gets a written map beside
its live map, set from each completed raid56 write's covered data sectors
(rbio_orig_end_io(), before the cache decision in unlock_stripe()).
cache_rbio() pins an rbio whose stripe is still waiting for writes
(btrfs_stripe_run_incomplete()): an open run's allocation frontier lies
inside it -- the rest is not even allocated yet, so every sector
allocated so far may already be written -- or a run covering it, open,
closed or kept for relocation, has live sectors there not yet written.
The prune skips live pins and re-checks each against the runs, so a pin
ends as soon as the stripe is complete or its run goes away. Nodatacow
runs are excluded from the written check: they are rewritten in place
for as long as the inode lives and are never complete. RBIO_PIN_MAX
bounds the pinned entries; no memory is allocated for this, the stripe's
pages are only released later. A completed write is marked in the run
that owns it, by [run->start, run->end): the maps reach past run->end
(they span at least BTRFS_STRIPE_RUN_LIVE_STRIPES stripes), so a write
must never be recorded in an earlier run whose map merely overlaps it.
written_not_live in stripe_park_stats counts completed-write sectors that
are not live in their run, which should stay near zero.
This removes the read rather than adding a recovery step for it: nothing
here depends on checksums, sectors without them, or the reconstruction
path. What remains is a stripe's first partial write, which reads only
sectors nobody has written yet (traced on the fixture: every remaining
data RMW read was of that kind). rbio_pinned in stripe_park_stats counts
the pins.
An ordered extent can end without writing all of its sectors (a write
error, truncation of folios never submitted, a short direct write, teardown
of an extent whose IO never started). When it reports to its run, its
whole range is marked written: no write of those sectors is still coming,
which is all the pin (and any other waiter on the written map) needs to
know. Marking them written rather than dead keeps them from being padded,
so nothing a truncated extent kept is zeroed. Waiters are woken while the
run lock is held, since the run and its group can be freed once it is
dropped.
[zygo: 7.3 port: 7.3 has no per-chunk stripe length (map->stripe_len,
bioc->stripe_len, the two-argument btrfs_stripe_nr_to_offset()):
BTRFS_STRIPE_LEN is used instead; the change itself is unchanged]