applied at log commits, lets a crash with all devices present recover
completed fsyncs intact, and bounds a degraded crash's fsync-window
loss to detectable absence -- never silent, never damaging committed
-data (patches 14-16); patches 17-18 and 22-24 remove the latency and
-IO overheads this adds to fsync. Patches 19-21 restore
+data (patches 13-15); patches 16-17 and 21-23 remove the latency and
+IO overheads this adds to fsync. Patches 18-20 restore
write-in-place for nodatacow files and preallocated extents as an
opt-in policy, with the write hole those writes reintroduce provably
-confined to the writing file alone. Patch 39 extends the same
+confined to the writing file alone. Patch 38 extends the same
allocation rule to raid56 metadata, closing the write hole for tree
-blocks whenever the metadata profile is raid56.
+blocks whenever the metadata profile is raid56. Patch 39 makes the
+policy safe to switch on at runtime on a filesystem the legacy
+allocator was writing a moment earlier.
Relationship to the other mitigation strategies: the raid-stripe-tree
addresses issues specific to zoned (ZNS) devices and remains the right
4 btrfs: report stripe run data IO through the ordered extent lifecycle
5 btrfs: retire stripe runs at commit and gate stripe_alloc
6 btrfs: add a write-hole invariant checker to the raid56 write path
- 7 btrfs: stripe_alloc: refuse space_cache=v1
- 8 btrfs: account stripe_alloc trapped free space for honest statfs
+ 7 btrfs: account stripe_alloc trapped free space for honest statfs
Packing density:
- 9 btrfs: stripe_alloc: keep an open run per size band to trap less
- 10 btrfs: stripe_alloc: grow the frontier run instead of stranding its tail
+ 8 btrfs: stripe_alloc: keep an open run per size band to trap less
+ 9 btrfs: stripe_alloc: grow the frontier run instead of stranding its tail
IO batching and robustness:
- 11 btrfs: raid56: batch stripe_alloc partial writes into full-stripe writes
- 12 btrfs: stripe_alloc: never claim stripes covered by a live run
+ 10 btrfs: raid56: batch stripe_alloc partial writes into full-stripe writes
+ 11 btrfs: stripe_alloc: never claim stripes covered by a live run
Interface:
- 13 btrfs: stripe_alloc: control the policy with a filesystem property
+ 12 btrfs: stripe_alloc: control the policy with a filesystem property
fsync guarantee:
- 14 btrfs: stripe_alloc: settle a logged extent's stripes at log commit
- 15 btrfs: stripe_alloc: private per-inode stripe runs for log-active inodes
- 16 btrfs: stripe_alloc: carry an inode's log tail forward at fsync
+ 13 btrfs: stripe_alloc: settle a logged extent's stripes at log commit
+ 14 btrfs: stripe_alloc: private per-inode stripe runs for log-active inodes
+ 15 btrfs: stripe_alloc: carry an inode's log tail forward at fsync
fsync latency:
- 17 btrfs: stripe_alloc: kick parked rbios before the fast fsync's writeback wait
- 18 btrfs: raid56: pad sub-stripe writes to full stripes in open runs
+ 16 btrfs: stripe_alloc: kick parked rbios before the fast fsync's writeback wait
+ 17 btrfs: raid56: pad sub-stripe writes to full stripes in open runs
nodatacow/prealloc write-in-place:
- 19 btrfs: stripe_alloc: isolate nodatacow and preallocated extents by stripe
- 20 btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents
- 21 btrfs: stripe_alloc: persist nocow runs across commits and remounts
+ 18 btrfs: stripe_alloc: isolate nodatacow and preallocated extents by stripe
+ 19 btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents
+ 20 btrfs: stripe_alloc: persist nocow runs across commits and remounts
Parking edges and observability:
- 22 btrfs: raid56: do not park sync writes into nocow runs
- 23 btrfs: raid56: expose parking statistics in sysfs
- 24 btrfs: raid56: skip parking for sync rbios finished by their unplug batch
+ 21 btrfs: raid56: do not park sync writes into nocow runs
+ 22 btrfs: raid56: expose parking statistics in sysfs
+ 23 btrfs: raid56: skip parking for sync rbios finished by their unplug batch
Reservation integrity:
- 25 btrfs: stripe_alloc: count open stripe run remainders against data reservations
- 26 btrfs: stripe_alloc: pessimistic data reservation margin
- 27 btrfs: stripe_alloc: claim fully-free stripes that span free space entries
+ 24 btrfs: stripe_alloc: count open stripe run remainders against data reservations
+ 25 btrfs: stripe_alloc: pessimistic data reservation margin
+ 26 btrfs: stripe_alloc: claim fully-free stripes that span free space entries
Coverage observability:
- 28 btrfs: stripe_alloc: warn at mount when raid56 metadata is not covered
- 29 btrfs: stripe_alloc: report read-modify-write of uncovered stripes
- 30 btrfs: raid56: say whether a metadata read-modify-write is a write hole
+ 27 btrfs: stripe_alloc: warn at mount when raid56 metadata is not covered
+ 28 btrfs: stripe_alloc: report read-modify-write of uncovered stripes
+ 29 btrfs: raid56: say whether a metadata read-modify-write is a write hole
Parking refinement:
- 31 btrfs: stripe_alloc: tunable park deadlines, and say why padding declined
- 32 btrfs: stripe_alloc: trace why padding declined, at the moment it declined
- 33 btrfs: stripe_alloc: complete parked writes on allocation coverage, not clocks
- 34 btrfs: stripe_alloc: track per-sector liveness and pad from the map
- 35 btrfs: stripe_alloc: count claimable whole-stripe supply directly
- 36 btrfs: raid56: adaptive park deadline from the parked-rbio backlog
+ 30 btrfs: stripe_alloc: tunable park deadlines, and say why padding declined
+ 31 btrfs: stripe_alloc: trace why padding declined, at the moment it declined
+ 32 btrfs: stripe_alloc: complete parked writes on allocation coverage, not clocks
+ 33 btrfs: stripe_alloc: track per-sector liveness and pad from the map
+ 34 btrfs: stripe_alloc: count claimable whole-stripe supply directly
+ 35 btrfs: raid56: adaptive park deadline from the parked-rbio backlog
Admission and allocation cost:
- 37 btrfs: stripe_alloc: gate data admission on claimable whole-stripe supply
- 38 btrfs: stripe_alloc: cache a failed stripe run scan per block group
+ 36 btrfs: stripe_alloc: gate data admission on claimable whole-stripe supply
+ 37 btrfs: stripe_alloc: cache a failed stripe run scan per block group
Metadata coverage:
- 39 btrfs: stripe_meta: allocate tree blocks by whole stripes
+ 38 btrfs: stripe_meta: allocate tree blocks by whole stripes
+
+Runtime enable:
+ 39 btrfs: stripe_alloc: drain the legacy allocator's writes before arming the write-hole check at a runtime enable
+
+A further 26 patches -- reservation accounting at the fill edge, the
+relocation and reclaim paths, and the metadata reserve under raid56
+metadata -- are complete and validated but still under review, and
+will be folded into this series before it is posted. They are
+described under "Pending" at the end of this letter so that nothing
+measured on the current test kernels is attributed to a patch that is
+not in the list above.
Prerequisites
=============
These are not part of this series -- they are independent raid56 and
error-handling fixes, sent separately -- but the series is developed and
tested on top of them, and the guarantees below are stated with them
-applied:
+applied. Every one was found by this series' test harness.
+
+raid56 fixes (a separate series):
btrfs: propagate a split bio's error when the failing split completes last
Required for "detectable, never silent". Without it, a read of
unreconstructible stripe and proven byte-exactly (the parity
solved uniquely as XOR including pre-corruption bytes that
existed nowhere on disk). stripe_alloc additionally reorders its
- padding check ahead of the cached-pages shortcut (patch 18), so a
+ padding check ahead of the cached-pages shortcut (patch 17), so a
claimed stripe's first partial write rewrites every column it
does not cover.
- btrfs: don't abort raid56 data scrub on first uncorrectable sector
- Not required by the series. The harness scrubs deliberately damaged
- filesystems, which stock scrub abandons at the first uncorrectable
- stripe, so the verification below depends on it.
-
- btrfs: log a message when dropping an extent due to IO error
- Not required by the series; this is the patch referred to in "A
- second failure mode" that makes the silent extent drop visible.
+ btrfs: raid56: keep a reference on a stolen cached rbio until it is dropped
+ btrfs: raid56: do not unhash a cached rbio that has rbios plugged on it
+ Two use-after-free races around the rbio cache, both reachable
+ with the in-tree cache shrink and any concurrent lock_stripe_add():
+ the steal path in lock_stripe_add() drops the hash reference
+ before the cache is done with the rbio, and unlock_stripe()
+ caches an rbio with RBIO_RMW_LOCKED_BIT still set so a
+ lock_stripe_add() in that window can only plug onto it, after
+ which a cache removal hits BUG_ON(!list_empty(&rbio->plug_list)).
+ Both crashed the test VMs in the stock-allocator pass of the
+ acceptance suite under a silently corrupting device; the previous
+ patch's cache trimming only makes them frequent.
btrfs: raid56: verify reconstructed tree blocks
Reconstructed data sectors are csum-verified before use, but a
metadata read path performs. Independent of this series; found
while extending the harness to raid56 metadata.
- btrfs: do not write repaired read sectors back on a read-only mount
- btrfs_end_repair_bio() write-backs every repaired sector with no
- check of the mount state, tripping
- ASSERT(!(sb->s_flags & SB_RDONLY)) in btrfs_repair_io_failure()
- -- and without assertions it writes to a filesystem the user was
- promised is read-only. A degraded raid56 mounted ro,degraded
- hits it on every stray-sector read, since reconstruction through
- parity counts as a repair. Found by the metadata crash matrix
- below.
+ btrfs: don't abort raid56 data scrub on first uncorrectable sector
+ Not required by the series. The harness scrubs deliberately damaged
+ filesystems, which stock scrub abandons at the first uncorrectable
+ stripe, so the verification below depends on it.
+
+ btrfs: log a message when dropping an extent due to IO error
+ Not required by the series; this is the patch referred to in "A
+ second failure mode" that makes the silent extent drop visible.
+
+delalloc fixes (a separate series, both found running this series to
+ENOSPC under fsstress, neither specific to it):
+
+ btrfs: release the space of delalloc ranges abandoned by a failed writeback
+ writepage_delalloc()'s error branch only unlocks the delalloc
+ ranges after the one that failed in a multi-range folio; the
+ folio was already cleared dirty, so EXTENT_DELALLOC sits on a
+ clean folio until reclaim clears it without a release, leaking
+ the inode's reservation and the data space_info's bytes_may_use
+ (WARN_ONs at eviction and unmount). Marked for stable.
+
+ btrfs: drain pending NOCOW writes before making a block group read-only
+ A nodatacow write() that finds no data space decides to write in
+ place and holds nothing; scrub, balance and zoned reclaim make the
+ group read-only without flushing, and the writeback then falls
+ back to COW with a reservation that never went through admission
+ -- dropped at ENOSPC. The fix drains the group before the flip.
+
+Deadlock fixes (a separate series; all three surfaced on the soak rig
+below, none involves raid56):
+
+ btrfs: fix flushoncommit deadlock when cloning an inline extent inside i_size
+ btrfs: reflink: never block on space reservations while the locked range has delalloc
+ btrfs: relocation: reserve all of a folio's slices before dirtying any
+ The same shape three times: a task blocks on a transaction join or
+ a space reservation while holding a locked dirty folio or an
+ extent lock over delalloc, the flusher needs that lock, and the
+ commit (flushoncommit, or the reclaim commit that services the
+ ticket) waits for the flusher. The first reproduces on a stock
+ single-device 6.12 with flushoncommit plus dedupe of inline
+ extents; the third is reachable on plain x86 because btrfs uses
+ large data folios (up to 64 sectors per folio).
+
+lib/raid6 (sent to the md maintainers, Cc btrfs and stable):
+
+ lib/raid6: fix the x1 SIMD gen_syndrome variants for a single data disk
+ The sse1x1/sse2x1/avx2x1/avx512x1 gen_syndrome loops preload
+ dptr[z0-1] before iterating from z0-2, so with one data disk
+ (disks == 3, z0 == 0) they read pointers[-1]: a NULL oops or, if
+ the neighbouring bytes happen to be a pointer, P and Q silently
+ XORed with an unrelated page. md never builds a 3-disk raid6;
+ btrfs does (devs_min = 3), and any raid6 array whose devices are
+ uneven in size produces 3-wide chunks once the small devices fill.
+ Found by the raid6 coverage run below as a "rare" panic that was
+ in fact deterministic on every boot where the boot-time benchmark
+ picked an x1 variant.
+
+Two more bugs found the same way were fixed upstream independently
+before this posting and are carried only on the 6.18 backport:
+btrfs_end_repair_bio() writing repaired sectors back on a read-only
+mount (upstream 8ceaad6cd6e7, v7.0; a stable backport is the right
+action for 6.18) and defrag reserving with a flush while holding a
+locked cluster (upstream ba02eab28041f, v7.3, fixed more broadly than
+our patch).
The write hole
==============
excludes partially filled stripes statelessly, so retired stripes
become unallocatable with nothing written to disk.
-Two allocator refinements keep the packing dense (patches 9-10). Open
+Two allocator refinements keep the packing dense (patches 8-9). Open
runs are kept per power-of-two band of remaining space (segregated best
fit), so a small write lands in an already-open, nearly full stripe
instead of opening a fresh one; and when an allocation outgrows every
all, a partial write rbio for such a stripe can be parked -- stripe
lock held, RMW not started -- collecting later writes through the
existing merge path until it goes down as one full-stripe write: one
-parity computation, no reads (patch 11). Parking is strictly
+parity computation, no reads (patch 10). Parking is strictly
edge-driven for anything with a waiter behind it: the fast fsync kicks
the parked rbios covering its ordered extents before waiting for
-writeback (patch 17), sync writes skip parking entirely when their
-stripe belongs to a nodatacow file's private run (patch 22) or when
-their submission batch was already merged at unplug (patch 24), and a
-deadline (100ms, 3ms for sync writes) backstops the paths with no
-waiter at all. A flushed partial write whose uncovered sectors all lie
-at or past its run's allocation frontier is zero-padded into a single
-full-stripe write -- no read phase, one parity pass; the zeros are
-physically written, so scrub stays consistent (patch 18).
+writeback (patch 16), sync writes skip parking entirely when their
+stripe belongs to a nodatacow file's private run (patch 21) or when
+their submission batch was already merged at unplug (patch 23), a
+parked write completes as soon as allocation covers its stripe (patch
+32), and a deadline (100ms, 3ms for sync writes) backstops the paths
+with no waiter at all. A flushed partial write whose uncovered sectors
+all lie at or past its run's allocation frontier is zero-padded into a
+single full-stripe write -- no read phase, one parity pass; the zeros
+are physically written, so scrub stays consistent (patch 17). Padding
+decides from a per-sector liveness map (patch 33), so it never
+zero-fills a sector an allocation already owns; the map is the one
+place where an under-report would be silent data loss rather than a
+forced RMW, and it is what the balance-reclaim regression test below
+guards.
Invariants (stated so the design can be checked against them):
still accepts joins, deliberately: the drain waits on data IO from tasks
that may themselves need a join to make progress (a chunk allocation in
the middle of a delalloc flush, for example), so retiring after joins
-stop would deadlock. It is a pure data-IO wait, never an ordered-extent
-wait -- the latter needs a join too.
+stop would deadlock. It is a data-IO wait -- parked partial writes in
+the closed runs are kicked first, and the wait covers their completion
+-- and never an ordered-extent wait, which needs a join too.
A join that arrives between the retirement and the join barrier is safe
by sequence numbering: the retirement atomically bumps a sequence and
for it -- exactly the path that keeps writeback and reclaim flowing
while the commit drains.
-Log commits get the same treatment at finer grain (patches 14-16).
+Log commits get the same treatment at finer grain (patches 13-15).
Before the log super is written, the log commit closes the open run
covering every extent it logs and drains that stripe's data IO -- the
log-window analogue of I2. A closed run is never extended, so once
extents. So that one fsync-heavy inode does not keep closing runs it
shares with other writers -- fragmenting their streams and trapping the
shared tails -- a log-active inode (sticky, set at its first fsync)
-allocates from its own private runs (patch 15), and its log commits
+allocates from its own private runs (patch 14), and its log commits
settle only its own stripes. The stripe tail trapped per log commit is
-then reclaimed by carry-forward (patch 16): at each log commit the
+then reclaimed by carry-forward (patch 15): at each log commit the
settling walk records which of the inode's logged extents sit in the
closed run's final partial stripe, and the inode's next fsync
re-dirties those ranges (the defrag pattern) so its delalloc flush COWs
frees whole. The private-run scoping makes this tractable: nobody else
ever writes a log-active inode's stripes, and reflinked ranges just
stay put (foreign referents keep the old extent alive in its sealed
-stripe). Steady-state trapped space per fsync-active inode drops from
+stripe). Crash safety does not regress: a carried extent's old copy is
+dropped through the normal paths, and a logged-but-uncommitted one has
+its ADD and DROP delayed refs cancelled in cleanup_ref_head(), which
+pins must_insert_reserved heads, so the old blocks cannot be
+reallocated before the log that references them is superseded. The
+re-dirty reserves with NO_FLUSH: it runs with the folios and the extent
+range locked, and a flushing reservation there is the deadlock shape
+described under Prerequisites; carrying is best effort and the carried
+data is already durable in its old stripe, so on ENOSPC the carry is
+skipped. Steady-state trapped space per fsync-active inode drops from
one stripe tail per fsync to at most one partial stripe.
In-place writes (nodatacow, and writes into preallocated extents) would
relocation-class runs that never share a stripe with other data (I3),
and relocation depends on in-place writes.
-Patches 19-21 make the forced COW a policy rather than a hard rule.
+Patches 18-20 make the forced COW a policy rather than a hard rule.
Nodatacow files' extents and preallocated extents are placed in private
-per-inode runs of their own class (patch 19), so they never share a
+per-inode runs of their own class (patch 18), so they never share a
stripe with any other file's data; with the allow-rmw policy enabled,
an extent whose full stripes are provably owned by the writing inode
alone (its own run, or the committed extent tree shows sole ownership)
-is written in place (patch 20). Such a write can tear, on a degraded
+is written in place (patch 19). Such a write can tear, on a degraded
crash, only the writing file's own data -- which is the nodatacow
contract, and no worse than stock raid5, where the same write endangers
arbitrary neighbors. Anything that fails the ownership test -- extents
generation. The private runs persist across transaction commits
(nothing in them needs invariant I2) and their partial-stripe tails are
re-adopted after a remount, so a slowly appended nocow file packs
-sequentially instead of burning a stripe per commit (patch 21).
+sequentially instead of burning a stripe per commit (patch 20). (The
+pending patches narrow this: a private run is only worth keeping while
+the policy actually lets the file be written in place -- see Pending.)
-Patch 39 extends the same rule to raid56 metadata. When the metadata
+Patch 38 extends the same rule to raid56 metadata. When the metadata
profile is raid56, tree blocks allocate from metadata-class stripe runs
under the identical claim rule -- only fully-free stripes, filled
forward, never revisited -- so no tree block write ever lands in a
and close only after every tree block the transaction allocated is on
disk -- closing earlier would strand blocks the commit has not
allocated yet, draining earlier would wait on writes it has not
-issued. Coverage is driven purely by the block group profile:
-raid1c3/c4 metadata is simply untouched, and system chunks remain
-uncovered (patch 28 warns at mount; patches 29-30 report any
-uncovered read-modify-write and say whether it rewrote parity over
-committed tree blocks -- raid5 system chunks measurably take real
-write holes, and extending the rule to them is future work).
+issued. Coverage is driven purely by the block group profile: there
+is no separate option or property, raid1c3/c4 metadata is simply
+untouched, and system chunks remain uncovered (patch 27 warns at
+mount; patches 28-29 report any uncovered read-modify-write and say
+whether it rewrote parity over committed tree blocks -- raid5 system
+chunks measurably take real write holes, and extending the rule to
+them is future work).
Driving that retirement to saturation -- a workload of
multi-hundred-item fsyncs keeps log tree IO in flight at every
commit, where ordinary load only occasionally does -- exposed three
-defects in it, all fixed in patch 39. First,
+defects in it, all fixed in patch 38. First,
the retirement wait was only ever woken when a run was freed, and
metadata runs are not freed on drain (they are kept for padding and
merging until the retirement collects them) -- so a run whose last
blocks' writeback then rewrites parity over tree blocks the earlier
transaction has already committed, which for metadata is exactly the
write hole this series exists to close. The uncovered-RMW reporter
-of patches 29-30 showed it directly under the saturating load: parity
+of patches 28-29 showed it directly under the saturating load: parity
rewrites over blocks exactly one generation old, on every commit.
(That adjacent-generation signature is the fingerprint of live
sharing; the reporter peeks only a stale header's address and fsid,
The policy is controlled by a "stripe_alloc" filesystem property -- a
btrfs. namespace xattr on the top-level subvolume's root directory,
following the compression property precedent -- rather than by a mount
-option (patch 13). Set once (setfattr -n btrfs.stripe_alloc -v 1), it
+option (patch 12). Set once (setfattr -n btrfs.stripe_alloc -v 1), it
is persistent: the kernel applies it when the root directory inode
loads its properties during mount, before any user IO, so protection
cannot lapse through an fstab edit or a rescue mount -- exactly the
xattr disables the policy and open runs drain at the next commit.
Stray copies on received or cloned subvolumes are ignored; only the
top-level root carries the policy. A mount option of the same name
-remains as a non-persistent override and could be dropped.
+remains as a non-persistent override and could be dropped. The
+property path refuses every unsupported configuration itself (no free
+space tree, the remap-tree feature, zoned, mixed block groups), so a
+property set on such a filesystem is ignored with a message rather
+than enabling a policy the mount path would have rejected.
+
+Setting the property on a live filesystem is safe with respect to the
+write-hole checker (patch 39): the legacy allocator may still have
+writes in flight when the option flips, and those extents sit in partly
+used stripes no run covers, so the first of them to reach the raid56
+write path would trip the invariant check although nothing violated
+the policy -- the data was placed before the policy existed. The
+checker therefore stays disarmed from the flip until a worker has
+started delalloc on every root and waited for every ordered extent
+that could predate it; a mount-time enable has nothing in flight and
+skips the drain.
A second property, "stripe_alloc_allow_rmw", holds a word list naming
the cases where stripe_alloc may permit the legacy unsafe RMW of live
-stripes (patch 20). It is "allow_rmw", not "allow_overwrite": the
+stripes (patch 19). It is "allow_rmw", not "allow_overwrite": the
fsync case overwrites nothing, but all three words permit
read-modify-write of stripes a degraded crash can then tear.
"nodatacow" and "prealloc" enable the confined write-in-place above;
-"fsync" waives the close-at-log-commit guarantee of patches 14-16
+"fsync" waives the close-at-log-commit guarantee of patches 13-15
(none of its costs, and the fsync-window exposure of the allocator-only
patches returns). A mount option of the same name is the
non-persistent override (colon-separated, since mount splits options at
testing period. If no use case surfaces where stripe-exclusive
allocation is worse than the write hole it prevents, it can eventually
become the only allocation mode and the property a no-op. The
-log-commit protection (patches 14-16) currently shares the stripe_alloc
+log-commit protection (patches 13-15) currently shares the stripe_alloc
gate; whether it should be gated separately for its own long-tail
period is an open question for this RFC. Later stages with different
risk profiles (e.g. a global force-datacow policy) are planned as
- zoned filesystems (own allocator, no hole);
- the remap-tree feature (its relocation writes bypass ordered-extent
accounting -- integration is future work);
- - space_cache=v1 (writes cache data in place during commit): v2 required;
+ - filesystems without the free space tree: the claim rule reads free
+ space from the tree, so nospace_cache mounts are refused (the v1
+ space cache no longer exists as of v7.3; on kernels that still
+ have it, v1 writes its cache data in place during commit and is
+ refused as well);
- system chunks: the metadata coverage keys on the METADATA block
group flag, so SYSTEM chunks keep the hole (real: measured parity
rewrites over committed chunk-tree blocks on raid5 system chunks;
- patch 28 warns at mount) -- extending the rule to them is future
+ patch 27 warns at mount) -- extending the rule to them is future
work. raid1c3/raid1c4 metadata remains a fine choice and is
- simply untouched by patch 39;
- - mixed data+metadata block groups: refused, permanently.
+ simply untouched by patch 38;
+ - mixed data+metadata block groups: refused, permanently, on both
+ the mount and the property path.
fill_data_csums() decides by block group whether to verify csums,
so in a mixed group it cannot tell a data stripe from a metadata
stripe; and data and metadata sharing one space_info lets a data
A crash with all devices still present recovers the fsync window intact
by normal log replay. This bound -- committed data untouchable,
fsync-window loss detectable, degraded reconstruction never trusted --
-is the improvement.
+is the improvement. For raid6 the same statements hold with two
+missing devices, and the raid6 coverage below exercises exactly that.
The end state is three operating models:
data (the write hole), and a degraded array cannot safely accept
RMW writes at all.
- stripe_alloc, default policy: no RMW, ever. Data from committed
- transactions and completed fsyncs survives a crash plus a missing
- device, and the array keeps taking new writes while degraded
- without endangering any of it, because no write ever lands in a
- stripe holding earlier data. nodatacow and prealloc writes are
- forced to COW. (Stated for raid56 data. With raid56 metadata,
- patch 39 applies the same rule to tree blocks and the statement
- extends to the trees themselves; system chunks are the remaining
- uncovered case, warned at mount.)
+ stripe_alloc, default policy: no RMW of any stripe holding earlier
+ data, ever. Data from committed transactions and completed fsyncs
+ survives a crash plus a missing device, and the array keeps taking
+ new writes while degraded without endangering any of it, because no
+ write ever lands in a stripe holding earlier data. nodatacow and
+ prealloc writes are forced to COW. (Stated for raid56 data. With
+ raid56 metadata, patch 38 applies the same rule to tree blocks and
+ the statement extends to the trees themselves; system chunks are
+ the remaining uncovered case, warned at mount.)
stripe_alloc + allow_rmw: the named cases RMW as before and their
write-hole exposure returns, confined to what each case's machinery
just-fsynced data, detectably. Data written by every other path
keeps the default-model guarantee.
+"No RMW of a stripe holding earlier data" is the precise claim; "no
+RMW" is not. A partial write to the frontier stripe of an open run
+whose remaining sectors are all allocated but not yet written declines
+padding (it must not zero a sector an allocation owns) and goes down
+as a read-modify-write of its own run's stripe. That RMW reads only
+sectors the same run allocated in the same transaction, so it cannot
+tear committed data and cannot read another file's blocks, but it is a
+real read phase; the "why padding declined" counters of patches 30-31
+attribute every such case, and on an 8-device raid5 small-file fill
+95% of the remaining RMW reads vanish when file sizes are stripe
+multiples. The pending patches do not change this.
+
Today a sub-stripe write in one transaction shares a stripe with data
committed by an earlier transaction, so a degraded crash tears that
earlier, already-committed data. In the common case the allocator fills
Testing
=======
-All measurements and harness runs in this letter come from one rig: a
-raid5 of four 700MiB devices (64KiB stripe elements, 192KiB of data per
-full stripe), driven with deterministic workloads so patched and
-unpatched runs write byte-identical data.
+Everything below was run on two 8-core test VMs with root ssh and a
+serial console, on loop devices or LVM volumes, with deterministic
+workloads so patched and unpatched runs write byte-identical data.
+The harness is a git repository of its own (raid56-wh-harness) with a
+test catalog listing every campaign, what it proved and where its
+results live, and a revalidation driver that re-runs the
+regression-relevant subset after every rebase; the regression plan is
+described at the end of this section. Where a paragraph names a rig it
+is the rig that paragraph's numbers come from; the original one is a
+raid5 of four 700MiB devices (64KiB stripe elements, 192KiB of data
+per full stripe).
A dm-log-writes crash harness records every raid5 member device, runs a
phased small-file workload, then replays each member's log to a cut point
that down is what produced the split-bio error-propagation fix in the
prerequisites; with it applied the same probe returns EIO for those
files instead, and no configuration in the sweep returns silently wrong
-content anywhere.
+content anywhere. (One caveat on the sweep's raid6 rows, discovered
+later: they ran over raid1 metadata, and a two-device drop can take
+both copies of a metadata chunk, so some raid6 cases died as
+"unmountable" instead of testing data recovery. The raid6 coverage
+described below uses raid1c3 metadata for that reason.)
Repeated under a flush-consistent crash model -- every member cut at
the same barrier, then an independent unflushed suffix each, which is
already destroyed committed data: the RMW's verification now refuses,
loudly, to reconstruct through the genuinely torn stripe rather than
bake garbage over it. The full-protection configuration passed the
-same test on the unfixed kernel, 56 of 56: it never RMWs, so it never
-enters the failing path.
+same test on the unfixed kernel, 56 of 56: it never RMWs a shared
+stripe, so it never enters the failing path.
A separate fsync-window mode makes the torn phase's data durable by
fsync only (log tree, no transaction commit) before the cut. On the
-allocator-only patches (1-13) it showed the expected residual exposure:
+allocator-only patches (1-12) it showed the expected residual exposure:
committed data intact in every case, damage confined to just-fsynced
-files and csum-flagged. With the log-commit settling (patches 14-16)
+files and csum-flagged. With the log-commit settling (patches 13-15)
that residual loss goes to zero under the safe recovery: every mounted
case verified all committed files intact with the fsync-window loss
reduced to nothing beyond detectable absence -- zero silent
instead of six), and the trapped-space counter decomposed exactly into
the expected tails in every case.
+raid6 data with raid1c3 metadata -- the pairing this series recommends
+for production -- has its own coverage driver, run on both hosts after
+every candidate: the replay matrix on both arms (stock must be RED,
+stripe_alloc GREEN) with the two-device drop axis and the degraded
+read-write continuation, the strict fsync-window matrix under the
+flush-consistent model, within-tolerance whole-device corruption
+(detection must be visible in the csum counters, recovery must be
+byte-exact, scrub must repair, a cache-cold re-read must be clean,
+plus one-corrupt-plus-one-missing combinations), and two
+beyond-tolerance A/B stages -- two devices wiped, and stripes
+corrupted beyond parity's reach -- where stock drops writes and
+stripe_alloc must not. On the current candidate all seven stages pass
+on both hosts with the kernel untainted: 72 matrix rows per arm with
+27/27 controls clean, the write hole reproduced on the stock arm and
+absent on the stripe_alloc arm, 43/43 degraded continuations intact
+(stock: 12 incomplete), the strict fsync-window criterion met in all
+24 mounted cases, recovery 30/30, and on the beyond-tolerance stages
+570 (stripe corruption) and 10 (two devices wiped) writes dropped on
+the stock arm against 0 on stripe_alloc, from identical fixtures.
+raid1c3 is load-bearing here for the reason given above: raid1
+metadata tolerates one lost device and the raid6 drop axis removes
+two.
+
The second failure mode was reproduced directly (no crash): 8 devices, six
small and two several-GB-larger so metadata lands only on the large ones;
fill to ENOSPC with 4K..512K files then delete half, leaving only sub-stripe
stock kernel: 20 new extents silently dropped (RMW recover-fail).
with this series: 0 extents dropped.
+A harsher variant of the same test (the same wipe, then 180 s of
+fsstress plus a content-verifying churn writer on the two corrupted
+columns) does not reach zero: 360 dropped extents on stock against
+32-37 with the series, reproduced on both hosts on the current base.
+That is a ~91% reduction, not the elimination the fixture above shows,
+and the residual is not yet attributed; it is tracked as an open item
+(the dropped writes read back as zero-filled holes for a writer that
+did not fsync, and the "dropping unwritten extent" message it counts
+fires for any ordered extent finishing with an error, so the count is
+an upper bound on RMW-caused drops).
+
Additionally: a multi-hour fsstress + balance + scrub soak (25 rounds, ~1.25M
ops) with the in-kernel write-hole invariant checker (patch 6,
CONFIG_BTRFS_DEBUG) armed -- zero violations, all scrubs clean. The
fill-to-ENOSPC/delete storms, balance, and fsstress -- the pressure
pattern that exposed a run accounting race (a stripe fully re-freed by
discarded reservations could be re-claimed while its old run was still
-draining); the fix, run-range exclusivity at claim time (patch 12), ran
+draining); the fix, run-range exclusivity at claim time (patch 11), ran
the same load cleanly for ten hours where the race fired in minutes. A
final twelve-round fsstress + balance + scrub soak and an xfstests
-g quick pass complete the gate; every xfstests failure was attributed
stripe claim rule, so commits and mid-transaction free-space returns
could trap the space out from under an admitted reservation and
delalloc writeback dropped the pages (measured at its worst: a fill
-"wrote" 12.5 GiB into a 5 GiB filesystem). Patches 25-26 close this
+"wrote" 12.5 GiB into a 5 GiB filesystem). Patches 24-25 close this
with the same shape metadata reservations use -- pessimistic at
admission (claimed remainders and one full stripe width per
outstanding delalloc extent are counted against reservations, probed
fill edge is exact: zero dropped writebacks, zero allocation failures,
capacity unchanged, and fullness surfaces only as ENOSPC at write(2).
A raid6 run later found an ordering race in that accounting, fixed in
-patch 25: the per-block-group remainder deltas were applied to the
+patch 24: the per-block-group remainder deltas were applied to the
space_info aggregate outside the lock that computed them, so a
negative delta could overtake the positive one it trailed and
underflow the aggregate -- a spurious warning, and a clamp that then
inflated the total. The deltas now apply in the order they were
-computed.
+computed. Patch 36 then gates admission itself on the counted
+whole-stripe supply, so write(2) returns ENOSPC where statfs reaches
+zero instead of writeback discovering the shortfall. The fill-edge
+campaign that followed -- fsstress with thousands of preallocating
+writers, background reclaim, relocation and scrub all at the edge at
+once, under a dead or silently corrupting device -- found the cases
+the gate still missed: they are the pending patches, and with them a
+healthy-filesystem fill to ENOSPC loses zero acknowledged writes
+(checksum-manifest verified) and reports zero writeback drops, the
+only allocator ENOSPC left at the edge being relocation refusing
+cleanly.
Balance reclaims trapped space completely (stripe_unusable collapses
to ~0 and a refill reuses the space), with the usual balance caveat
reserving headroom before filling -- shrink each device ~1G at mkfs
time, grow back to max before balancing, the same technique admins
already use against metadata ENOSPC -- lets balance complete on a
-fully aged filesystem.
+fully aged filesystem. The liveness map of patch 33 has a regression
+test of its own here: a per-file-fsync fill followed by balance once
+found that the grown-frontier path did not mark its cross-stripe spill
+live, so padding zeroed it and computed parity over the zeros --
+committed data silently destroyed, in raid5 and raid6 alike, visible
+only when balance read it back. The fix is one line in patch 9; the
+test (balance-reclaim, both profiles, both allocators) is what would
+catch its return.
The metadata coverage was validated with the same crash harness on a
raid5 data + raid5 metadata rig: the full replay matrix -- 24
(zero committed-data damage, every continuation intact, zero silent
corruption anywhere). The matrix's recovery mounts (log replay plus
orphan cleanup in every case) are also what exposed the two metadata
-completion-report paths now handled in patch 39: each run drives both
+completion-report paths now handled in patch 38: each run drives both
discard paths tens of times, and a single missed report reproduced as
a leaked block group reference at unmount within minutes -- on both
hosts, deterministically -- until both were reported.
with the fixes in place: all controls clean, zero committed-data
damage, and 46 of 46 degraded read-write continuations intact.
-Six pre-existing, unrelated bugs surfaced along the way and are fixed
-by the standalone patches listed under Prerequisites: the soak exposed
-the raid56 rbio use-after-free, the fsync-window sweep exposed the
-split bio error propagation gap, the degraded-write test exposed the
-recovered-sector csum verification bug, the rolling-failure test
-exposed the stale rbio-cache parity window, extending the harness to
-metadata exposed that reconstructed tree blocks were returned to
-readers unverified, and the metadata crash matrix exposed the repair
-write-back on read-only mounts. None is caused by this series and
-none is part of it.
+Regression plan and acceptance suite
+------------------------------------
+
+Everything above is a campaign that answered a question once. What
+runs after every rebase and on every candidate is a fixed plan with
+declared expected outcomes, so a green run means something and a
+surprise stops the loop:
+
+ sanity -- environment and tooling preflight
+ stage0 -- the dm-log-writes replay matrix, both crash models
+ fsync -- the fsync-window and torn-log suite
+ directed -- every single-purpose reproducer for a bug this series
+ had (balance-reclaim liveness, the fill edge, scrub on a
+ fragmented array, the block-group gate, nocow private
+ runs, the runtime enable, the legacy-layout warnings);
+ a script exits 77 when the kernel lacks the feature it
+ tests, and an undeclared kernel WARN during any script
+ fails the stage
+ octopus -- the acceptance suite below, 4 h budget
+ soak -- the deadlock rig below, 12 h
+
+The acceptance suite runs thirteen tests under a fault that is present
+for the whole run, on a 6-device raid5 with one spare: read-after-write
+with a cache drop, no IO errors visible to applications, scrub, balance,
+fsstress, ENOSPC without data loss (the live filesystem is shrunk with
+resize and filled to the edge), compression, continuous kernel health
+(dmesg and D-state watchdog), nodatacow, fsync durability, fsync-heavy
+log-tree stress with renames, device replace, and an offline check of
+the healed array. Each host presents a different fault model: one
+member returning EIO on every access (a dead disk, through dm-error),
+or one member silently dropping every write and returning rotating
+random garbage on every read (dm-flakey over a random backing whose
+offset changes every second, so no block ever reads the same twice and
+reconstruction is exercised on every read). Three passes per loop:
+stripe_alloc with the fault injected online, stripe_alloc mounted
+degraded from the start, and the stock allocator on the same kernel.
+The declared expectation is that both stripe_alloc passes pass all
+thirteen (a single fault is inside the redundancy, so the only
+acceptable application-visible error is ENOSPC at the fill edge), the
+stock pass may fail (marked as expected), and nodatacow is skipped
+under silent corruption where its loss is undetectable by design. A
+positive control -- the same suite on a pristine stable kernel with
+none of the prerequisites -- fails four of the tests (ENOSPC data loss
+and post-replace corruption under the dead disk; write errors reaching
+the application and fsynced data not durable under silent corruption),
+which is the evidence that the green results are not vacuous.
+
+On the current candidate: under the dead-disk model, three loops of
+all three passes in a 12-hour budget -- every stripe_alloc pass
+13/13, the stock pass 13/13 once and 12/13 twice, the one failure a
+debug-config artifact (a 20 ms expedited-RCU stall timeout tripping
+the health watchdog at unmount) -- no unexpected failure, kernel
+untainted. Under the silent-corruption model, sanity, stage0, fsync,
+directed and the full acceptance suite all pass (12/12 per pass plus
+the declared nodatacow skip). The raid6 coverage above is 7/7 on
+both hosts. The same suite on the 7.3-rc2 base gives 13/13 and 13/13
+for the stripe_alloc passes; its stock pass scored 7 passes and 6
+expected failures and ended in a relocation ENOSPC abort after the
+last test, the small-geometry metadata exhaustion the suite now
+provisions against. An earlier 24-hour
+endurance run of the suite (four loops per host, before the fill-edge
+campaign) was clean under both models.
+
+The soak is the deadlock rig: eight LVs with one removed (degraded),
+optionally dm-crypt underneath (the production shape), mounted
+degraded,compress=zstd:3,flushoncommit,stripe_alloc,discard=async, with
+churn writers, a deduper, a snapshot loop, a balance loop and mass
+deletes, a heartbeat watching commit progress and a capture (SysRq,
+stacks, ftrace ring, drgn probes) tripping on a genuine stall. It is
+the rig that reproduced the production retire/finish-ordered deadlock
+described under The approach, the carry-forward folio deadlock that
+made the carry reserve NO_FLUSH, and all three deadlock-fixes
+prerequisites, and it is where the 60-230 s self-resolving commit
+stalls of a large delayed-ref backlog were shown (NMI-sampled) to be a
+generic delayed-ref grind and not this series.
+
+Fifteen pre-existing, unrelated bugs surfaced along the way. Thirteen
+are fixed by the standalone patches listed under Prerequisites: the
+soak exposed the raid56 rbio use-after-free, the fsync-window sweep
+exposed the split bio error propagation gap, the degraded-write test
+exposed the recovered-sector csum verification bug, the rolling-failure
+test exposed the stale rbio-cache parity window, extending the harness
+to metadata exposed that reconstructed tree blocks were returned to
+readers unverified, the acceptance suite's stock pass under silent
+corruption exposed the two rbio-cache use-after-free races, the fill
+edge exposed the abandoned-delalloc reservation leak and the
+nodatacow-versus-read-only race, the deadlock rig exposed the three
+lock-ordering deadlocks, and the raid6 coverage exposed the lib/raid6
+single-data-disk bug. Two more -- the read-only repair write-back and
+the defrag reservation deadlock -- were fixed upstream independently in
+the meantime. None is caused by this series and none is part of it.
Performance
===========
fsync stream is zero (every flush is a padded full-stripe write; on
rotational media the eliminated RMW read round trip is the dominant
cost). These fsync/sync latencies depend on parking being edge-driven for
-every waiter (patches 17, 23, 25): no measured waiter path pays a
+every waiter (patches 16, 21, 23 and 32): no measured waiter path pays a
parking deadline. An async writeback with no waiter behind it can pay
one; at a small-file fill to ENOSPC that becomes a throughput cost (see
Known costs).
-A /sys/fs/btrfs/<uuid>/stripe_park_stats file (patch 23) reports how
+The allocator's own cost showed up once, on a production array rather
+than a test rig: an aged 85 TiB raid5 with legacy-packed block groups,
+where every 4K allocation rescanned a whole block group's free space
+under the free-space lock looking for a run that did not exist -- a
+delalloc worker at 100% CPU and commits at KiB/s. Patch 37 caches the
+failed scan per block group (invalidated by any free-space return), and
+on a rig shaped like that array (60 GiB raid5 fragmented by
+delete-every-other) the average claim went from 1447 us to 0.8 us and a
+fsync write storm from 48 to 218 files/s, now fsync-bound.
+
+On filesystems that are not raid56 the series is dormant: its hooks on
+the buffered write path are early-outs measured at 0.2% of
+btrfs_buffered_write()'s CPU time (ftrace function profile, single
+profile, no stripe_alloc), and a quiet-host raw-LV A/B of the two most
+recent revisions shows no bandwidth change (sequential +2.2%,
+sequential with fsync -0.4%, direct IO -5.9% with fully overlapping
+per-run ranges, 4K random write 0.0%, ten runs each).
+
+A /sys/fs/btrfs/<uuid>/stripe_park_stats file (patch 22) reports how
every park ended (filled to a full stripe, kicked by a waiter or a
-settle/retire flush, or expired at its deadline) and how many flushed
-writes were padded to full stripes versus needing the RMW read phase,
-so the deadline constants can be judged, and retuned if ever needed,
-from measurements rather than taste.
+settle/retire flush, completed on allocation coverage, or expired at
+its deadline), how many flushed writes were padded to full stripes
+versus needing the RMW read phase, why each declined padding, and the
+metadata RMW audit's live/dead/free classification, so the deadline
+constants can be judged, and retuned if ever needed, from measurements
+rather than taste.
Known costs
===========
capacity (zero trapped) by reusing every sub-stripe hole -- each such
reuse being a cross-transaction partial-stripe write. Trapped space is
real free space and balance reclaims it; garbage collection on aged
-filesystems is expected and by design.
+filesystems is expected and by design. The pending patches hand a
+block group whose trapped bytes cross the existing bg_reclaim_threshold
+to the existing reclaim worker, the way zoned filesystems recycle their
+unusable space, so an operator no longer has to run balance by hand.
-statfs reports the trapped bytes honestly (patch 8): they are tracked
+statfs reports the trapped bytes honestly (patch 7): they are tracked
per block group as stripe_unusable (summed into a space_info
bytes_stripe_unusable, recomputed at commit for groups whose free space
changed) and subtracted from f_bavail, so df no longer promises space the
allocator will refuse. The counter is exposed at
/sys/fs/btrfs/<uuid>/allocation/data/bytes_stripe_unusable. It is derived
from the existing free space (no on-disk format change; the free space
-tree and cache are left untouched). Patches 25-26 make the reservation
+tree and cache are left untouched). Patches 24-25 make the reservation
layer respect it as well: trapped fragments, claimed run remainders and
a pessimistic per-extent margin are all counted against data admissions,
so a writer that cannot be covered waits in the ticketed data flush and
receives ENOSPC at write(2) -- a delalloc writeback is never left to
-discover the shortfall after the pages are dirty. Patch 37 gates
+discover the shortfall after the pages are dirty. Patch 36 gates
admission on a directly counted claimable whole-stripe supply, so
-write(2) reaches ENOSPC exactly as statfs reaches zero, and patch 38
-caches a failed claim scan per block group, which cut the claim path's
-cost about 1800x on a fragmented aged array. Balance remains the
-reclaim mechanism for trapped space.
+write(2) reaches ENOSPC exactly as statfs reaches zero, and patch 37
+caches a failed claim scan per block group. Balance -- or, with the
+pending patches, the reclaim worker -- remains the reclaim mechanism
+for trapped space.
+
+The admission gate is pessimistic by design (the direction chosen for
+this RFC: refuse space early rather than admit a write the allocator
+cannot place). Measured on a fragmented raid5 after a delete-half and
+balance, the refill accepts about 24% less than the pre-gate revision
+did -- but the pre-gate revision's extra came with dropped writebacks
+in the same run (26 in one boot against 0) and a statfs that promised
+1.9 GiB more than the allocator could place, so the 24% is the honest
+number. A big sequential file to ENOSPC is within 1.5% either way and
+a small-file fill to ENOSPC is equal.
Async small-file fill to ENOSPC pays the parking deadline. When the
workload is many sub-stripe files whose neighbours land in the next
stuck cap, while a large sequential fill of the same filesystem writes
12 GiB in 34s with none -- sequential writes are already full stripes and
never park. The cost is throughput only: every such write reaches disk
-correctly at any fill level (no path returns wrong or zero data). Patches 33-35 shrink the cost
+correctly at any fill level (no path returns wrong or zero data). Patches 32-34 shrink the cost
where it can be shrunk: a parked write completes as soon as allocation
covers its stripe rather than when a clock expires, and padding
decides from a per-sector liveness map so it never zero-fills a live
-sector. Patch 36 makes the deadline adaptive: a stripe_park_congestion knob (default off)
+sector. Patch 35 makes the deadline adaptive: a stripe_park_congestion knob (default off)
shortens a park only while the parked-rbio backlog runs ahead of it, so a
small-file flood -- which builds that backlog -- drains at the short
deadline, while a moderate steady stream stays under it, keeps the full
"operator fills the disk" accident is one large writer, which is already
full stripes, parks nothing, and pays neither cost.
- fs/btrfs/... | 39 patches
- RED->GREEN harness, soak, and xfstests logs available on request.
+Pending: reservation, relocation and reclaim hardening
+======================================================
+
+Twenty-six further patches on top of this series are complete and
+carried by every test kernel quoted above from the acceptance-suite
+paragraph on; they are under review and will be folded into the series
+before it is posted. They exist because the fill-edge campaign -- the
+acceptance suite's ENOSPC test under a dead or corrupting device, with
+fsstress preallocating from eight processes while background reclaim,
+relocation and scrub all reach the edge together -- found every case
+the admission gate of patch 36 still missed, and each was fixed as its
+own patch with the counter or dump that proved it. Titles:
+
+ btrfs: stripe_alloc: do not count trapped free space twice when making a block group read-only
+ btrfs: stripe_alloc: keep the admitted bytes placeable when a group goes read-only
+ btrfs: stripe_alloc: never let relocation wait on the whole-stripe gate
+ btrfs: stripe_alloc: give a queued stripe group a chunk to relocate into when the supply is short
+ btrfs: stripe_alloc: charge relocation a margin per reservation, not per extent
+ btrfs: stripe_alloc: keep the relocation group's stripes out of the admission supply
+ btrfs: stripe_alloc: admit relocation against the whole claimable supply, not only its dedicated group
+ btrfs: stripe_alloc: let a data write fall back to the relocation group rather than drop
+ btrfs: stripe_alloc: hand groups full of trapped free space to the reclaim worker
+ btrfs: stripe_alloc: relocate a queued stripe group even while it has reserved or pinned bytes
+ btrfs: stripe_meta: maintain trapped free space for metadata groups like data
+ btrfs: stripe_meta: back the global reserve with usable stripes, and say when a tree block finds none
+ btrfs: stripe_meta: hold back whole stripes so tree blocks always land and trapped groups can be reclaimed
+ btrfs: stripe_meta: do not steal from the global reserve below the relocation reserve
+ btrfs: stripe_meta: let the transaction machinery's own reservations draw on the reserve
+ btrfs: stripe_meta: let a group go read-only for relocation against the reserve
+ btrfs: stripe_alloc: show per-group trapped bytes in the space info dump
+ btrfs: stripe_alloc: tell a dead tree block from a live one in the metadata RMW audit
+ btrfs: stripe_alloc: do not read an inline backref off a keyed-ref extent item
+ btrfs: stripe_alloc: count whole stripes stranded behind live runs as trapped
+ btrfs: stripe_alloc: give nocow extents a private run only while they may be written in place
+ btrfs: stripe_alloc: count nocow run lifecycles and stranded stripes in sysfs
+ btrfs: stripe_alloc: charge the held stripe margin against every admission
+ btrfs: stripe_alloc: admit data reservations by whole stripes
+ btrfs: stripe_alloc: hold a stripe of margin across a preallocation
+ btrfs: stripe_alloc: hold the probe margin until the metadata reservation charges it
+
+What they change, in the order the campaign found it:
+
+ - Scrub and replace on a fragmented array. inc_block_group_ro()
+ asked the space_info to absorb the group's free space in bytes
+ while the trapped part of that free space was already counted as
+ used, so a churned raid56 data group could never go read-only and
+ every device's scrub failed with "failed setting block group ro:
+ -28" at 51% used. The read-only transition now also refuses while
+ admitted bytes would no longer fit in the remaining whole-stripe
+ supply (scrub, balance and reclaim retry), the fail-early direction.
+
+ - Relocation at the fill edge. Relocation's cluster reservations
+ went through the data gate and could wait as a ticket for the very
+ stripes relocation was about to free, with the exclusive operation
+ and the source group held: writers queued behind that ticket and
+ unmount hung on their writeback. Relocation now never waits on the
+ gate, is charged one stripe of collateral per reservation instead
+ of one per extent (relocating a group of 16-48K files had inflated
+ the margin to 150 MB), is admitted against the whole supply while
+ its dedicated group's stripes stay out of everyone else's supply,
+ can be handed a fresh chunk by the reclaim worker when the
+ remaining groups are the trapped ones it was queued to move, and
+ an ordinary write admitted a moment before a group became
+ dedicated may fall back into it rather than be dropped.
+
+ - Automatic reclaim of trapped space, keyed on bg_reclaim_threshold
+ like zoned, with the reclaim worker taking a stripe group even
+ while it has reserved or pinned bytes (a stripe group is queued
+ for trapped space, which waiting does not free).
+
+ - raid56 metadata at the fill edge. Trapped bytes in metadata groups
+ are now maintained like data (they were frozen at their first
+ value), and the global reserve is backed by whole stripes: a chunk
+ is allocated while unallocated space still exists, two
+ transactions' worth of whole-stripe claims are held back so a tree
+ block admitted in bytes always finds a stripe, the global-reserve
+ steal refuses below the relocation reserve so an unlink returns
+ ENOSPC instead of the commit aborting, the transaction machinery's
+ own NO_FLUSH reservations may draw on the reserve, and the
+ read-only transition that starts a relocation may too. Each of
+ those was a transaction abort in the tiny-file metadata fill test
+ before its patch; with all of them the fill stops at a clean
+ ENOSPC and the delete phase runs through.
+
+ - The metadata RMW audit distinguishes a dead block (freed with its
+ stripe, harmless under a torn parity write) from a live one by
+ asking the committed extent tree, which turned six hundred
+ "write hole" reports per fill into zero live ones.
+
+ - A keyed backref on a data extent item (the leaf had no room for an
+ inline one) made the sole-ownership walk read the next item's bytes
+ and WARN.
+
+ - The admission gate's premise "claimable implies placeable" had
+ three holes. Whole stripes freed inside a live run's range cannot
+ be claimed (a second run over the same range would make the
+ lookups ambiguous) but were counted claimable: on a raid56
+ filesystem filled by fsstress, thousands of cached inodes each
+ held a private nodatacow run over stripes that had since freed,
+ the gate admitted ~100k writes per loop against them, and
+ writeback dropped every one. They are now counted as trapped
+ (bytes_stripe_stranded), a nodatacow private run only exists while
+ the allow-rmw policy would actually let the file be written in
+ place, and a stripe_run_stats file counts the lifecycles so a test
+ can prove the paths ran. Then the held stripe margin was charged
+ against total_bytes but not against the supply the gate checks;
+ then preallocation was admitted for its bytes while it claims whole
+ stripes, holds no margin, and eight of them at once at the edge
+ each took a stripe the gate had counted for someone else; then the
+ probe margin was released between the data check and the metadata
+ reservation that re-charges it, and eight writers were admitted
+ into each other's released margins. Fill-edge writeback failures
+ per loop fell from ~100k to tens with the stranded-stripe and
+ nocow-run patches, to 2 with the held probe margin, and to 0 on a
+ healthy filesystem.
+
+Validation of the folded result is the acceptance-suite, raid6 and
+soak results quoted under Testing (every kernel there carries these
+patches). The directed evidence: on a healthy filesystem the
+fill-edge reproducer reports zero dropped writebacks, and four
+fill-to-ENOSPC runs of the manifest-verified data-loss writer (~5000
+files each, no fsync) lose zero acknowledged writes; the only allocator
+ENOSPC left at the edge is relocation's, which fails a balance or
+shrink cleanly rather than losing data. The scrub-on-a-fragmented-
+array reproducer passes at 88% used where it failed at 83%; the
+nodatacow-strand test shows the default policy stranding nothing and
+the waived policy refusing at write() rather than dropping, and
+recovering the space at eviction. Against the same base without these
+patches, on raid5 with stripe_alloc: sequential write +32% on a fresh
+and on a fragmented filesystem, 4K random write +12%, small files on a
+fragmented filesystem +86%, balance 24% faster, bytes_stripe_unusable
+14% lower; the one cost is the capacity pessimism quoted under Known
+costs, accepted for this RFC.
+
+ fs/btrfs/... | 39 patches, 34 files changed, 6338 insertions(+), 49 deletions(-)
+ harness, logs and the test catalog available on request.
projects / linux / commitdiff