Zygo Blaxell [Thu, 1 Oct 2026 07:54:52 +0000 (03:54 -0400)]
DO-NOT-UPSTREAM: btrfs: raid56: debug knob for the rbio stripe cache size
Test fixture, not for submission. A read-modify-write whose stripe is
still in the rbio stripe cache takes the other strips from memory instead
of reading them, so a test that injects faults into reads (dm-stripfault)
cannot reach the RMW read path for recently written stripes.
With CONFIG_BTRFS_DEBUG, /sys/fs/btrfs/<uuid>/debug/rbio_cache_size
replaces the RBIO_CACHE_SIZE limit (default unchanged, 1024); 0 disables
the cache, except for the stripes stripe_alloc pins there until their run
has written all of them. Lowering it empties the cache, and cache_rbio()
passes the limit to rbio_cache_prune(), which prunes as far as needed, so
a lowered limit holds.
It sits in topics/test-knobs, on top of topics/stripe-alloc, whose rbio
pins and cache prune it adapts to.
Zygo Blaxell [Thu, 22 Oct 2026 08:42:22 +0000 (04:42 -0400)]
DO-NOT-UPSTREAM: btrfs: let queued exclusive operations in between block group reclaims
DO-NOT-UPSTREAM: this works around the reclaim worker starving queued
exclusive operations. Whether upstream wants to do anything about that
starvation, and what, is upstream's decision; this stays on our lanes.
The block group reclaim worker runs under the balance exclusive operation.
At the fill edge, with writers active, each relocation takes tens of
seconds (2 to 61 s measured) and the list keeps refilling, so the
worker held the operation for most of the time writers ran: 84-90% of a
600 s churn in reclaim-futile-test.sh, and for an hour of fsstress in the
octopus P1 pass, where a resize and a replace could not start.
btrfs-progs makes this worse than a plain refusal: before it calls the
ioctl it reads sysfs exclusive_operation. Without --enqueue it gives up
there; with --enqueue it waits for the notification that
btrfs_exclop_finish() sends and then calls the ioctl. Either way the
kernel never sees a refusal, and the worker retakes the operation right
after each group, before a waiter can get in: on the lane a queued resize
waited the whole churn (0 of 40 one-shot resizes succeeded, the longest
refusal 601 s).
After each group that was relocated, release the operation
(btrfs_exclop_finish() sends the sysfs notification) and, if more groups
are queued, leave it free for BTRFS_RECLAIM_GAP (12 s) before taking it
back; if someone took it meanwhile, leave the rest of the list for a
later run. A group that was skipped or dropped (the backoff, or the
checks before relocation) held the operation only briefly and gets no
gap. Neither does a filesystem that is closing or has failed: the
worker only drops the rest of the list there, and close_ctree() waits
for it in cancel_work_sync(). 12 s comes from btrfs-progs: after the
notification, check_running_fs_exclop() reopens the file and sleeps in a
second select() for (remaining % 10) + 1 s "to randomize the order"
before it re-reads the operation, so a waiter can take up to 10 s, plus
wakeup latency under load, to get to the ioctl; with a 5 s gap about
half the queued waiters woke after the worker had retaken the operation
and waited another group. A progs change that re-reads right after the
notification (and randomizes only when it loses a race) would allow a
much shorter gap. With nobody waiting the gap costs only reclaim
throughput: reclaim resumes after it.
A caller that does reach btrfs_exclop_start() while the worker holds
the operation (an ioctl user that does not pre-check sysfs) opens a
BTRFS_RECLAIM_YIELD_WINDOW (60 s, longer than such a caller's retry
interval and short next to the five-minute reclaim hysteresis) in which
the worker neither starts nor takes another group. Both are backoffs
only: no drain or accounting wait is bounded by them.
A group in progress is not interrupted, so a queued operation waits for
the rest of the current group plus at most one gap. Measured (before
the gap was limited to relocated groups) on a stripe_alloc filesystem,
whose trapped space keeps the reclaim list full at the fill edge, with
reclaim-futile-test.sh (6 x 1 GiB raid5 at the fill edge, 128 MiB per
member added, "all" written to stripe_reclaim_now, 600 s of fsstress, a
no-op `btrfs filesystem resize` every 15 s and a `--enqueue` one every
60 s), three traced runs per kernel:
without this patch: the worker held the operation for a whole sweep
(184, 176 and 616 s); the first queued resize waited 184, 176 and about
655 s, i.e. for every queued group; one-shot resizes 27/40, 28/40, 0/40.
with it: every queued resize got in at the first hand-off after it
arrived (one sat through three short groups, 23 s), 0.3-9.4 s after the
worker's finish; longest queued waits 82, 11 and 23 s (the 82 s one sat
behind a single 127 s group); one-shot resizes 12/40, 31/40, 27/40.
Reclaim kept its pace: 10, 10 and 14 groups per 10 minutes without the
patch, 13, 11 and 10 with it.
Zygo Blaxell [Thu, 1 Oct 2026 16:14:42 +0000 (12:14 -0400)]
btrfs: stripe_alloc: keep a stripe in the rbio cache until its run has written all of it (debug_meta part)
The hunks of "btrfs: stripe_alloc: keep a stripe in the rbio cache until
its run has written all of it" (earlier in this series) that need code
from "btrfs: raid56: say whether a metadata read-modify-write is a write
hole", which comes after it in this series: they are applied here, once
"btrfs: raid56: say whether a metadata read-modify-write is a write
hole" is in, instead of with the rest of "btrfs: stripe_alloc: keep a
stripe in the rbio cache until its run has written all of it".
Zygo Blaxell [Tue, 11 Aug 2026 06:53:38 +0000 (02:53 -0400)]
btrfs: raid56: adaptive park deadline from the parked-rbio backlog (debug_meta part)
The hunks of "btrfs: raid56: adaptive park deadline from the parked-rbio
backlog" (earlier in this series) that need code from "btrfs: raid56:
say whether a metadata read-modify-write is a write hole", which comes
after it in this series: they are applied here, once "btrfs: raid56: say
whether a metadata read-modify-write is a write hole" is in, instead of
with the rest of "btrfs: raid56: adaptive park deadline from the
parked-rbio backlog".
btrfs: stripe_alloc: attribute each data read-modify-write to why padding declined (debug_meta part)
The hunks of "btrfs: stripe_alloc: attribute each data read-modify-write
to why padding declined" (earlier in this series) that need code from
"btrfs: raid56: say whether a metadata read-modify-write is a write
hole", which comes after it in this series: they are applied here, once
"btrfs: raid56: say whether a metadata read-modify-write is a write
hole" is in, instead of with the rest of "btrfs: stripe_alloc: attribute
each data read-modify-write to why padding declined".
It also drops the pad_decline_live and pad_decline_unallocated counters:
these hunks remove their last readers.
Zygo Blaxell [Sat, 8 Aug 2026 07:57:22 +0000 (03:57 -0400)]
btrfs: stripe_alloc: trace why padding declined, at the moment it declined (debug_meta part)
The hunks of "btrfs: stripe_alloc: trace why padding declined, at the
moment it declined" (earlier in this series) that need code from "btrfs:
raid56: say whether a metadata read-modify-write is a write hole", which
comes after it in this series: they are applied here, once "btrfs:
raid56: say whether a metadata read-modify-write is a write hole" is in,
instead of with the rest of "btrfs: stripe_alloc: trace why padding
declined, at the moment it declined".
Zygo Blaxell [Sat, 8 Aug 2026 06:03:40 +0000 (02:03 -0400)]
btrfs: stripe_alloc: tunable park deadlines, and say why padding declined (debug_meta part)
The hunks of "btrfs: stripe_alloc: tunable park deadlines, and say why
padding declined" (earlier in this series) that need code from "btrfs:
raid56: say whether a metadata read-modify-write is a write hole", which
comes after it in this series: they are applied here, once "btrfs:
raid56: say whether a metadata read-modify-write is a write hole" is in,
instead of with the rest of "btrfs: stripe_alloc: tunable park
deadlines, and say why padding declined".
Zygo Blaxell [Wed, 5 Aug 2026 22:15:06 +0000 (18:15 -0400)]
btrfs: raid56: say whether a metadata read-modify-write is a write hole
meta_rmw counts sub-stripe metadata writes, which is a proxy for exposure
rather than a measurement of it. A stripe modified in place is only a write
hole if it holds data some completed transaction is relying on; a stripe
that two writes of the *same* transaction happen to split costs an extra
read but risks nothing, because a tear loses that whole transaction anyway.
The counter cannot tell those apart, so it cannot answer the only question
that matters.
The rbio can. A read-modify-write has already read every column the write
does not cover, so at the point of the report it is holding the stripe's
on-disk contents. Walk the uncovered tree block positions and read their
headers: a block whose bytenr and fsid match belongs there, and its
generation says which transaction put it there. Compare that against the
generation of the blocks this write is carrying, taken from the same rbio,
so a transaction committing concurrently cannot skew the verdict:
meta_rmw_cur the same transaction's own blocks -- a cost
meta_rmw_old an earlier transaction's -- a write hole
meta_rmw_free no tree block there at all
System block groups are classified too, not just metadata ones. A system
chunk carries no METADATA bit, so a flags test excludes it, but it holds
the chunk tree: the same tree block header, the same generation field, and
a worse consequence if a degraded crash tears it, since without the chunk
tree no logical address can be mapped at all. meta_rmw already counted
system chunks; only the classification skipped them, so their read-modify-
writes were visible as a rate and never as a verdict. Whether a raid56
system chunk ever rewrites parity over a committed tree block is therefore
an open question that this answers by measurement rather than by reasoning
about how the chunk tree is laid out.
This trusts nothing the allocator says about itself. The stripe runs, the
liveness map and the drain accounting are all bookkeeping that could be
wrong in the same way twice; the header in the sector is what a degraded
read would actually have to reconstruct.
On a 3-device raid5 filesystem with raid5 metadata, 6000 small files with
periodic syncs and then a third rewritten, plain stripe_alloc reports 249
sub-stripe metadata writes of which 174 rewrite parity over committed tree
blocks. Adding stripe_meta leaves 37 sub-stripe writes and none of them.
An older header under the write is not yet a write hole: most belong to
blocks that died with their stripe, which a torn parity write cannot hurt.
So the committed extent tree, the witness a degraded read after a crash
would consult, is asked whether the block is still referenced, with an
extent item of the header's generation (a sector re-allocated in the
current transaction carries a new item while the old header is still on
disk). A live one is reported as before, with its bytenr and reference
count; a dead one is counted as meta_rmw_ghost without a warning.
The lookup runs in rmw_rbio() under the full-stripe lock, where reading a
tree block of the same stripe would queue a rebuild behind that lock, so
it does not block: commit_root_sem is taken with a trylock and the search
uses path->nowait. A block not in memory is counted as meta_rmw_unknown.
Zygo Blaxell [Fri, 9 Oct 2026 23:13:11 +0000 (19:13 -0400)]
btrfs: stripe_alloc: count and log ordered extents that end with sectors unwritten (debug_meta part)
The hunks of "btrfs: stripe_alloc: count and log ordered extents that
end with sectors unwritten" (earlier in this series) that need code from
"btrfs: stripe_alloc: share LOG data runs per log transaction
(stripe_meta part)", which comes after it in this series: they are
applied here, once "btrfs: stripe_alloc: share LOG data runs per log
transaction (stripe_meta part)" is in, instead of with the rest of
"btrfs: stripe_alloc: count and log ordered extents that end with
sectors unwritten".
They append the unwritten_on_end counters to the stripe_run_stats list
after the log seal counters, which that patch adds to it.
Zygo Blaxell [Tue, 6 Oct 2026 04:11:38 +0000 (00:11 -0400)]
btrfs: stripe_alloc: share LOG data runs per log transaction (stripe_meta part)
The hunks of "btrfs: stripe_alloc: share LOG data runs per log
transaction" (earlier in this series) that need code from "btrfs:
stripe_alloc: give each log commit its own metadata stripe run", which
comes after it in this series: they are applied here, once "btrfs:
stripe_alloc: give each log commit its own metadata stripe run" is in,
instead of with the rest of "btrfs: stripe_alloc: share LOG data runs
per log transaction".
Zygo Blaxell [Mon, 5 Oct 2026 03:15:56 +0000 (23:15 -0400)]
btrfs: stripe_alloc: give each log commit its own metadata stripe run
A metadata stripe run is keyed by transaction id alone, so every fsync log
commit inside one transaction appends at the same frontier and into the same
full stripe. A log commit's tree blocks are durable the instant fsync
returns, so a later writer sharing that stripe -- the next log commit, or the
transaction's ordinary metadata at commit -- has its sub-stripe write
recompute parity over metadata the user was already promised. That is the
raid56 write hole, on data a user was told was safe. Padding cannot cover it
either: the earlier blocks are live in the run's liveness map, so
rmw_try_pad_full() must decline.
Give the log tree its own class. A BTRFS_STRIPE_RUN_META_LOG run is keyed by
the owning subvolume's root id and that subvolume's log_transid, and is
private like the per-inode runs: never in the shared band slots, found by
walking for its key, so nothing else can join it. Each log commit therefore
gets its own whole stripes and the sharing cannot arise, rather than being
detected and worked around after the fact.
The space cost is temporary by construction, which is the point of keying on
log_transid rather than merely separating log blocks from other metadata. The
log tree is freed at transaction commit, so a stripe holding only one log
commit's blocks is wholly free once that happens and the commit-time rescan
credits the whole stripe back. Measured on a 4-device raid5 filesystem with
raid5 metadata and 16K nodes under six fsync writers, trapped space falls from
a 20.2 MiB loaded peak to 1.25 MiB once the load stops and three commits pass,
which is below the 23.7 MiB the plain allocator settles at. Metadata
total_bytes settles at 2.15x the plain allocator's, and since allocated
metadata chunks are not available to data, that multiple is the real cost on a
small or full filesystem. Filling to ENOSPC leaves 384 KiB of df-avail, the
same figure the plain allocator gives, so nothing is over-committed.
Both directions of the sharing go to zero, and so does the disk-side witness:
an audit counting uncovered tree blocks whose stripe holds log-tree headers
reads 0 under load, against 223847 without the change. Padding declines fall
from 36732 uncovered to 1.
Retirement handles both metadata classes through
btrfs_stripe_run_class_is_meta(), since a log run must not be closed and
waited on in the commit-time drain that settles the data classes. Drain is by
bytenr through end_bbio_meta_write() and clean_log_buffer(), so it is
class-agnostic and covers the log tree's private free path. A sysfs knob,
stripe_log_run, turns the class off so the change can be measured against
itself on one boot.
log_transid starts over with every transaction's log tree, so a run also
carries its transaction id, and a run from an earlier transaction is
closed rather than joined even when its log_transid matches: that
transaction's commit retires the run and waits for its blocks, and
would wait out its timeout for a block only the later transaction
writes. Commit-heavy load (relocation of metadata groups during an fsync
storm) hit that within minutes.
Metadata runs drain only when their tree blocks are written, in a commit,
so making a raid56 metadata group read-only (relocation, scrub) commits
the current transaction first when the group has metadata runs, instead
of waiting up to commit_interval for the next one while relocation holds
reclaim_bgs_lock.
The drain waits for metadata runs, at commit and when a group goes
read-only, stay unbounded: a run that never drains has a tree block freed
or allocated without a report, an accounting bug that should show as a
hung commit where it happens rather than as later symptoms. After 60
seconds they warn once and dump the group's runs (every run class) and
keep waiting. Only a filesystem error ends them.
Zygo Blaxell [Sat, 10 Oct 2026 04:19:31 +0000 (00:19 -0400)]
btrfs: stripe_alloc: skip RAID5/6 groups that are mapped as RAID1/RAID1C3 (stripe_meta part 2)
The hunks of "btrfs: stripe_alloc: skip RAID5/6 groups that are mapped
as RAID1/RAID1C3" (earlier in this series) that need code from "btrfs:
stripe_meta: cover raid56 system chunks", which comes after it in this
series: they are applied here, once "btrfs: stripe_meta: cover raid56
system chunks" is in, instead of with the rest of "btrfs: stripe_alloc:
skip RAID5/6 groups that are mapped as RAID1/RAID1C3".
The chunk tree lives in system chunks, which carry no METADATA bit, so the
whole-stripe rule that stripe_meta applies to raid56 tree blocks never
applied to it: chunk tree blocks were placed by the clustered allocator,
sharing stripes with chunk tree blocks committed earlier, and every chunk
tree COW was a read-modify-write of such a stripe. The RMW audit measured
it (meta_rmw_old over system chunks during chunk churn), and the mount
warned about it. A torn parity rewrite there is the worst write hole the
filesystem can take: with a member then lost, the committed chunk tree
block that shared the stripe reconstructs wrong, the checksum rejects it,
and nothing maps -- the degraded mount fails, and there is no second copy.
Include system block groups in btrfs_is_stripe_meta_bg(). Everything
downstream already keys on that predicate: the whole-stripe allocation in
do_allocation_stripe(), the trapped-space accounting, the covered/uncovered
decision in the RMW report and the audit. The four walks that select
groups by space_info flags (the enable-time sweep, the unusable-space scan,
the reclaim tick, the commit-time metadata run retirement) gain SYSTEM
alongside DATA and METADATA, so system groups are armed, scanned, reclaimed
when trapped, and have their runs closed after the commit's tree writes
like metadata groups.
reserve_chunk_space() sizes its request in tree blocks, but a chunk tree
COW that needs a fresh run takes a whole stripe for its first block; the
trapped bytes and the open run's remainder are already counted as used, so
what is left is whole free stripes, and the check now demands one stripe
of headroom beyond the request -- deciding when to allocate the next
system chunk, without reserving the headroom -- or the chunk item for that
very allocation could find no stripe to land in.
The "does not cover raid56 system chunks" warning has nothing left to
warn about and goes.
btrfs: stripe_alloc: count whole stripes stranded behind live runs as trapped (stripe_meta part)
The hunks of "btrfs: stripe_alloc: count whole stripes stranded behind
live runs as trapped" (earlier in this series) that need code from
"btrfs: stripe_meta: hold back whole stripes so tree blocks always land
and trapped groups can be reclaimed", which comes after it in this
series: they are applied here, once "btrfs: stripe_meta: hold back whole
stripes so tree blocks always land and trapped groups can be reclaimed"
is in, instead of with the rest of "btrfs: stripe_alloc: count whole
stripes stranded behind live runs as trapped".
btrfs: stripe_meta: hold back whole stripes so tree blocks always land and trapped groups can be reclaimed
Two failures at the metadata fill edge under stripe_meta, both seen
with the tiny-file fill test on raid5 metadata.
The first is a transaction abort. A tree block is reserved in bytes
and admitted while claimable whole stripes cover the outstanding
reservations, but it is placed in whole stripes: the open run it would
have joined is closed at the transaction boundary, so every group
claims at least one fresh full stripe per transaction. The last few
megabytes of claimable supply go to those claims, made for reservations
admitted earlier, and the next admitted tree block finds no stripe at
all ("stripe_meta: tree block allocation of 16384 bytes returned
ENOSPC: claimable 0 open 1622016 trapped 1614266368"), which aborts the
transaction. Pre-pay two transactions' worth of claims -- twice the
number of writable groups times the widest full stripe -- in the
space_info's stripe margin, recomputed at each commit scan. Best-fit
run selection opens a fresh run only when a group's runs are full, so
this is a few megabytes, not a fraction of the space_info.
The second is a filesystem that quietly runs out at half capacity.
COW frees trap stripes rather than free them: a stripe is claimable
again only once every tree block in it is dead, which random deaths in
a sixteen-block stripe essentially never achieve, so the METADATA
space_info settles around half used, half trapped and no claimable
stripe left (3.04 GiB metadata, 49% used, fs not writable, deletes
included). Only relocation packs live tree blocks back into whole
stripes, and it needs the moved group's live bytes claimable elsewhere
before it starts -- by the time the trigger fires there is nothing
left. Hold back a reserve of whole stripes -- the largest writable
group's length, capped at a quarter of the space_info, nothing with a
single group -- from every metadata reservation except those made by
the relocation task, identified by a task pointer that reloc_ctl
already implies. Reservations admitted against the reserve reach the
same tickets, so the flag travels with the ticket. The reserve is an
accounting hold-back, not a partition: a committing transaction still
allocates from those stripes when it must, and the next scan restates
the reserve from what remains. Both counters are cleared when
stripe_alloc is turned off, since the scan that maintains them stops
with it.
The reserve is reported through sysfs as bytes_stripe_reserve and in
the space_info dump, whose stripe_claimable and stripe_margin values
were printed under each other's label.
Who may draw on the reserve:
- Relocation, for the whole of btrfs_relocate_block_group(), which
reloc_chunk_start() serializes: the reservation create_reloc_inode()
makes before the first stage must not stay behind the reserve, or every
metadata relocation fails exactly when the supply is below it (511 of
536 failed relocations of a raid6/raid6 octopus run ended so). Making a
group read-only, relocation's first step, draws on it too.
- The transaction machinery's own reservations, NO_FLUSH, FLUSH_LIMIT and
EMERGENCY: the reserve holds back user operations so relocation and the
commit after it have room, and must not hold back the commit (an
extent tree COW freeing deleted leaves aborted the transaction with
163 MiB of whole stripes unused). User reservations (FLUSH_ALL,
ALL_STEAL, EVICT, DATA) still stop at it.
- The global reserve steal is refused once the claimable supply is down
to the reserve plus the claim margin: an inline unlink frees no stripe
and each transaction of unlinks takes fresh ones, so stolen
reservations end in an abort. The unlink fails with ENOSPC instead.
The orphan cleanup at mount and ro->rw remount (the task is recorded in
btrfs_start_pre_rw_mount()) steals regardless: a failed eviction there
fails the read-write mount, and a filesystem that cannot be mounted
read-write can never run the relocation the reserve is for.
The reserve is a multiple of the metadata full stripe (stripe_round_up(),
not round_up(): the widths are not powers of two).
Raid56 metadata groups go through a runtime enable or disable too: the
arm and disarm sweep covers METADATA space_infos, the commit-time
metadata retirement runs whether or not the policy is on (so a disable
does not leave the running transaction's metadata runs and their group
references until unmount), and the enable drain commits the running
transaction before arming the write-hole check, since tree blocks the
legacy allocator placed in it are written by that commit into partly used
stripes. That one commit's exposure is inherent.
Zygo Blaxell [Thu, 22 Oct 2026 20:47:05 +0000 (16:47 -0400)]
btrfs: stripe_meta: extend stripe reclaim to metadata block groups
The stripe reclaim of the reclaim subtopic handles stripe_alloc data block
groups only. stripe_meta allocates tree blocks in whole stripes too, so
its block groups trap free space in partially used stripes the same way,
and that space comes back only by moving the live tree blocks out of those
stripes. Metadata needs it more than data: a metadata reservation that
finds no claimable stripe aborts the transaction instead of failing one
write, so trapped metadata space that is never reclaimed ends in a forced
read-only filesystem.
Extend each reclaim mechanism to stripe_meta block groups:
- btrfs_reclaim_block_group(): allocate a chunk to relocate into when the
other groups' claimable stripes cannot take the group's live bytes, and
relocate the group even while it has reserved or pinned bytes. Under
COW, a metadata group always has tree blocks reserved or pinned, so the
generic bail-out would defer it forever.
- The cleaner's periodic re-evaluation scans the metadata space_info as
well as the data one.
- Compaction counts metadata extent items: a tree block is nodesize long
and never spans a stripe, so for a metadata group the closure is the
partial stripes themselves.
- stripe_reclaim_now accepts stripe_meta block groups, for one group or
for all of them.
The comments that name the groups reclaim handles now name both.
Zygo Blaxell [Sat, 10 Oct 2026 04:19:31 +0000 (00:19 -0400)]
btrfs: stripe_alloc: skip RAID5/6 groups that are mapped as RAID1/RAID1C3 (stripe_meta part)
The hunks of "btrfs: stripe_alloc: skip RAID5/6 groups that are mapped
as RAID1/RAID1C3" (earlier in this series) that need code from "btrfs:
stripe_meta: allocate tree blocks by whole stripes", which comes after
it in this series: they are applied here, once "btrfs: stripe_meta:
allocate tree blocks by whole stripes" is in, instead of with the rest
of "btrfs: stripe_alloc: skip RAID5/6 groups that are mapped as
RAID1/RAID1C3".
Zygo Blaxell [Thu, 1 Oct 2026 16:14:42 +0000 (12:14 -0400)]
btrfs: stripe_alloc: keep a stripe in the rbio cache until its run has written all of it (stripe_meta part)
The hunks of "btrfs: stripe_alloc: keep a stripe in the rbio cache until
its run has written all of it" (earlier in this series) that need code
from "btrfs: stripe_meta: allocate tree blocks by whole stripes", which
comes after it in this series: they are applied here, once "btrfs:
stripe_meta: allocate tree blocks by whole stripes" is in, instead of
with the rest of "btrfs: stripe_alloc: keep a stripe in the rbio cache
until its run has written all of it".
Zygo Blaxell [Sun, 9 Aug 2026 22:05:48 +0000 (18:05 -0400)]
btrfs: stripe_meta: allocate tree blocks by whole stripes
Metadata gets the placement rule data already has: tree blocks are handed
out from runs of fully free stripes, and a stripe is never revisited once
the transaction that filled it has written it. No separate mount option: the
chunk profile already decides it. raid56 metadata has the same write hole
as raid56 data, so it gets the same protection at the same time, and
metadata on any other profile has no write hole to close -- the policy's
RAID56_MASK test simply does not match it, so nothing changes there.
It does cost a stranded stripe tail per commit, and a metadata allocation
that fails aborts the transaction rather than failing one write. Those
are the price of covering raid56 metadata at all rather than a reason to
make it opt-in separately: a filesystem unwilling to pay them does not
want raid56 metadata, and converting it to raid1c3/raid1c4 removes both
the cost and the write hole.
Only system chunks are left uncovered: btrfs_is_stripe_meta_bg() tests the
METADATA flag and a system chunk does not carry it, so a raid56 system
chunk keeps the legacy read-modify-write. The mount-time warning is
narrowed to say exactly that instead of claiming metadata is uncovered,
which it no longer is.
The machinery is the data machinery, including the per-sector liveness
map the previous patch gave it. Metadata needed three things of its own:
- A completion report. A run drains when every byte allocated from it
has been accounted for, and metadata had no equivalent of ordered
extent completion, so its runs could never drain -- the deadlock that
mixed block groups hit. end_bbio_meta_write() now reports, and
btrfs_open_stripe_write_abandoned() covers a tree block freed before
it was ever written (COWed twice in one transaction, say).
- A two-phase closure. btrfs_retire_open_stripes() runs before the
commit writes tree blocks, so it skips metadata runs; they are closed
and drained afterwards by btrfs_retire_meta_stripes(), once every block
is on disk. Closing earlier would strand blocks the commit has not
allocated yet, draining earlier would wait for writes it has not
issued. Metadata runs are also freed there rather than when their
counter reaches zero: a stripe's columns can be written by separate
rbios, and freeing on the first completion leaves the second with no
run to pad or merge against.
- Whole stripes per writeback pass. The opportunistic writeback passes
do not know about stripes and will happily write three tree blocks now
and five later; btrfs_defer_stripe_meta_write() leaves the blocks
dirty for the commit (or a WB_SYNC_ALL waiter) instead, the same shape
as the zoned write-pointer deferral.
Two free paths bypass the usual completion reporting and must report the
block abandoned themselves: btrfs_free_tree_block() can send a
same-transaction unwritten block through the delayed-ref machinery when
check_ref_cleanup() finds its ref head already processed, and
clean_log_buffer() discards unwritten log tree blocks via
btrfs_pin_reserved_extent() without ever visiting btrfs_free_tree_block().
Either miss leaves the block counted as inflight forever: the run never
drains, the block group reference never drops, and unmount asserts. Both
were found by the dm-log-writes replay matrix, whose crash-recovery mounts
exercise both paths routinely.
Trapped free space is maintained for stripe_meta groups as for data: the
disarm, rescan and commit-time scan accept them, the commit scan sums the
METADATA space_info, and inc_block_group_ro() applies the trapped-bytes
subtraction to metadata groups too. btrfs_update_global_block_rsv()
compares the global reserve against the capacity minus the trapped bytes,
so a metadata chunk is allocated while unallocated space remains, and a
stripe_meta tree block allocation that fails with ENOSPC (which aborts the
transaction) prints the stripe counters.
btrfs_retire_meta_stripes() syncs the open-run remainder into the
space_info after it closes the runs, as the data retirement does.
report_uncovered_rmw() keeps its "not covered by stripe_alloc" message for
what it was written for, system chunks and mixed groups, and no longer
says it of a covered stripe_meta group. btrfs_free_tree_block() checks
its block group lookup.
A mount whose log replay fails after allocating tree blocks, and the
unmount of an aborted filesystem, never commit again, so the commit-time
retirement never releases their open runs and block group references:
both release every open run before the block groups are freed, reporting
any run that still counts bytes in flight.
Mark Korenberg [Fri, 6 Nov 2026 17:33:10 +0000 (22:33 +0500)]
btrfs: raid56: pass REQ_META to per-device bios
RAID5/6 writes, and reads that need a rebuild, go through an rbio,
which builds new per-device bios from a bare REQ_OP_READ or
REQ_OP_WRITE. So all tree block writes on RAID5/6 lose REQ_META, while
on the other profiles the per-device bios are clones and keep it. The
same goes for a tree block read rebuilt from the other stripes, for
example with a missing device.
Pass REQ_META on to all per-device bios of an rbio, data and P/Q alike,
if any bio it serves has it. Scrub and relocation do not set REQ_META,
so their rbios stay untagged, as do rbios that only serve file data.
[zygo: 6.18 backport: 6.18's rbio_add_io_sector() takes the place of
rbio_add_io_paddrs() and sizes the bio with BTRFS_STRIPE_LEN; context
adjusted, the change itself is unchanged]
Zygo Blaxell [Sun, 25 Oct 2026 03:41:05 +0000 (23:41 -0400)]
btrfs: stripe_alloc: log the space counters when data writeback hits ENOSPC
A data writeback allocation that fails with ENOSPC under stripe_alloc
drops the buffered data in its range. The reservation margin is meant to
make that impossible, so the warning is the only evidence that admission
and the allocator disagreed, and today it names only the allocation size.
A drop seen once in a long test cannot be explained afterwards.
Log what admission decided on in the same line: the whole-stripe supply
(claimable, the part dedicated to relocation, unmanaged free space), what
is held against it (may_use, the outstanding extents' margin), the open,
trapped and stranded bytes, reserved and total, and the margin unit. The
"despite reservation margin" wording is kept, so existing log scanners
still match it.
On CONFIG_BTRFS_DEBUG builds, also log one line per block group with its
full stripe length, claimable bytes, largest known claimable run, open
remainder and trapped and stranded bytes. Unlike btrfs_dump_space_info()
this walks no free space entries. It takes groups_sem for read and each
group's lock, as the ENOSPC_DEBUG dump a few lines above already does at
this call site, so it adds no lock ordering.
The line and the walk share one rate limit, so a burst of failures cannot
flood the log with group lines.
Zygo Blaxell [Fri, 9 Oct 2026 23:13:11 +0000 (19:13 -0400)]
btrfs: stripe_alloc: count and log ordered extents that end with sectors unwritten
When an ordered extent reports to its stripe run, sectors it allocated
that no write reached are marked written so nothing waits for them (a
failed write, truncated folios, a short direct write, error teardown).
Whether any of these paths can still leave a waiter stuck in practice is
not known: the log settle that motivated the marking seals and frees a
shared LOG run before it would wait.
Count such extents and their unwritten bytes in stripe_run_stats
(unwritten_on_end, unwritten_on_end_bytes) and log each one, rate
limited, with the extent's identity, truncated length and flags, to see
which paths produce them.
One class is harmless and is counted but not logged: an extent marked
TRUNCATED, with no IOERR and not compressed (disk_num_bytes == num_bytes),
whose unwritten sectors are exactly the part truncation cut off,
[disk_bytenr + ALIGN(truncated_len, sectorsize), disk_bytenr +
disk_num_bytes). It comes from a truncate or O_TRUNC racing writeback and
from a direct write whose bounce-buffer copy faults before copying
anything, and it would bury the lines the message is there to find.
btrfs_open_stripe_write_done_run() returns the lowest unwritten sector so
that the tail can be told from merely as many sectors. Those extents go to
unwritten_on_end_truncated and unwritten_on_end_truncated_bytes;
unwritten_on_end and unwritten_on_end_bytes count only the logged ones, so
any increase there is worth a look.
Zygo Blaxell [Sun, 4 Oct 2026 21:46:24 +0000 (17:46 -0400)]
btrfs: stripe_alloc: break rbio pins and open stripe runs down by run class
Under fsync load the rbio cache holds up to ~190 pins its runs still need,
against 35-55 with the same load on raid1 metadata, so most of them look
like metadata stripes: tree-log blocks share the transaction's META run
with the tree blocks COWed in it, a log commit writes its part of a stripe
and the rest waits for the transaction commit. Data runs are private per
fsyncing inode and closed at every log commit, so they should be many and
short. Say which, instead of inferring it.
stripe_park_stats gains rbio_pinned_live_<class> (live pins by the class of
the run that still needs the stripe) and open_runs_<class> (open runs by
class), for class cow, reloc, log, nocow and meta.
btrfs_stripe_run_incomplete() becomes a wrapper around
btrfs_stripe_run_incomplete_class(), which also says which run's class.
btrfs: stripe_alloc: optionally audit data read-modify-writes against the committed extent tree
The attribution counters say which RMWs the allocator's design explains;
they do not show that an RMW put nothing at risk. That depends on what
the uncovered sectors hold: an RMW recomputes parity under them while
they stay put, so a crash mid-write can tear them, and that matters
only if a committed transaction references them and the owner is not
relocation's own orphaned copy.
Add an audit that answers it from the tree. For every data RMW, walk
the commit root of the extent tree (no locking, as
rmw_block_referenced() does; and like it, cached blocks only and
commit_root_sem only when it is free, because the full stripe lock is
held: a miss counts in audit_err, and mixed block groups, whose tree
blocks share the stripe, are not audited) for extents overlapping the
stripe's uncovered sectors, and read each one's data backrefs, inline
and keyed. An extent owned only by the data reloc tree counts as
relocation's (data_rmw_hole_reloc_only); any other owner, a shared ref
(owner unknown), or no ref at all is foreign and counts as a write hole
(data_rmw_hole_no_run / _uncovered / _other, by pad verdict). Log the
writer of every foreign hole and of every NO_RUN RMW, rate-limited from
separate call sites so a flood of the second cannot starve the first:
root, inode, file offset, ordered extent flags, disk bytenr, stripe run
sequence, and the covering run as the pad oracle saw it. The commit
root switches just before the super lands, so the check can report a
hole early, never hide one.
A tree search per RMW is not free (a stock raid5 filesystem does
thousands a minute), so the audit is built only with CONFIG_BTRFS_DEBUG
and runs only while /sys/fs/btrfs/<uuid>/debug/stripe_rmw_audit is set,
which it is not by default. Its counters are in
debug/stripe_rmw_audit_stats, with data_rmw_audited so a reader can
tell an audit that found nothing from one that never ran.
btrfs: stripe_alloc: attribute each data read-modify-write to why padding declined
data_rmw counts read-modify-writes, and the pad_decline_* counters count
the pad oracle's refusals, but a refusal does not always lead to a read
(a full stripe or a cached one does not read), so the two cannot be
subtracted: nothing says which RMWs the allocator explains. With
allow_rmw empty, a data RMW in a stripe_alloc group is expected in two
places only:
- at the open run's own frontier (the pad verdict UNCOVERED): the
sectors it reads belong to the same run, in the same transaction,
and are not durable yet;
- from relocation writing into its own preallocated extents, which no
root but the data reloc tree references until relocation moves the
pointers after writeback.
Record the pad oracle's verdict on the rbio (rmw_try_pad_full()) and,
at the RMW site, split data_rmw by it (data_rmw_<reason>, with
data_rmw_not_tried when the oracle was not consulted and
data_rmw_pad_ok_nomem when it agreed but the zero pages could not be
allocated). NO_RUN is the oracle's catch-all exit, so split it further
by the exit taken (data_rmw_norun_*) and count those in a read-only
group (data_rmw_norun_bg_ro). Count the data RMWs written by the data
reloc inode outside the frontier (data_rmw_reloc_writer, disjoint from
data_rmw_uncovered). All in stripe_park_stats.
A test can then assert that data_rmw - data_rmw_uncovered -
data_rmw_reloc_writer is zero on a stripe_alloc filesystem, on a
production kernel, with reclaim relocating during the run -- which it
could not do from the totals alone.
The cost is a byte of state per rbio and a few counter increments per
RMW, plus one block group lookup for NO_RUN RMWs.
btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents (debug part)
The hunks of "btrfs: stripe_alloc: allow-rmw policy, and write-in-place
for isolated nocow extents" (earlier in this series) that need code from
"btrfs: add a write-hole invariant checker to the raid56 write path",
which comes after it in this series: they are applied here, once "btrfs:
add a write-hole invariant checker to the raid56 write path" is in,
instead of with the rest of "btrfs: stripe_alloc: allow-rmw policy, and
write-in-place for isolated nocow extents".
btrfs: stripe_alloc: say at mount whether the policy is on (debug part)
The hunks of "btrfs: stripe_alloc: say at mount whether the policy is
on" (earlier in this series) that need code from "btrfs: add a
write-hole invariant checker to the raid56 write path", which comes
after it in this series: they are applied here, once "btrfs: add a
write-hole invariant checker to the raid56 write path" is in, instead of
with the rest of "btrfs: stripe_alloc: say at mount whether the policy
is on".
Zygo Blaxell [Tue, 4 Aug 2026 19:51:54 +0000 (15:51 -0400)]
btrfs: stripe_alloc: report read-modify-write of uncovered stripes (debug part)
The debug half of "btrfs: stripe_alloc: report read-modify-write of
uncovered stripes", earlier in this series: the meta_rmw counter in
stripe_park_stats, which goes with the series' other debug statistics.
The warning itself is in that patch.
btrfs: add a write-hole invariant checker to the raid56 write path
With stripe-exclusive allocation, a raid56 data stripe may only be
written while an open or draining stripe run covers it: after its run
retires and drains at a transaction commit, nothing may ever write to
it again, and a write outside any run means an allocation bypassed the
policy. Both cases are the write hole about to happen.
Check the invariant (under CONFIG_BTRFS_DEBUG) for every raid56 write
operation, full-stripe and sub-stripe alike, at rmw_rbio() time. This
turns every upstream violation -- a missed allocation path, a
retirement ordering bug, an accounting leak -- into a deterministic
WARN at the moment of the offending write, instead of silent damage
that needs a crash plus a device failure plus a scrub to observe. The
bios gathered in an rbio have not reported their IO done yet, so their
runs cannot drain under the check: no false positives from completion
races.
Block groups that ever hosted relocation-class runs are skipped
(sticky, debug-only flag): relocation legitimately overwrites its
preallocated extents in place after their runs drain, and the write
path cannot tell those writes from violations.
A runtime enable of the policy (the btrfs.stripe_alloc property set on a
live filesystem) flips it while the legacy allocator may still have writes
in flight, in partly used stripes that no run covers, and the check would
warn on the first of them. So a runtime enable holds the check off until
a worker has started delalloc on all roots and waited for every ordered
extent (the property is applied inside a transaction, where that cannot
wait); the same wait writes any delalloc tail a writeback racing the
enable left unsubmitted. A mount-time enable has nothing in flight.
close_ctree() flushes the worker.
Zygo Blaxell [Tue, 6 Oct 2026 04:11:38 +0000 (00:11 -0400)]
btrfs: stripe_alloc: share LOG data runs per log transaction
An fsyncing inode's data goes into a private LOG-class stripe run, which
each log commit settles: the run is closed, the commit waits for all of the
run's data IO, and the extents left in its final partial stripe are
recorded for carry-forward, so the next fsync re-dirties them and moves
them into the inode's new run. Every fsync therefore leaves one partial
stripe per inode, and a back-to-back fsync loop rewrites the previous
fsync's tail each time.
Share the runs instead:
- LOG-class runs are keyed by (subvolume, log_transid at allocation)
instead of the inode, so the inodes logging in one log transaction fill
the same stripes. An allocation of a full stripe or more keeps the
inode's private run, so small fsync extents never share the partial last
stripe of a large one and wait for its IO.
- A log commit does not close a shared run. It seals the stripe holding
the end of the logged extent: the run's frontier moves to that stripe's
end, the skipped sectors go back to the free space cache like a closed
run's tail (the stripe's final write pads them), and the run goes on in
the next stripe. Nothing allocates beside the extent after its log
commit.
- The settle kicks only the logged extent's stripes and waits until every
allocated sector in them is written (the run's written map), rather than
for the whole run to drain. Writers sharing those stripes park and merge
into the same full-stripe write, so the wait couples an fsync to them and
to nothing else. It has no timeout: returning while a neighbour's sector
is unwritten would let its later partial write RMW the extent after the
log made it durable. Completed writes wake the waiters.
- Nothing is carried forward: a final partial stripe stays until the
extents in it are freed or compaction moves them.
Measured on 5 x 8 GiB raid5 data with raid1 and raid5 metadata, against
private runs with carry-forward, same rep: at 16 and 64 writers and with
one fsync per inode, 1.2-2.9x the fsyncs/s (one 64-writer rep of eight:
0.8x), 3-8x fewer device bytes per fsync and less slab; one writer is at
parity on latency with 17-29% fewer device bytes. The one workload that
can lose is two large streaming writers beside small fsync writers: a small
fsync waits for neighbours that joined its stripe and sit behind the large
IO (0.5-1.9x the fsyncs/s; the p99.9 there is dominated by a multi-second
log-commit tail that both modes, and stock raid5, show).
/sys/fs/btrfs/<uuid>/stripe_log_data_shared (default 1) switches a
filesystem back to private runs with carry-forward. stripe_run_stats gains
log_settle_waits and log_settle_wait_ns (both), log_carry_ranges and
log_carry_bytes (private runs), and log_seals and log_seal_bytes (shared
runs).
Private LOG and NOCOW runs are matched by (subvolume root id, inode
number), since inode numbers are unique only within a subvolume (every
subvolume's first file is inode 257): matching by inode alone let
same-numbered inodes of different subvolumes join each other's runs,
breaking NOCOW isolation, and let an eviction close other subvolumes'
runs and the META_LOG run of a subvolume whose id equals the inode number.
Eviction closes only the inode's own LOG and NOCOW runs, and the shared
LOG key uses the whole root id.
btrfs_log_settle_stripes() reads stripe_log_data_shared once per settle,
and every completed write wakes the settle waiters whatever the knob says,
so a sysfs change cannot leave an fsync waiting for a run only the commit
it blocks would provide.
btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents (fsync part 2)
The hunks of "btrfs: stripe_alloc: allow-rmw policy, and write-in-place
for isolated nocow extents" (earlier in this series) that need code from
"btrfs: stripe_alloc: carry an inode's log tail forward at fsync", which
comes after it in this series: they are applied here, once "btrfs:
stripe_alloc: carry an inode's log tail forward at fsync" is in, instead
of with the rest of "btrfs: stripe_alloc: allow-rmw policy, and
write-in-place for isolated nocow extents".
btrfs: stripe_alloc: carry an inode's log tail forward at fsync
Closing a log-active inode's private stripe run at each log commit
traps the run's final partial stripe tail every time the inode fsyncs:
a steady fsync stream burns one stripe per log commit until the stripes
free or balance runs.
Reclaim the tails by carrying the partial stripe's live data forward.
At each log commit, the settling walk now computes the closed run's
final partial stripe, and the logging paths record the file ranges of
the logged extents that live inside it on a small per-inode table
(bounded; overflow just means an extent is not carried). At the
inode's next fsync, before its delalloc flush, each recorded range that
still maps to the recorded disk bytenr is re-dirtied, defrag style
(reserve, reset delalloc state, mark the folios dirty; absent folios
are read back, which is a plain read of settled data). The flush then
COWs the carried ranges together with the new data into the inode's
current private run, and the old partial stripe empties and frees
whole.
Everything downstream is the ordinary COW pipeline: new extent maps,
ordered extents, checksums, file extent items and delayed refs, and
the log's modified-extents snapshot -- taken inside btrfs_log_inode
after the flush -- picks up the new addresses by itself. Reflinked
ranges need no special handling: foreign referents keep the old extent
alive in its closed (never again written) stripe, and only the space
reclaim degrades. A range that was rewritten, truncated, punched or
compressed simply fails the mapping check and stays put.
Crash safety does not regress. A carried extent's old copy is dropped
through the normal paths: if it was committed, the free pins until the
transaction commits; if it was logged but never committed, its ADD and
DROP delayed refs cancel in cleanup_ref_head(), which also pins
must_insert_reserved heads -- either way the old blocks cannot be
reallocated before the log that references them is superseded.
The settling walk also learned to report the partial stripe through a
widened btrfs_log_settle_stripes() signature (inode and file range
instead of fs_info); compressed extents pass a zero file length to opt
out of carrying while still settling.
The re-dirty reserves with NO_FLUSH. carry_one_range() takes the folio
locks and the extent range lock before reserving, and a flushing
(ticketed) reservation can sleep in wait_reserve_ticket() until the
flusher makes progress -- but every way forward needs the locks we
hold: FLUSH_DELALLOC has to write back the carried range, which blocks
in __folio_lock() on our locked folio, and a transaction commit (with
flushoncommit) waits for the same writeback. The whole filesystem then
wedges behind the stuck commit. Carrying is best effort by design and
the carried range's data is already durable in its old stripe, so on
ENOSPC just skip the carry and let the old stripe free the slow way.
The carry's reservation does not flush at all, including the qgroup
reserve: btrfs_check_data_free_space_noflush() reserves data space and
qgroup space without flushing, since try_flush_qgroup() would write back
the folios the carry holds locked. On EDQUOT, as on ENOSPC, the range is
just not carried. The carry also clears a stale EXTENT_NORESERVE (left by
a NOCOW write) with the other delalloc bits, so the data space it reserved
is released when the range is later written NOCOW.
btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents (fsync part)
The hunks of "btrfs: stripe_alloc: allow-rmw policy, and write-in-place
for isolated nocow extents" (earlier in this series) that need code from
"btrfs: stripe_alloc: settle a logged extent's stripes at log commit",
which comes after it in this series: they are applied here, once "btrfs:
stripe_alloc: settle a logged extent's stripes at log commit" is in,
instead of with the rest of "btrfs: stripe_alloc: allow-rmw policy, and
write-in-place for isolated nocow extents".
Zygo Blaxell [Sun, 2 Aug 2026 07:35:36 +0000 (03:35 -0400)]
btrfs: stripe_alloc: count open stripe run remainders against data reservations (fsync part)
The hunks of "btrfs: stripe_alloc: count open stripe run remainders
against data reservations" (earlier in this series) that need code from
"btrfs: stripe_alloc: settle a logged extent's stripes at log commit",
which comes after it in this series: they are applied here, once "btrfs:
stripe_alloc: settle a logged extent's stripes at log commit" is in,
instead of with the rest of "btrfs: stripe_alloc: count open stripe run
remainders against data reservations".
btrfs: stripe_alloc: settle a logged extent's stripes at log commit
Stripe-exclusive allocation bounds degraded-crash damage to the current
transaction, but within that window two log commits can still share a
stripe: fsync 1 writes the head of a stripe, the open run keeps filling
it, and a later write's read-modify-write rewrites the parity that
protects fsync 1's data. A degraded crash during the second write tears
the first -- the write hole's shape, confined to the log window, and the
reason the fsync guarantee has so far been "loss is detectable" rather
than "completed fsyncs survive".
Close it with a case analysis. A logged extent in a full stripe is
already safe: nothing ever writes a full stripe again, since COW never
overwrites and there are no free sectors left to allocate. A logged
extent in a partial stripe is exposed only to future writes into that
stripe's remaining sectors -- so at log commit, close the stripe's open
run (nothing further allocates into it), kick any parked partial writes
for it, and wait for its in-flight data IO before the log super is
written. This is the log-window analogue of invariant I2, which the
commit-time retirement provides for full commits. Both log paths are
hooked: the fast path per extent map in log_one_extent(), and the
full-sync path where copy_items() walks new data extents (old-transaction
extents are skipped there, and are exactly the ones already settled by
their own commit).
The drain is join-free: write_done reporting needs no transaction join,
so waiting under the inode log mutex and a running transaction handle is
safe. It is not bounded: it has no timeout and waits for data IO to
complete. Run inflight is counted from allocation, which happens during
writeback with submission following in the same pass, so in practice the
wait is bio flight time plus the parked-write deadline that the flush
short-circuits. (A later patch shares LOG runs per log transaction and
makes that the default; its settle waits until every allocated sector in
the logged extent's stripes is written, other writers' sectors included,
and is not bounded either.)
The cost is spatial: every fsync that logs an extent in a partial stripe
retires that stripe early, trapping its unwritten tail like any other
partially filled stripe until it frees or balance reclaims it.
Fsync-heavy workloads therefore burn a stripe tail per touched stripe
per fsync; the follow-up per-inode log runs and copy-forward relocation
exist to reclaim exactly that cost, and are optimizations on top of the
guarantee this patch completes.
On a filesystem that has gone into error the settle wait does not sleep
for IO: the log commit fails anyway, and the ordered extents in flight are
only torn down at unmount.
Zygo Blaxell [Thu, 22 Oct 2026 08:42:21 +0000 (04:42 -0400)]
btrfs: stripe_alloc: back off stripe reclaim after repeated ENOSPC
The reclaim worker backs off a space_info after repeated -ENOSPC failures.
Under stripe_alloc two more paths queue trapped groups, the commit-time
rescan and the cleaner tick, every five minutes each; have them honour the
backoff too, and have the reclaim predicate require room for relocation's
up-front metadata reservation as well. A stripe_reclaim_now request is
still tried during a backoff.
It showed in two ways:
- The octopus P1 pass (raid5 6 x 3 GiB, misc-next, bhive 2026-10-20):
with the data space fully allocated and ~2 GiB trapped, reclaim held the
exclusive operation for over an hour of fsstress and long after it,
"error relocating chunk ...: -28" repeating (28 attempts, 13 errors in
one loop), so a resize could not start and the replace failed. The
worker sat in a metadata reservation (btrfs_block_rsv_refill under
relocate_block_group): metadata had no room either.
- reclaim-futile-test.sh (raid5 6 x 1 GiB at the edge, fsstress churn):
226 reclaim errors in 600 s. Of the -28 returns, relocation's data
reservation lost the claimable stripes the room check had counted to
the concurrent writers.
The room check (claimable stripes in the other groups) is an estimate
taken when the group is queued, and it did not cover metadata: it now
also requires room for relocation's up-front metadata reservation (512
tree blocks, twice RELOCATION_RESERVED_NODES), which the octopus failure
needed.
Tested: reclaim-futile-test.sh measured 226 reclaim errors in 600 s on the
lane without the backoff (the fill-edge shape that loops; it does not loop
on every run). With it no run logged more than 5 errors per 600 s; in one,
five relocations failed with -ENOSPC and reclaim backed off once
(reclaim_backoffs +1), then resumed. stripe-reclaim-test and
stripe-reclaim-now-test pass.
Zygo Blaxell [Tue, 20 Oct 2026 23:40:35 +0000 (19:40 -0400)]
btrfs: stripe_alloc: add stripe_reclaim_now to reclaim stripes on demand
Background reclaim of stripe_alloc raid56 groups runs when a
commit-time rescan, or the cleaner's tick at most every five minutes per
group, finds a group past bg_reclaim_threshold. Nothing lets an operator
say "reclaim now": during an idle period, before a large write, or for one
group seen trapped. The only tool is balance, which relocates whole groups
whatever their state and cannot compact.
echo all > stripe_reclaim_now (or "1")
Queue every stripe_alloc raid56 group that the
reclaim predicate picks: relocate below the threshold once enough
of its free space is trapped, compact above it once stripe_compact
percent of its length is trapped. The five-minute hysteresis is
ignored.
echo <start> > stripe_reclaim_now
Reclaim the group at that logical start (decimal or 0x hex, as
`btrfs inspect-internal dump-tree -t chunk` and the "reclaim: "
debug lines print it) whatever its trapped space:
compact it if it is above the threshold and has trapped stripes,
relocate it otherwise.
The threshold is the space_info's bg_reclaim_threshold, or 75
(BTRFS_DEFAULT_RECLAIM_THRESH) where that is 0, the non-zoned default,
since the request is explicit. Both forms keep the room check: the other
groups' claimable stripes, plus the one chunk the worker allocates when
those fall short, must take what moves. The write returns once the groups
are queued; it does not wait for the reclaim. The worker re-checks its
thresholds and would drop a group it was asked to reclaim, so a group
queued by this knob carries the kind of reclaim to do
(stripe_reclaim_forced), which the worker honours once and clears.
Errors: -ENOENT when no group starts there; -EINVAL for anything but a
stripe_alloc raid56 group (balance's vrange filter relocates
the others); -EBUSY when the group is read-only (a balance or scrub has
it); -EAGAIN when it is already on a list (queued, unused or new);
-ENOSPC when there is no room; -EROFS on a read-only or errored
filesystem. The worker takes the balance exclusive operation and gives the
list back when another holds it, so a request made while a balance,
replace, resize or device add/remove runs would wait unseen: it fails at
once with -EAGAIN and nothing is queued; retry when the operation is done.
The reclaim worker's own hold (a balance with no balance control) is not
refused, as the worker picks up the new entries.
Expected use: an idle-time job (cron) writes "all"; an operator writes the
start of one group taken from the "reclaim: " debug lines or from the
chunk tree. The queued groups are logged at debug level with the same
"reclaim: " prefix.
There is no in-tree btrfs sysfs documentation; the btrfs-progs sysfs
chapter should list the file.
Tested with stripe-reclaim-now-test.sh (raid56-wh-harness): with
background reclaim off nothing moves; one group is relocated alone, and a
group above the threshold with trapped stripes is compacted; "all"
reclaims and trapped space falls; no such group, not a number, a raid1
metadata group and a read-only mount fail with the codes above, and a
paused balance and a running replace fail with -EAGAIN; data verifies and
btrfs check is clean.
The stripe_meta block group part of this change moved to
"btrfs: stripe_meta: extend stripe reclaim to metadata block groups".
Zygo Blaxell [Tue, 20 Oct 2026 17:04:21 +0000 (13:04 -0400)]
btrfs: stripe_alloc: log compaction and trapped bytes in the reclaim log
Say in the reclaim debug lines whether a stripe group is being compacted
or relocated, with the trapped bytes the reclaim trigger counted, and
whether it was compacted or relocated. Without that, stripe-reclaim-test,
which counts reclaims in the log, could not tell automatic compaction from
relocation, and an octopus replace stalled behind a reclaim the log did not
explain.
The hunks of "btrfs: stripe_alloc: keep relocation's stripe runs until
relocation ends" (earlier in this series) that need code from "btrfs:
stripe_alloc: compact mostly live groups instead of skipping them",
which comes after it in this series: they are applied here, once "btrfs:
stripe_alloc: compact mostly live groups instead of skipping them" is
in, instead of with the rest of "btrfs: stripe_alloc: keep relocation's
stripe runs until relocation ends".
btrfs: stripe_alloc: compact mostly live groups instead of skipping them
Reclaim only takes a stripe_alloc group whose live data is below
bg_reclaim_threshold: moving everything out of a fuller group costs nearly
a group's worth of writes and needs that much claimable space elsewhere.
But trapped free space is not confined to emptyish groups. Under steady
COW churn a group can stay mostly live while every commit leaves
partially used stripes behind. Metadata under stripe_meta showed it
first: on a raid6 metadata fill the trapped share of the metadata free
space reached 88% with no group ever falling below the threshold:
reclaim never ran, and the space never came back.
A group like that does not need all of its data moved. The trapped space
is in its partially used stripes; moving only the extents in those
stripes makes them whole again. An extent spanning several stripes frees
its bytes in all of them when it moves, so the stripes it shares with
others must move too, transitively: the selection is the closure of the
partial stripes under "shares an extent with".
Compaction runs from the reclaim worker, for a group whose live data
does not fall below the threshold but whose trapped bytes reach
stripe_compact percent of the group's length. Measuring them against the
group's free space, the test whole relocation uses, never qualifies a
mostly live group: its little free space is mostly whole stripes, and
compaction never ran during raid6/raid6 aging with gigabytes trapped.
- The group goes read-only as for relocation. After a commit, so the
free space cache and the extent tree's commit root agree, the partial
stripes are read from the free space cache and the closure is found
with a union-find over the group's stripes in one pass over its extent
items, each union carrying the live bytes of its extents.
- If the closure would move more than half the group's live bytes, the
group is relocated whole and removed instead: that costs about the
same and also gives back the group's raw space.
- Otherwise relocation runs with find_next_extent() skipping every
extent outside the selection, through both data stages, and the group
is committed (so what moved out is unpinned) and made writable again
instead of removed. replace_file_extents() rewrites only the pointers
to selected extents: the others did not move and have no new location
(a leaf referencing both kinds is common).
Balance, device shrink and removal still relocate whole groups
(btrfs_relocate_chunk() is unchanged; btrfs_compact_chunk() is the new
entry point). sysfs stripe_compact (default 10; 0 turns compaction off), and
stripe_run_stats counts compaction passes, fallbacks to whole relocation,
stripes evacuated and bytes moved.
The destination is ordinary relocation's: the moved extents need whole
claimable stripes in other groups, and a commit during the pass can leave
a partial stripe at the tail of a relocation run there. The admission
test for compaction therefore asks for half the group's live bytes
claimable elsewhere, the most a compaction can move.
The stripe_meta block group part of this change moved to
"btrfs: stripe_meta: extend stripe reclaim to metadata block groups".
btrfs: stripe_alloc: count whole stripes stranded behind live runs as trapped (reclaim part)
The hunks of "btrfs: stripe_alloc: count whole stripes stranded behind
live runs as trapped" (earlier in this series) that need code from
"btrfs: stripe_alloc: hand groups full of trapped free space to the
reclaim worker", which comes after it in this series: they are applied
here, once "btrfs: stripe_alloc: hand groups full of trapped free space
to the reclaim worker" is in, instead of with the rest of "btrfs:
stripe_alloc: count whole stripes stranded behind live runs as trapped".
btrfs: stripe_alloc: hand groups full of trapped free space to the reclaim worker
Zoned filesystems mark a block group for reclaim as soon as its
unusable bytes reach bg_reclaim_threshold percent of its capacity, so
space that can no longer be written is recycled without an operator
running balance. stripe_alloc had no equivalent. The non-zoned
trigger consults the threshold only where a free drops a group's used
bytes across it, and the test hosts were already running every mount
with bg_reclaim_threshold=75 when this was measured: trapped tails
still accumulated until balance (1.86 GiB of 6.5 GiB after a few hours
of churn), while every byte-based check counted them as free, because
that predicate keys on used bytes only and fires only at the crossing
-- a group hollowed out after it was already below the threshold, or
one whose used bytes never moved while its stripes trapped, is never
queued, and a group at the same used fraction with nothing trapped is.
Mark a group for the existing reclaim worker when its trapped bytes
reach the threshold; the worker re-checks with
should_reclaim_block_group(). The predicate, factored out as
btrfs_stripe_bg_wants_reclaim(), compares the trapped bytes with the
group's free space (length minus used) rather than its length -- a
stripe_alloc group also holds claimable whole stripes, and in practice
fragmentation settles around two thirds of the group trapped with a
fifth used, nothing left worth writing into yet below any sensible
fraction of the length -- and requires room for the group's live data
in the rest of the claimable supply after the bytes already promised to
admitted writers, so a group that cannot be relocated is left alone
until space is freed instead of being retried every commit while the
read-only transitions starve writers.
The trigger runs from the cleaner, which the transaction thread wakes
every cycle whether or not a transaction exists, as well as at the
commit-time rescan: on an idle filesystem nothing rescans, and a group
already on another list (a chunk created in the running transaction is
still on trans->new_bgs) cannot be linked to the reclaim list, so the
five-minute hysteresis stamp is taken only when the mark actually
lands. The worker re-checks whatever it takes, so a stale read costs at
most one wasted attempt. bg_reclaim_threshold defaults to 0 on
non-zoned filesystems, so nothing changes until the sysfs knob is set;
the default is left for review. Relocation needs inc_block_group_ro()
to succeed, hence the earlier fix.
The worker does not defer a queued stripe group that has bytes reserved
or pinned: it was queued for space no waiting frees, and under COW churn
it can have blocks reserved or pinned at any moment (under stripe_meta,
a metadata group always does).
Relocation waits for in-flight allocations and writers on its own.
The stripe_meta block group part of this change moved to
"btrfs: stripe_meta: extend stripe reclaim to metadata block groups".
Zygo Blaxell [Sun, 2 Aug 2026 07:35:36 +0000 (03:35 -0400)]
btrfs: stripe_alloc: count open stripe run remainders against data reservations (allow_rmw part)
The hunks of "btrfs: stripe_alloc: count open stripe run remainders
against data reservations" (earlier in this series) that need code from
"btrfs: stripe_alloc: persist nocow runs across commits and remounts",
which comes after it in this series: they are applied here, once "btrfs:
stripe_alloc: persist nocow runs across commits and remounts" is in,
instead of with the rest of "btrfs: stripe_alloc: count open stripe run
remainders against data reservations".
btrfs: stripe_alloc: persist nocow runs across commits and remounts
A nodatacow inode's private stripe run used to close at every
transaction commit like all runs, so a slowly appended nocow file burned
a fresh stripe per commit -- and after a remount its partial stripe's
free tail was abandoned outright. Neither cost buys anything: the run
machinery's commit-time closing exists for invariant I2, and a nocow
stripe holds only the owner's write-hole-waived data, so there is
nothing for I2 to protect.
Keep NOCOW-class runs open across commits: the commit-time retirement
and its drain predicate skip them (their extents insert without the
window-sequence deferral, which is fine -- the data is on disk when the
ordered extent finishes, and later same-stripe writes can tear only the
owner's own data). They still close on forced quiescing (read-only,
removal, unmount) and now on the owning inode's eviction, so a cached
but idle inode cannot pin a claimed tail forever.
Across remounts, re-adopt instead: when a nocow allocation's hint
points into a partial stripe that the committed extent tree proves is
wholly owned by the allocating inode, claim exactly the stripe's free
tail (a new exact-range claim that verifies every byte is free before
removing; nothing else can consume free space inside a partially used
stripe, so verify-then-remove cannot race) and continue the run at the
old frontier. Appends to a nocow file then pack sequentially through
commits, evictions and remounts alike.
The re-adoption lookup must not block. It runs from
btrfs_alloc_from_inode_stripe_run(), i.e. inside find_free_extent(),
which holds space_info->groups_sem for read, and stripe_extents_owned_by()
does a full btrfs_search_slot() on the extent tree. The reverse order is
longstanding upstream -- delayed ref processing holds extent tree locks
and then calls find_free_extent(), which takes groups_sem -- so waiting
here closes a cycle. Both sides only take groups_sem for read, which is
not enough: it is an rw_semaphore and block group creation and removal
take it for write, so a writer queued between the two readers blocks the
second one. The check cannot be hoisted above the lock, since it is a
question about the specific stripe the allocator has just settled on, but
it does not have to block: it is read-only and advisory, and every caller
already treats "not provably ours" as a reason to decline the
optimisation rather than as a fact about the extents. So set path->nowait
for the groups_sem caller and let -EAGAIN fall into the existing error
paths; the cost on contention is a re-adoption that does not happen.
btrfs_stripe_nocow_writable() keeps the blocking search -- it runs from
the nocow check with no groups_sem held, and a spurious "not ours" there
would force COW on an extent that does not need it.
Whether a nocow extent can be written in place at all is decided by the
stripe_alloc_allow_rmw policy, and with nothing waived btrfs_stripe_nocow_
writable() forces COW for every such write. A private run would then hold
only COWed extents and unwritten preallocation, and its survival would
only strand the whole stripes freed inside its range (an fsstress fill to
ENOSPC left hundreds of such runs in one group, holding 99% of its free
whole stripes). So prealloc and nodatacow extents are steered into a
NOCOW run only when the policy lets that kind of extent be written in
place, and a NOCOW run outlives commits only when the policy allowed
in-place writes when it was opened. The verdict is recorded in the run,
so the commit's close and settle predicates agree when the policy changes
underneath them.
full_stripe_len is a power of two only when the data stripe count is
(raid5 over 4 devices gives 192K), so stripes are rounded and tested for
alignment by division, in one helper each, never with round_down() or
IS_ALIGNED().
The cleaner does not delete an unused block group while a stripe run
still covers it (a persistent nocow run whose extents were all freed, or
the running transaction's metadata run): a run's unallocated remainder is
counted in none of used, reserved and pinned, and its tail would later be
added to the removed group's free space cache. The group stays read-only
and is left for a later pass; its runs close by the next commit.
The nocow check passes an IOCB_NOWAIT write's nowait down to its extent
tree search; when that cannot answer, the extent is treated as not the
inode's own and the write falls back to COW, which a NOWAIT write refuses
with -EAGAIN.
btrfs: stripe_alloc: drop the mount options, the property is the interface (allow_rmw part)
The hunks of "btrfs: stripe_alloc: drop the mount options, the property
is the interface" (earlier in this series) that need code from "btrfs:
stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow
extents", which comes after it in this series: they are applied here,
once "btrfs: stripe_alloc: allow-rmw policy, and write-in-place for
isolated nocow extents" is in, instead of with the rest of "btrfs:
stripe_alloc: drop the mount options, the property is the interface".
Zygo Blaxell [Sun, 9 Aug 2026 03:14:23 +0000 (23:14 -0400)]
btrfs: stripe_alloc: track per-sector liveness and pad from the map (allow_rmw part)
The hunks of "btrfs: stripe_alloc: track per-sector liveness and pad
from the map" (earlier in this series) that need code from "btrfs:
stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow
extents", which comes after it in this series: they are applied here,
once "btrfs: stripe_alloc: allow-rmw policy, and write-in-place for
isolated nocow extents" is in, instead of with the rest of "btrfs:
stripe_alloc: track per-sector liveness and pad from the map".
Zygo Blaxell [Sat, 8 Aug 2026 06:03:40 +0000 (02:03 -0400)]
btrfs: stripe_alloc: tunable park deadlines, and say why padding declined (allow_rmw part)
The hunks of "btrfs: stripe_alloc: tunable park deadlines, and say why
padding declined" (earlier in this series) that need code from "btrfs:
stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow
extents", which comes after it in this series: they are applied here,
once "btrfs: stripe_alloc: allow-rmw policy, and write-in-place for
isolated nocow extents" is in, instead of with the rest of "btrfs:
stripe_alloc: tunable park deadlines, and say why padding declined".
btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents
Re-enable write-in-place for nodatacow files and preallocated extents
under stripe-exclusive allocation, with the understood caveat that the
write hole cannot be prevented for data that opts out of COW: an
in-place write RMWs its stripe's parity, so a degraded crash can tear
the stripe. What CAN be guaranteed is the blast radius: in-place is
permitted only for extents whose full stripes are isolated to the
writing inode, so such a crash can tear only the writing file's own
data -- the nodatacow contract, no worse.
The gate is per extent, decided where nocow eligibility is already
checked: fast path, the stripe belongs to one of the inode's own
NOCOW-class runs (which only ever held its extents); slow path, the
committed extent tree proves sole ownership (single plain data ref,
count 1, matching root and objectid; anything shared, foreign or
metadata fails). The fully-free claim rule keeps uncommitted foreign
extents out of partially used stripes, so the committed tree is
authoritative. Extents that fail -- anything allocated before stripe
isolation existed, extents shared through reflink or snapshots,
relocated extents -- simply stay force-COWed, and because their rewrite
is steered into the inode's private NOCOW run, the next overwrite of
the same data passes: legacy nocow files migrate themselves to
isolation in one COW generation, with no tool and no flag day.
Log-commit settling skips NOCOW-class runs: nodatacow data gets no
fsync survival guarantee (its own later in-place writes can always
tear it), so closing the run would trap its tail for nothing. The
write-hole debug checker skips groups that hosted NOCOW-class runs,
like relocation-used groups, since isolated in-place writes land in
stripes whose runs have drained. Settling, the fsync steering and the
checker come later in this series, so these parts of the change are in
its "(fsync part)", "(fsync part 2)" and "(debug part)" patches.
The interface is a word-list policy naming the cases in which
stripe_alloc may permit the legacy unsafe RMW, each independently:
nodatacow in-place writes for nodatacow files' extents
prealloc in-place writes into preallocated extents
fsync waive the close-at-log-commit guarantee: no settling, no
per-inode LOG steering, no carry-forward; logged stripes
may be extended and RMWed as without stripe_alloc
It is "allow_rmw", not "allow_overwrite": the fsync case overwrites
nothing, but all three permit read-modify-write of stripes that a
degraded crash can then tear. The nodatacow and prealloc cases require
the per-extent stripe isolation above (the blast radius stays confined
to the writing file); fsync restores the exposure stock btrfs has,
where a degraded crash may cost just-fsynced data, detectably, in
exchange for none of the log-commit costs.
The policy is persistent as the btrfs.stripe_alloc_allow_rmw property
on the top-level root directory, following stripe_alloc's precedent
(applied when the root inode loads during mount, before any user IO),
with a mount option of the same name as a non-persistent override; the
effective policy is their union. Words are separated by comma, space
or colon -- the mount option form must use colon, since mount splits
options at commas. Everything defaults off: plain stripe_alloc keeps
forcing COW and keeps the full fsync guarantee.
Sync writes into a NOCOW-class run do not park: in-place overwrites arrive
one fsync at a time with nothing to merge and, unlike the datacow fsync
path, nothing kicks them before the page writeback wait (the sync park
deadline was most of a 3.7ms per-fsync regression against stock
nodatacow). Async writeback still parks there.
The ownership walk checks the extent item's size before reading its
inline backref: a data extent whose only backref is a keyed
EXTENT_DATA_REF item (the leaf had no room to grow the item) has none, and
counts as not owned.
Zygo Blaxell [Fri, 2 Oct 2026 06:41:38 +0000 (02:41 -0400)]
btrfs: stripe_alloc: pad a stripe's allocated sectors whose write is still to come (allow_rmw part)
The hunks of "btrfs: stripe_alloc: pad a stripe's allocated sectors
whose write is still to come" (earlier in this series) that need code
from "btrfs: stripe_alloc: isolate nodatacow and preallocated extents by
stripe", which comes after it in this series: they are applied here,
once "btrfs: stripe_alloc: isolate nodatacow and preallocated extents by
stripe" is in, instead of with the rest of "btrfs: stripe_alloc: pad a
stripe's allocated sectors whose write is still to come".
Zygo Blaxell [Thu, 1 Oct 2026 16:14:42 +0000 (12:14 -0400)]
btrfs: stripe_alloc: keep a stripe in the rbio cache until its run has written all of it (allow_rmw part)
The hunks of "btrfs: stripe_alloc: keep a stripe in the rbio cache until
its run has written all of it" (earlier in this series) that need code
from "btrfs: stripe_alloc: isolate nodatacow and preallocated extents by
stripe", which comes after it in this series: they are applied here,
once "btrfs: stripe_alloc: isolate nodatacow and preallocated extents by
stripe" is in, instead of with the rest of "btrfs: stripe_alloc: keep a
stripe in the rbio cache until its run has written all of it".
btrfs: stripe_alloc: count nocow run lifecycles and stranded stripes in sysfs (allow_rmw part)
The hunks of "btrfs: stripe_alloc: count nocow run lifecycles and
stranded stripes in sysfs" (earlier in this series) that need code from
"btrfs: stripe_alloc: isolate nodatacow and preallocated extents by
stripe", which comes after it in this series: they are applied here,
once "btrfs: stripe_alloc: isolate nodatacow and preallocated extents by
stripe" is in, instead of with the rest of "btrfs: stripe_alloc: count
nocow run lifecycles and stranded stripes in sysfs".
btrfs: stripe_alloc: isolate nodatacow and preallocated extents by stripe
Preallocated extents and nodatacow files' extents are candidates for
write-in-place, which reintroduces the raid56 write hole for every
stripe such a write touches: the RMW recomputes parity that also covers
whatever else shares the stripe. Before write-in-place can be
re-enabled for them (a later change; stripe_alloc still forces COW
today), their placement must guarantee the blast radius: such an extent
must never share a stripe with any other file's data.
Steer them into private per-inode stripe runs of a new NOCOW class,
reusing the log-active inode machinery: runs owned by one inode, never
in the shared band slots, found by owner-and-class lookup. Successive
allocations of the same file pack sequentially into the file's own
stripes; different files, and the datacow/relocation/log classes, never
share a stripe with them. A preallocation signals itself through a new
btrfs_reserve_extent() parameter; nodatacow files are recognized by the
inode flag. Placement remains best effort: when no fully-free stripes
are left for a private run the allocation falls back to the shared
runs, which is safe -- an extent that lands in a shared stripe simply
stays force-COWed when write-in-place arrives.
Zygo Blaxell [Sat, 10 Oct 2026 04:19:31 +0000 (00:19 -0400)]
btrfs: stripe_alloc: skip RAID5/6 groups that are mapped as RAID1/RAID1C3
Since commit 49ba3a3c0d67 ("btrfs: disguise single-data-RAID56 as
RAID1/RAID1C3"), a RAID5 chunk with one data stripe (two devices) or a
RAID6 chunk with one (three devices) keeps its RAID5/6 type on disk, in
the chunk map's on_disk_type and in the block group's flags, but is
mapped as RAID1/RAID1C3. Such chunks appear when only two or three
devices have unallocated space, e.g. on devices of unequal size.
stripe_alloc and stripe_meta decide by the block group's flags, so they
kept opening stripe runs in these groups. Their IO goes down the
mirrored path and never reaches raid56, so no write ever marked the
runs' sectors written: every ordered extent there ended with all its
sectors unwritten, and the groups paid stripe_alloc's per-allocation run
bookkeeping for a write hole they do not have. (Nothing was trapped there:
their full stripe is one sector, since btrfs_full_stripe_len() reads the
chunk map's type.)
Record at block group creation whether the chunk map's type differs from
its on_disk_type in this way (BLOCK_GROUP_FLAG_RAID56_MIRRORED), and
leave such groups to the normal allocator.
Their flags still match the RAID5/6 allocation profile, so writes land in
them, and with two of five raid6 devices missing every new data chunk is
one: the admission supply must include them, or every degraded write fails
with ENOSPC. Each space_info counts the free space of its writable
single-data-stripe RAID5/6 data groups, following ctl->free_space, which
every free space change goes through; btrfs_stripe_unmanaged_sync() runs
when a group joins its space_info and after each read-only transition, and
the read-only guard treats a counted group's free space as the supply it
takes away. Groups of other profiles are not counted: a RAID5/6 write
cannot use them. The full-stripe widths the admission charge is taken
over leave these groups out, as their full stripe is one sector.
Zygo Blaxell [Sun, 4 Oct 2026 16:26:48 +0000 (12:26 -0400)]
btrfs: stripe_alloc: count refused rbio pins and track rbio cache high water
The rbio cache keeps a stripe its run has not finished writing pinned, at
most RBIO_PIN_MAX (256) of them; the next pin is refused and that stripe
is cached evictable. If it is evicted before its run's next write into
it, that write reads back the sectors already written, unverified: the
case the pin exists to avoid. Nothing counted the refusals, so whether
the limit is ever reached was unknown.
stripe_park_stats gains:
rbio_pin_refused pins refused at RBIO_PIN_MAX (every attempt)
rbio_pinned_now pinned rbios now (nr_pinned: what the limit counts)
rbio_pinned_live of those, pins still needed: re-checked against the
runs at read time, as the prune does. A pin whose run
has finished the stripe stays counted until its entry
is evicted or cached again.
rbio_pinned_hwm high water of pinned rbios since mount
rbio_cache_now cached rbios now
rbio_cache_hwm high water of cached rbios since mount (above
RBIO_CACHE_SIZE only by live pins, which the prune
keeps)
meta_rmw_reads read-modify-writes of metadata/system stripes that had
to read (data_rmw counts these with the data ones).
The pin is what keeps a stripe of fsync log blocks
cached for the next log commit; a refused pin can bring
that read back.
The high water marks are only raised under the cache lock. Reading the
file walks the cache once under the cache lock (1024 entries at most).
nr_pinned counts a pin until its entry is evicted or cached again, also
after its run has finished the stripe, so the limit filled with stale pins
within seconds under any fsync-heavy load (13k-87k refusals a minute)
while buffered writeback needed 3-14. Two things release pins:
- A sweep, rbio_unpin_stale(), re-checks every pin against its run and
clears the stale ones under the cache lock. It runs when a pin is about
to be taken with RBIO_PIN_LOW (3/4 of the limit) or more counted, at
most once per RBIO_PIN_SWEEP_INTERVAL from that path, and at every
commit right after btrfs_retire_open_stripes(), which ends every run.
- If the pins left after a sweep (all live) are still RBIO_PIN_LOW or
more, the transaction kthread is asked to commit now, once per
transaction. After the commit a stripe's earlier writes are no longer
at risk from a read-back: data checksums are committed and the log
blocks are dead.
stripe_park_stats also has rbio_pin_swept and rbio_pin_commit_requests.
Zygo Blaxell [Fri, 2 Oct 2026 06:41:38 +0000 (02:41 -0400)]
btrfs: stripe_alloc: pad a stripe's allocated sectors whose write is still to come
A stripe run's first partial write into a stripe cannot pad today: the
stripe's other sectors are allocated (live) to writes that are still in
flight, so the pad oracle calls them live and the write goes down as a
read-modify-write of the whole stripe. With the stripe pin keeping the
stripe cached for every later write, this first write is the only data RMW
left in a stripe run (traced on the strip-fault fixture: every remaining
data RMW read was a first write), and the read it does is of sectors
nobody has written: it can only fetch garbage into the parity, and on a
device that returns wrong data it makes that parity disagree with what is
on the disk.
Use the run's written map: a live sector whose write has not completed
holds nothing anyone has written, so it is as paddable as a dead one.
Zeroing it is safe because its write is ordered after this one -- the
padding write holds the stripe lock, every raid56 write to the stripe
takes it, and rbio_orig_end_io() records a write's sectors in the written
map before it unlocks the stripe -- and nothing reads the sector before
its own write lands, since its extent is not committed until then. That
write replaces the zeros; it is a sub-stripe write into a stripe written
before, which the stripe pin serves from the cache.
A live sector that HAS been written still refuses the pad, as before:
that is now the precise meaning of the UNCOVERED decline. Nodatacow runs
keep no written map (their extents are rewritten in place for as long as
the inode lives) and are excluded. padded_unwritten in stripe_park_stats
counts the pads that zeroed such sectors.
Zygo Blaxell [Thu, 1 Oct 2026 16:14:42 +0000 (12:14 -0400)]
btrfs: stripe_alloc: keep a stripe in the rbio cache until its run has written all of it
A stripe run hands out its sectors in order, so a stripe it allocates is
often written in pieces: a write that ends inside the stripe leaves the
rest to later writes, and writeback allocates a whole range before any of
its bios complete, so even a stripe the frontier has already passed can
still have writes in flight. The later writes cannot pad (the earlier
sectors are live), so unless the stripe is still in the rbio cache each
is a read-modify-write that reads the earlier sectors back from the
devices. Those sectors are not yet covered by the csum tree (their
ordered extents have not finished), so what the devices return goes into
the new parity unverified. A device that returns wrong data there
destroys the earlier write, whose only intact copy at that point may be
the parity: the strip-fault fixture (a device that returns random bytes
for one strip per stripe) lost a file exactly this way, with every RMW of
the run in this same-run class.
The cache already prevents that read when the stripe is still cached: the
later write steals the earlier write's pages (lock_stripe_add()) and reads
nothing. But the cache is an LRU that any other stripe's write can push
such a stripe out of. Pin it instead. Each run gets a written map beside
its live map, set from each completed raid56 write's covered data sectors
(rbio_orig_end_io(), before the cache decision in unlock_stripe()).
cache_rbio() pins an rbio whose stripe is still waiting for writes
(btrfs_stripe_run_incomplete()): an open run's allocation frontier lies
inside it -- the rest is not even allocated yet, so every sector
allocated so far may already be written -- or a run covering it, open,
closed or kept for relocation, has live sectors there not yet written.
The prune skips live pins and re-checks each against the runs, so a pin
ends as soon as the stripe is complete or its run goes away. Nodatacow
runs are excluded from the written check: they are rewritten in place
for as long as the inode lives and are never complete. RBIO_PIN_MAX
bounds the pinned entries; no memory is allocated for this, the stripe's
pages are only released later. A completed write is marked in the run
that owns it, by [run->start, run->end): the maps reach past run->end
(they span at least BTRFS_STRIPE_RUN_LIVE_STRIPES stripes), so a write
must never be recorded in an earlier run whose map merely overlaps it.
written_not_live in stripe_park_stats counts completed-write sectors that
are not live in their run, which should stay near zero.
This removes the read rather than adding a recovery step for it: nothing
here depends on checksums, sectors without them, or the reconstruction
path. What remains is a stripe's first partial write, which reads only
sectors nobody has written yet (traced on the fixture: every remaining
data RMW read was of that kind). rbio_pinned in stripe_park_stats counts
the pins.
An ordered extent can end without writing all of its sectors (a write
error, truncation of folios never submitted, a short direct write, teardown
of an extent whose IO never started). When it reports to its run, its
whole range is marked written: no write of those sectors is still coming,
which is all the pin (and any other waiter on the written map) needs to
know. Marking them written rather than dead keeps them from being padded,
so nothing a truncated extent kept is zeroed. Waiters are woken while the
run lock is held, since the run and its group can be freed once it is
dropped.
Zygo Blaxell [Thu, 1 Oct 2026 04:57:48 +0000 (00:57 -0400)]
btrfs: stripe_alloc: keep relocation's stripe runs until relocation ends
Relocation preallocates each destination extent and writes the data into
it later, after the commit that ends relocate_block_group(). Under
stripe_alloc the preallocation lands in a BTRFS_STRIPE_RUN_RELOC run,
which drains at once (a preallocation has no IO to report) and is freed
when that commit closes it. By the time the relocated data is written,
no run covers its stripes: a write that does not cover a whole stripe
can neither park to merge with its neighbour nor pad, and goes down as a
read-modify-write. Relocated extents rarely line up with stripes, so
that is roughly one RMW per relocated extent: an audited data balance on
a 4-device raid5 made 2247 of them, every one NO_RUN, every one written
by the data reloc inode into its own preallocation, and none over
another owner's committed data (the stripes hold only relocation's own
orphaned copies) -- harmless, but every one a stripe read.
Keep RELOC runs after they close and drain, as metadata runs already are
for the same reason, until relocation releases them from
btrfs_release_data_reloc_bg(), which it calls on its way out (after the
pointer update, before the next relocation can allocate). A retained
run that has nothing in flight is not waited for by the commit's drain,
and any forced quiesce of its block group (read-only, removal, unmount)
releases it first. While retained, its range stays unclaimable and its
stripes are relocation's alone, so the writes into them park and merge
or pad like any other run's.
reloc_run_retained / reloc_run_released in stripe_run_stats count the
runs kept and released.
The open stripe self-test frees the runs it leaves retained, as unmount
does.
btrfs: stripe_alloc: drop the mount options, the property is the interface
stripe_alloc and stripe_alloc_allow_rmw were mount options and filesystem
properties both. Upstream will not take new mount options for this, and two
interfaces to one policy have to be kept in step by hand, each new check
added to both.
Remove the options. Nothing is lost:
- btrfs_stripe_alloc_check_support(), which the property calls, already
refuses everything btrfs_check_mountopts() refused -- v1 space cache,
zoned, mixed block groups.
- The runtime state is untouched. BTRFS_MOUNT_STRIPE_ALLOC stays;
btrfs_enable_stripe_alloc() and the property already own it, and every
btrfs_test_opt(fs_info, STRIPE_ALLOC) in the allocator is unchanged.
- btrfs_parse_stripe_rmw() and btrfs_show_stripe_rmw() keep their other
callers in props.c and block-group.c.
The mount-time report collapses to one source, so it no longer has to say
which of two things turned the policy on, and btrfs_fill_super() no longer has
to sample the flag before open_ctree() to tell them apart. The remount path
loses its report with the option: a remount can no longer change the policy,
and the property announces its own changes.
Enabling the policy is now: mount, then
setfattr -n btrfs.stripe_alloc -v 1 /mnt
on the top-level root directory, which persists across mounts.
The warning about the raid56 groups stripe_alloc does not cover moves from
btrfs_read_block_groups() to btrfs_enable_stripe_alloc(): block groups are
read before the root directory's properties apply, so with the option gone
the flag was always clear at the old call site and the warning could never
fire (bg-gate-test.sh caught it).
fs_info->stripe_rmw_opt, which only the stripe_alloc_allow_rmw mount option
set, goes with it: btrfs_stripe_allow_rmw() is the property alone.
btrfs: stripe_alloc: say at mount whether the policy is on
Two paths turn stripe-exclusive allocation on and neither announced it.
The mount option sets the flag while btrfs_emit_options() has no entry to
print for it, and the btrfs.stripe_alloc property applies later still,
when btrfs_fill_super() reads the root directory's inode and its
properties. btrfs_enable_stripe_alloc() did print a line, but only on
the property path, and even there it is skipped when the option already
set the flag, because the function returns early in that case.
So the log said nothing. An absent line meant "off" and "on via the
mount option" equally well, and the only way to answer the question was
to read /proc/mounts on a live filesystem -- no use when the filesystem
is someone else's, or when all that is left is a log from before a crash.
Analysing a lockup from a filesystem whose btrfs.stripe_alloc property
was set, I read that silence as the policy being off and said so, which
was wrong twice over: the property was set, and the option would have
been just as invisible.
State it once in btrfs_fill_super(), after the root inode has been read
so the property has had its say, naming which of the two turned it on.
The option is sampled before open_ctree(), not merely before that read:
the mount context has already reached fs_info by the time
btrfs_fill_super() runs, and open_ctree() itself goes far enough into
the mount to read the root directory's inode and apply the property, so
a sample taken after it reports every property-enabled filesystem as an
option-enabled one. The directed test caught exactly that. A remount
that changes the flag reports the change as well. The property path
keeps its own line for a set on a live filesystem, where there is no
mount to report. At mount that line still comes too, so a mount with
the property set logs the state twice, until "btrfs: stripe_alloc: say
at mount whether the policy is on (debug part)" later in this series
quiets it there: it needs the runtime enable worker of the write-hole
checker.
Nothing is printed when the policy is off, which keeps the common mount
quiet and makes the absence of the line mean exactly one thing.
btrfs: stripe_alloc: count nocow run lifecycles and stranded stripes in sysfs
Add a stripe_run_stats file beside stripe_park_stats with lifetime
counters for the paths the previous two patches touched, so a test can
show that it exercised them rather than merely passed: private NOCOW
runs opened, claim candidates refused for overlapping a live run, bytes
of whole stripes the scans found stranded behind live runs, and rescans
requested by a freed run to credit its stripes back.
A persistent nocow run whose allocations have all been freed is dead: it
owns no data and only strands the whole stripes in its range (99% of the
free whole stripes of every read-write group sat inside such runs after an
fsstress fallocate load). btrfs_stripe_run_note_free(), from
btrfs_add_free_space(), clears the live bits of freed sectors inside a
persistent run, and close_block_group_stripe_runs() closes a persistent
run that handed out space, has nothing in flight and no live bit set, on
the normal commit path that frees it and marks its group for the rescan.
A run with any live byte stays open, so a slowly appended nocow file still
burns no stripe per commit; a run whose map was retired is never taken as
dead. The free path only clears bits under bg->stripe_run_lock.
nocow_run_dead_closed counts the closes.
btrfs: stripe_alloc: count whole stripes stranded behind live runs as trapped
The data admission gate admits a reservation while the whole-stripe
supply, bytes_stripe_claimable, covers it, and the per-commit scan
measures that supply as the block group's wholly free full stripes.
But a wholly free stripe inside a live stripe run's range cannot be
claimed: btrfs_stripe_run_range_usable() rejects any claim overlapping
a run object, because a second run over the same range would make the
range-to-run lookups ambiguous. The run was claimed over the stripe,
the extents allocated there were freed again, and the run has not been
freed yet. For a COW run that window closes at the next commit; a
nodatacow inode's private run deliberately survives commits and lives
until the inode is evicted.
On a raid56 filesystem filled to ENOSPC by fsstress (thousands of
fallocated files, so thousands of cached inodes each holding a private
NOCOW run) a live walk of the free space cache showed the accounting
exact -- the wholly free stripes summed to bytes_stripe_claimable to the
byte -- and 99% of them inside the ranges of open NOCOW runs with no
inflight IO: 22.8 MiB of 22.8 MiB behind 387 runs in one group, 73.4 MiB
of 73.4 MiB behind 976 runs in another. Every claim failed in O(1) on
the "no claimable run" verdict, every write() admitted against the 99 MiB
of phantom supply failed at writeback, and the data was dropped ("data
writeback allocation ... returned ENOSPC despite reservation margin"),
about 100,000 delalloc ranges per run of the degraded acceptance suite
where the stock allocator drops none.
Have the scan take such stripes out of the claimable supply: count them
in stripe_unusable, which btrfs_space_info_used() already charges so the
gate cannot admit against them, and record them separately as
stripe_stranded so that btrfs_stripe_bg_wants_reclaim() does not ask
relocation to recover space that comes back by itself. When a run is
freed, mark its group for rescan so the next commit credits the stripes
back. The space_info total is exposed as bytes_stripe_stranded in sysfs
and in the space info dump.
This makes the gate honest -- write() gets the ENOSPC instead of
writeback dropping the data -- but does not return the stranded space
while the runs live; that is the run lifecycle's business.
The space_info totals are kept equal to the sum of the armed groups:
claim, arm, disarm and rescan update a group's value and the total
together under sinfo->lock, and the commit scan sums the groups under
sinfo->lock while it holds groups_sem. Reading the groups unlocked and
assigning the totals later lost or doubled updates made in between, and a
group made read-only could have its whole claimable supply counted again
until the next commit, admitting writes that writeback could not place.
The scan does not restore the stripe_meta margin and reserve when
stripe_alloc was disabled after it started.
btrfs: stripe_alloc: show per-group trapped bytes in the space info dump
btrfs_dump_space_info() prints each block group's available bytes, and
under stripe_alloc those include the free space trapped in partly used
stripes, which no allocation can use until the group is reclaimed
(stripe_unusable). An ENOSPC dump then shows space in the groups that
cannot be had.
Print stripe_unusable next to zone_unusable and leave it out of the
available bytes. A group whose trapped bytes have not been counted yet
(stripe_unusable_ready not set) prints 0 and the old available figure.
Zygo Blaxell [Fri, 14 Aug 2026 01:12:02 +0000 (21:12 -0400)]
btrfs: stripe_alloc: gate data admission on claimable whole-stripe supply
Wire stripe_claimable into the DATA reservation path so write() returns -ENOSPC
when there is no fully-free whole stripe to place the write crash-safely --
matching statfs f_bavail, which already subtracts the trapped partial-stripe
space. Add stripe_claimable_admit() and AND it into the two data-admit sites:
__reserve_bytes() and btrfs_try_granting_tickets() (the latter is required, or a
flushed data ticket would be granted by the plain used<=total rule, bypassing the
gate). Scoped to raid56 stripe_alloc DATA (stripe_margin_unit != 0); metadata,
non-raid56 and zoned space_infos are unaffected.
The gate refuses at reservation time, before the range becomes delalloc and later
parks at writeback with nowhere to land -- so ENOSPC is clean and there is no
fill-edge parking collapse.
Phase 1 is pessimistic: it gates on bytes_stripe_claimable alone, which errs LOW,
so it can refuse a write that would have landed in a partially-open stripe
(over-refuses by the open remainder, healed by the commit rescan raising claimable
and the FLUSH_DATA ticket retry). A later change adds bytes_stripe_open to the
bound for statfs-exact behavior.
What the gate charges, and what it holds against:
- A data reservation is charged need(L): the largest roundup(L, f) over
the full-stripe widths f of the filesystem's raid56 data groups, with L
its data bytes without the per-extent probe margin. The margin is
collateral that stays accounted (in bytes_may_use as the probe, then in
bytes_stripe_margin) until the extent is written, so later admissions
already see it; charging it inside the roundup would ask the supply for
it twice. Groups narrow at the fill edge, which the max over widths
covers. fs_info records the distinct widths next to stripe_margin_unit
and never forgets one; with more widths than it holds, the charge falls
back to L + unit - sectorsize. The probe margin reaches the gate through
btrfs_reserve_data_bytes(), __reserve_bytes() and the ticket. Rounding is
by division (stripe_round_up(), div64_u64()): the widths are not powers
of two, and round_up() would charge 512K for 384K of 192K stripes.
- The held margin, bytes_stripe_margin, is added to the admitted bytes:
it is owed to writers already admitted, and each commit shrinks the
supply by the tails it traps.
- The delalloc probe margin is not released until the metadata
reservation has charged it into bytes_stripe_margin (buffered write,
page_mkwrite, block truncation, direct IO, log-tail carry,
btrfs_delalloc_reserve_space()), so concurrent writers cannot be
admitted into each other's released margins. The v1 cache write-out,
which reserves no metadata, releases it at once.
- A preallocation has no outstanding extent, so it holds one stripe in
bytes_stripe_margin from the reservation until its allocations land.
The hold is charged inside __reserve_bytes(), in the section that grants
the bytes (directly or by btrfs_try_granting_tickets()), and handed back
through btrfs_alloc_data_chunk_ondemand_held() /
btrfs_reserve_data_bytes_held() to fallocate, zero-range, relocation's
cluster preallocation and the encoded write.
- Relocation is not charged the per-extent margin for extents already
allocated; its reservation is charged roundup(L, unit) plus one stripe,
what its own open runs can lose at the next commit. It has its own
flush type, BTRFS_RESERVE_FLUSH_DATA_RELOC, under the same bound but
never waiting: a relocation waiting on the gate held the exclusive
operation and the group it was emptying while the gate waited for the
stripes it was to free. Since a reservation that cannot ticket never
reaches the flush states that allocate data chunks, a refused relocation
reservation forces data chunk allocations while they succeed and
retries, and fails only when that does not help.
- The group dedicated to relocation (fs_info->data_reloc_bg) serves only
relocation, so its claimable bytes are tracked as
bytes_stripe_claimable_reloc and ordinary reservations are admitted
against the supply without them. Relocation is admitted against the
whole supply.
- inc_block_group_ro() refuses the read-only transition while the
admitted bytes would no longer fit in the claimable supply without the
group; scrub, balance and reclaim retry later. The subtraction
saturates, since the group's claimable value can be read ahead of the
space_info total.
- The reclaim worker allocates a chunk before it relocates a stripe group
whose live bytes exceed the claimable supply of the other groups (the
reservation path does not: a live device shrink must fail cleanly at
its reservation rather than grow the data space_info).
Zygo Blaxell [Tue, 11 Aug 2026 06:53:38 +0000 (02:53 -0400)]
btrfs: raid56: adaptive park deadline from the parked-rbio backlog
Full-stripe batching parks a sub-stripe write's rbio for a deadline so the
writes that fill the rest of its stripe can merge into it, turning a
read-modify-write into one full-stripe write. The deadline is a fixed
per-filesystem value. It is long enough to catch those merges, but at a
small-file fill to ENOSPC -- where a run's stripes never fill because each
file's neighbours land in the next file's stripe rather than this one --
every partial write waits the deadline out, and then its 10x stuck cap, on
an arrival that never comes before padding to a full stripe and going down.
Lowering the deadline globally fixes that fill but throws away the merges
the deadline exists to catch on ordinary moderate writes.
Feed the deadline back from the parking machinery's own backlog instead of
guessing at the workload. Track stripe_parked_now, the number of
currently-parked rbios, incremented as a partial write parks and
decremented as it unparks. When it runs ahead of the stripe_park_congestion
knob (0 disables it, the default), a new async partial write takes the short
sync deadline through the existing sync-park path rather than the full one.
This is negative feedback: the shorter deadline drains the backlog, so once
it falls back below the knob later writes regain the full deadline and full-
stripe batching. No space-state estimate is needed -- a small-file flood
builds the backlog directly, while a large sequential fill (already full
stripes, never parked) and a moderate steady stream do not.
Measured on an eight-device raid5. A small-file balance-reclaim soak that
times out at the default deadline completes at a congestion of 4; on a
throttled moderate stream, where the backlog stays low, a congestion of 32
never trips, keeps the full deadline, and merges three times as much with
38% fewer RMW reads as the same filesystem under a flat 3ms deadline. The
gauge and a congestion_short counter join the stripe_park_stats sysfs file
so the knob can be set from measurement. Off by default, and no data-path
change: a shortened park still pads or reads exactly as it would have, only
sooner.
Data reservations are admitted against arithmetic -- free space minus
trapped fragments minus open-run remainders minus a per-extent margin --
and under sustained near-full churn the arithmetic and the claim rule
disagree for long enough that admitted buffered writes reach writeback
with nothing claimable left: measured, ~5.2-5.5k reserved writebacks
dropped per fsstress churn run, silently for anything not waiting on
fsync. Stock refuses the same write()s up front. Nothing between
"durable by fsync" and "refused by write()" is acceptable when an
operator accident fills the disk.
First step, accounting only: measure the constraint instead of deriving
it. bg->stripe_claimable counts bytes of fully free whole stripes --
exactly what btrfs_claim_free_stripe_run() can take, the complement of
stripe_unusable within each stripe. The commit rescan derives it from
the same pass that computes trapped bytes (free minus trapped) and is
the sole upward correction; incremental maintenance under
ctl->tree_lock only ever DEBITS -- removals round their decrement OUT
to every touched stripe, claims subtract exactly what they took. The
counter can therefore only under-count between commits, never
over-count.
Crediting freed whole stripes incrementally on the add side is
deliberately not done, having been tried and dropped. Several add
paths re-add free space that a low-level remove never de-credited --
the async discard trim (unlink_free_space / bitmap_clear_bits, then
do_trimming's re-add through __btrfs_add_free_space) and
btrfs_remove_free_space's middle-split tail re-add -- so crediting on
add double-counts and drives the counter ABOVE the authoritative scan.
That is the dangerous direction: once admission gates on this counter,
an over-count admits reservations against phantom supply that writeback
then drops, while an under-count is an early, clean write()-time
ENOSPC. Debit-only makes over-counting structurally impossible,
whatever a future re-add path forgets to de-credit. (Reproduced with
compress+autodefrag+discard=async on a legacy-converted raid5
filesystem, incremental ~= 2x scanned; credit_return attribution
confirmed the phantom entered through do_trimming's re-add, not the
open-stripe allocator returns.)
A WARN_RATELIMIT at the rescan catches the dangerous direction anyway
(incremental above scanned), which under a debit-only rule means a
missed consumption site, and localizes it; the clamp to the scanned
value keeps every rescan authoritative regardless. Rate limited rather
than _ONCE because such a gap recurs on every commit, and reporting
only the first hit hides that it is ongoing.
The space_info aggregate bytes_stripe_claimable follows the
stripe_unusable pattern (incremental between commits, re-totaled from
armed groups at the rescan), shows in the ENOSPC dump and in sysfs.
No admission change yet; that comes once the counter proves accurate
under the fill, churn, balance and reclaim suites. The by-size
fast-fail in the claim keeps its early exit: the point of measuring is
to make such inputs trustworthy, not to search harder around them.
Data block groups only for now; raid56 metadata and mixed block groups
join when the data counter has settled.
A trim takes a free range out of the tree, drops tree_lock for the
discard and adds the range back, and a rescan in that window would find
fewer claimable stripes than the incremental count (seen as "incremental
stripe_claimable ... above scanned" with discard=async). Each step
changes exactly the stripes the range belongs to, so the unlink debits
every one of them that was claimable (wholly free, not inside a live run:
the scan's rule) and the re-add credits every one that is claimable now,
in the tree_lock section that unlinks or links the range. Rounding
instead would over-debit a stripe per trimmed sub-stripe hole, and async
discard trims thousands after a delete. The wholly-free test falls back
to the bitmap lookup at offset_to_bitmap() as the other callers do, since
an extent entry starting inside a bitmap's window hides the bitmap from a
non-fuzzy search.
The nocow tail claim, btrfs_claim_stripe_tail(), now counts and removes
the tail in one ctl->tree_lock section, with claim_stripe_run_pieces()
doing the removal as for any claimed run, and fails with -ENOSPC unless
the whole tail is in the cache, debiting the
claimable count for what it removes in the same lock section. It counted under the lock and then
called btrfs_remove_free_space(), which returns 0 when the range is not
in the cache at all: two allocations for the same inode (two direct
writers, writeback racing fallocate) or a trim holding an entry out could
both "remove" the same tail and hand out the same extent. A self-test
claims the same tail twice.
Zygo Blaxell [Sun, 9 Aug 2026 03:14:23 +0000 (23:14 -0400)]
btrfs: stripe_alloc: track per-sector liveness and pad from the map
Padding used to ask where the run's allocation frontier was, which
cannot see an allocation abandoned below it -- a reservation released, an
allocation the finder discarded, an extent freed before its write was
ever issued. Runs now carry a bit per sector, set at allocation and
cleared by the new btrfs_open_stripe_write_abandoned() report, and
padding fills exactly the sectors that are dead: this replaces
btrfs_stripe_run_pad_start() with btrfs_stripe_run_pad_mask(). The map
is sized once, with headroom for frontier growth, because the growth
path holds the run lock where nothing may sleep; a run that would
outgrow its map stops growing instead. A closed run's final stripe
overhangs its shrunk end by construction, and those sectors -- the
returned tail, unclaimable while the stripe holds live data -- are dead
and paddable too.
The abandonment report is what closes a writeback ENOSPC race the
fsync-heavy near-full workload hits: an allocation undone without a
report left phantom inflight bytes and phantom-live sectors, and
reservations admitted against that state failed at writeback
(cow_file_range -28) where stock refuses the write() up front. With
the report in place the same workload shows zero writeback failures at
every reservation margin setting, including margin disabled.
Claiming also learns an O(1) fast fail: the by-size index walk is
extracted into find_free_stripe_run(), stopping at the first entry whose
largest contiguous free run cannot hold a full stripe -- near-full, where
free space degenerates into many sub-stripe holes, this replaces a scan
of the whole tree on every allocation.
Moved out of the stripe_meta patch, which needs all of this for tree
blocks but introduced it tangled with the metadata machinery; data wants
it on its own.
A parked rbio does not wait for sectors whose write will never come: a
sector the liveness map has as dead is not counted as missing.
The O(1) fast fail extends to the failed-scan bound: claims stop at once
in a group whose last exhaustive scan proved there is no claimable run,
until free space is added.
With padding decided by the liveness map, the LIVE and UNALLOCATED pad
refusal reasons, which only the frontier oracle produced, go; padding now
declines only for NO_RUN and UNCOVERED.
Zygo Blaxell [Sat, 8 Aug 2026 15:27:36 +0000 (11:27 -0400)]
btrfs: stripe_alloc: complete parked writes on allocation coverage, not clocks
A parked partial-stripe write goes down its RMW path when its deadline
expires or a retirement flush kicks it, even when every byte it is
missing below the run's frontier belongs to an allocation whose data IO
is already in flight -- allocation happens at writeback submission, and
the commit's retirement drain waits for exactly those arrivals. Timing
out such a park buys nothing and costs a stripe read plus a second write
of the same stripe: measured on an 8-device raid5 buffered fill, the
100ms deadline turns ~2200 parked stripes per 2GB into read-modify-
writes (pad_decline_live), and stretching the clock 20x recovers only
14% -- the clock is the wrong instrument.
Replace the clock with a coverage test. A parked rbio is ready when its
gathered bios cover everything the covering run has allocated inside its
stripe (btrfs_stripe_run_alloc_ceiling); the remainder lies at or past
the frontier, so the existing pad turns it into a single full-stripe
write with no read phase. Merges check readiness as they land, the park
timer holds unready parks instead of expiring them, and run-retirement
flushes leave unready parks parked: the retirement has already closed
the run (freezing the allocated prefix) and its drain waits on the very
arrivals that will complete them. Waiter-driven flushes (fsync's
pre-writeback kick, ordered-extent waits) and sync parks keep today's
forced behaviour: a blocked waiter's latency beats a saved stripe read.
A stuck cap (10x the park deadline) bounds the wait when an arrival can
never come: a writeback error abandoned the allocation, or the stripe
was already written once by a forced sync park. Such parks are forced
down the old path and counted.
New sysfs stripe_park_stats counters: unparked_ready (parks completed by
the coverage test) and stuck (parks forced at the cap).
Zygo Blaxell [Sat, 8 Aug 2026 07:57:22 +0000 (03:57 -0400)]
btrfs: stripe_alloc: trace why padding declined, at the moment it declined
The counters say how often padding refused and broadly why, but not what
the run looked like when it happened, and reconstructing that from the
extent tree afterwards cannot distinguish "the stripe was already
allocated when this write arrived" from "it was allocated shortly after".
Those want different fixes, so record the decision where it is made.
Also count the second refusal, which had no counter at all: the oracle can
allow padding and the sector walk still refuse, because a sector below the
frontier is not covered by this write. That is the case where another
allocation shares the stripe and its data is not in this rbio -- a
different thing from the oracle finding the whole stripe allocated.
The tracepoint carries the stripe, how much of it this write covers, the
refusal reason, and the run's start/end/frontier/inflight, so a trace says
directly whether the frontier had already run past the stripe when the
write showed up.
Zygo Blaxell [Sat, 8 Aug 2026 06:03:40 +0000 (02:03 -0400)]
btrfs: stripe_alloc: tunable park deadlines, and say why padding declined
The deadline constants are meant to be judged "from measurements rather
than taste", but they are compile-time, so every data point costs a kernel
build and a reboot. Measuring a buffered fill -- dd, no fsync, so nothing
kicks the park and the backstop is the only exit -- found 1861 of 3984
parks expiring at the 100ms deadline, which is precisely the no-waiter case
the constant exists for and the one with no numbers behind it.
Expose both deadlines as writable sysfs files, clamped to 60s because a
parked rbio holds its stripe lock, with 0 disabling parking for that class
(useful as an experiment in itself):
Add the counters the stats file cannot supply. rmw_reads counts only
parked writes that still had to read, so it cannot answer "does this
workload read-modify-write at all"; data_rmw counts every data RMW, which
with allow_rmw empty on a covered block group is the number the
stripe-exclusive claim is about, and it should be zero.
And record why padding refused, because the reasons are different defects:
the rest of the stripe is allocated and its data has not arrived yet
(wait longer, or kick when the frontier advances); no open run covers the
stripe, so the run closed before its own write went down; or the frontier
never reached the stripe at all. Attributing them turns "some RMW
remains" into a specific thing to fix.
Zygo Blaxell [Tue, 4 Aug 2026 19:51:54 +0000 (15:51 -0400)]
btrfs: stripe_alloc: report read-modify-write of uncovered stripes
The mount-time warning says which block groups stripe_alloc does not
cover. It cannot say whether anything is actually landing there, and on
a filesystem with raid56 metadata that is the interesting question: every
sub-stripe write to an uncovered stripe is a write hole window, where
parity and data reach the disk separately and a crash in between leaves
the stripe unreconstructible.
Report it from the one place that knows the write is going out as a
read-modify-write rather than as a full or padded stripe. Rate limited,
because a raid56-metadata filesystem does this continuously and the point
is to make the exposure visible rather than to fill the log. (A
meta_rmw counter in stripe_park_stats, to read the rate off without
grepping dmesg, is debug code and is added in the debug subtopic.)
Silent when stripe_alloc is off: there the whole filesystem works this
way and the user has asked for nothing else.
Zygo Blaxell [Tue, 4 Aug 2026 06:43:20 +0000 (02:43 -0400)]
btrfs: stripe_alloc: warn at mount when raid56 metadata is not covered
stripe_alloc closes the write hole for raid56 data. It does nothing for
raid56 metadata, which keeps read-modify-write and keeps the hole, and
on a mixed-block-group filesystem it now does nothing at all. A user
who mounts -o stripe_alloc on -d raid5 -m raid5 has every reason to
believe the filesystem is covered, and finds out otherwise only after a
crash on a degraded array -- with metadata damage, which is worse than
the data damage they were protecting against.
Say it once at mount, after the block groups are read, so the message
reflects what is actually on disk rather than what was asked for. A
warning, not an error: the data guarantee is real and worth having on
its own, and a filesystem can be converted to raid1c3/raid1c4 metadata
without unmounting.
Zygo Blaxell [Fri, 7 Aug 2026 01:35:56 +0000 (21:35 -0400)]
btrfs: stripe_alloc: claim fully-free stripes that span free space entries
The claim fast path searches single entries, so a fully-free full
stripe whose free space spans an entry boundary -- an extent entry
adjoining a bitmap, or two neighbouring bitmap windows -- was
unclaimable. stripe_unusable accounting is entry-blind and counts
exactly those stripes as claimable, so admission reserves data against
them; at writeback the claim finds nothing, cow_file_range() gets
-ENOSPC, and the already-dirtied pages are dropped. full_stripe_len
is not a power of two, so stripe boundaries drift through the fixed
128M bitmap windows and a straddling stripe is a certainty near full,
not a corner case.
Observed live at the raid6 fill edge (rolling-failure, then isolated
by fill-edge-debug with enospc_debug): two 384K stripes, each
straddling a bitmap window boundary, held the space_info accounting
786432 bytes above what the claim could reach, and every writeback
allocation against that phantom margin failed -ENOSPC while ~991MB of
genuinely trapped free space sat in the cache. The reservation margin
cannot absorb this: the gap is per straddling stripe, not per
outstanding extent.
Add an entry-blind slow path: one offset-ordered walk accumulating
contiguous free coverage across entry boundaries, and a piecewise
removal that runs in the same tree_lock critical section as the find,
so a racing claimer cannot see a half-removed run. The fast path is
unchanged and still serves the common case; the slow path runs only
after it fails, which is the near-full case where a stranded stripe
matters most.
A bitmap piece of a split run is taken only to the end of the set bits
that start at it, so that an extent entry starting inside the same bitmap
window is found and removed as the next piece. A self-test covers that
layout.
On a large aged filesystem (85 TiB, mostly legacy data, heavily fragmented
free space) the slow path walked the whole group for every small
allocation and found nothing, at 100% CPU in the delalloc workers. So a
failed exhaustive scan is remembered: bg->max_claimable_run is an upper
bound learned from the scans ((u64)-1 unknown, 0 after a scan proved no
claimable run), claims fail in O(1) while it is below one full stripe or
the group's free space is, and adding free space (frees, unpins, stripe
run returns, and a cluster returning its entries) resets it. It is only
set and trusted once the group is fully cached, since cache loading links
entries without the reset.
When a claim splits one extent entry, the entry keeps the tail and the
head is linked back with a spare entry allocated before ctl->tree_lock is
taken, so a failed allocation cannot lose it from the cache; without a
spare it is added after the lock as before.
Zygo Blaxell [Mon, 3 Aug 2026 07:22:52 +0000 (03:22 -0400)]
btrfs: stripe_alloc: pessimistic data reservation margin
A byte-counted data reservation holds no collateral against the stripe
claim rule. Between admission at write() time and the allocation at
writeback, the trapped-space picture keeps moving: commits close open
stripe runs and trap their sub-stripe tails, and free space returning
mid-transaction (unpinned deletions, same-transaction frees, drained
run tails, freed reservations) lands in the free space cache as
fragments the claim rule can never hand out but that no counter yet
reflects. The counters were honest at every instant for NEW
admissions while OLD admissions were left holding air; delalloc
writeback gets no second chance, so the pages were dropped. Measured
at the fill edge: ~19 MiB of orphaned bytes_may_use with accounting
and allocator in perfect agreement, and in a later round the admitted
may_use+margin exactly equaled the phantom mid-transaction fragments.
Be pessimistic at reservation time and optimistic at allocation time,
the same shape metadata reservations already use:
- bytes_stripe_margin: one full stripe width (fs_info->
stripe_margin_unit, the widest raid56 data full stripe) per
outstanding delalloc extent, charged and released inside
btrfs_mod_outstanding_extents() so it stays exact across delalloc
merges and splits -- including async compression, where the io-tree
split hook grows the margin to one stripe per compressed piece,
matching the true worst case. Counted in btrfs_space_info_used()
so it holds back admissions. The margin exists only for in-flight
dirty data; steady-state capacity and statfs are untouched.
- Admission probes len + margin through the ticketed FLUSH_DATA path
(then immediately re-releases the probe, which the delalloc hooks
re-charge), so a writer that cannot be covered waits for the
flusher -- commits convert pinned deletions into claimable whole
stripes -- and receives an honest ENOSPC at write(2) if flushing
cannot produce cover.
- Every btrfs_add_free_space() into an armed group counts the added
range's sub-stripe head and tail fragments as trapped immediately;
whole stripes fully inside the range are trivially fully free and
stay claimable. A bounded overcount in the safe direction, never
an undercount; the commit rescan remains the authority and
reconciles (and now grants tickets when it lowers the counter,
since admission waiters may be blocked on exactly that headroom).
This closes the retire-to-rescan window the previous patch left
open, and closes it in the one place every returning range passes
through: counting at the individual close/drain call sites instead
misses paths (the drain returns have two) and gets the arithmetic
wrong (round_up() on the non-power-of-two 448K stripe width).
- When a margin-backed data allocation must split across free space
fragments, every non-final piece is trimmed to whole stripes of the
group it lands in, so at most one piece per delalloc extent can strand
a tail -- the margin pre-paid exactly one. The trim is done in
do_allocation_stripe() for the group being tried, not up front in
btrfs_reserve_extent(): groups narrow at the fill edge (8 devices of
which two are small give 448K and then 320K full stripes), and a piece
sized for the widest stripe strands a tail in every narrower group it
lands in. btrfs_reserve_extent() only marks the split
(ffe_ctl->split_bytes, the untrimmed length, so a group that undoes its
allocation does not pass a trimmed length on). A piece shorter than one
stripe of the group is left alone: that is the tail the margin pays
for.
- A data writeback allocation returning ENOSPC anyway is loudly
warned: the margin should make it impossible, and the pages are
dropped.
With this, the fill-to-ENOSPC harness goes fully clean for the first
time: zero writeback drops, zero warnings, zero allocation failures,
margin drains to zero at idle, and fill capacity is unchanged. A
writeback ENOSPC retry loop that this replaces is dropped entirely.
Known bounds, accepted: compressed writes are probed one unit per
128M range but charged per 128K piece (probe optimistic, charge
honest); direct IO is probed but carries no margin (its reserve-to-
allocate window is one syscall); a remount toggling stripe_alloc
with dirty delalloc drains the counter (charges are gated on the
mount option, releases are not, and the counter clamps at zero).
Zygo Blaxell [Sun, 2 Aug 2026 07:35:36 +0000 (03:35 -0400)]
btrfs: stripe_alloc: count open stripe run remainders against data reservations
Close the remaining reservation-vs-allocation windows the commit-time
stripe_unusable rescan cannot see:
- bytes_stripe_open (new): the sum of open runs' unallocated
remainders, maintained per block group under stripe_run_lock at
every open/alloc/grow/close, synced into the space_info after each
mutation, and counted in btrfs_space_info_used(). Claimed bytes are
invisible to the free space cache and will either be allocated or
become trapped tails at close, so reservations must not be admitted
against them.
A closed run's returned tail is still only counted as trapped by the
commit-time rescan, so reservations can race the retire-to-rescan window
inside a commit and be admitted against freshly trapped tails. The next
patch closes that window from the free space cache side, where every
returning range is seen and no call site can be missed.
Measured on the fill-to-ENOSPC rolling test before this change (with
only the bytes_stripe_unusable accounting): tree 'b' still lost 3.3%
(8532 blocks) and tree 'c' 30% (62371 blocks) to silent writeback
allocation failure as trapping compounded.
/sys/fs/btrfs/<uuid>/stripe_park_stats reports lifetime counters for
the parking machinery: rbios parked, bytes merged into parked rbios,
and how each park ended (filled to a full stripe, kicked by a waiter or
a settle/retire flush, or expired at its deadline), plus how many
flushed writes were padded to full stripes versus still needing the
RMW read phase. The filled/kicked/expired split shows directly whether
the deadlines are sized right for a given system -- expired parks that
later reappear as rmw_reads are the batching the window failed to
capture -- so any future retuning (or an adaptive deadline) can argue
from measurements instead of taste.
btrfs: raid56: pad sub-stripe writes to full stripes in open runs
A sub-stripe write does a full RMW: read every untouched data sector of
the stripe, recompute parity, write. Under stripe-exclusive allocation
the read phase is usually pointless: a stripe covered by a live stripe
run has never been written at or past the run's allocation frontier, so
the sectors being read contain nothing.
When every data sector the rbio does not cover lies at or past the
frontier, zero-fill those sectors instead of reading them and write
them out with the stripe -- the zeros must reach the disk, or the
parity (computed over them) would not match what scrub reads back. The
partial write becomes one full-stripe write: no read phase, one parity
pass. A frontier that grows during the attempt is safe: the newer
allocation's write serializes behind this rbio's stripe lock and lands
over the zeros. If any uncovered sector is below the frontier (already
allocated to someone else), fall back to the normal RMW.
Together with kicking parked rbios before the fast fsync's writeback
wait, this removes both stalls the raid56 layer added to fsync under
stripe_alloc: the park deadline and the RMW read round trip.
The pad sets the rbio's data bitmap under bio_list_lock: until
RBIO_RMW_LOCKED_BIT is set another writer can merge into the rbio and OR
its bits in under that lock, and a lost pad bit would leave rows out of
the write while the cached stripe marks their zero-filled pages up to date.
btrfs: stripe_alloc: private per-inode stripe runs for log-active inodes
An fsync-heavy inode shares the open band runs with every other writer.
Closing the open stripe run covering every logged extent at each log
commit, which "btrfs: stripe_alloc: settle a logged extent's stripes at
log commit" does later in this series to make completed fsyncs
crash-safe, would then close a shared run at each of its log commits,
fragmenting concurrent write streams and trapping the shared run's tail.
Give log-active inodes their own runs instead. The first fsync of an
inode sets a sticky runtime flag (at btrfs_sync_file entry, before the
fsync flushes its own delalloc, so even that first fsync's allocations
are steered). Datacow allocations for a flagged inode come from a
private LOG-class run: claimed like any run but never placed in the
shared band slots, owned by the inode, and found by owner lookup under
the block group's stripe_run_lock. A per-inode hint seeds the
allocator's search with the run's location; the hint is advisory (a
stale hint costs a lookup miss, never a wrong run, since the owner
match is authoritative). When no fully-free stripes remain for a
private run the allocation falls back to the shared runs, restoring
the previous placement with unchanged safety.
One inode's log commit then settles only its own stripes: other
writers' runs stay open and their streams stay contiguous, and the
only trapped tails are the fsyncing inode's own. Private runs also
give each log-active inode exclusive stripes, which a later change
uses to copy live tails forward and reclaim them without touching
foreign extents.
The commit-time retirement closed runs by walking the band slots,
which private runs never occupy; walk the block group's run list
instead so every open run, slotted or private, is closed and drained
under invariant I2 (no open run survives a transaction commit).
btrfs: stripe_alloc: control the policy with a filesystem property
A mount option is an awkward fit for write-hole protection: it occupies
a mount-option bit, and protection silently lapses whenever the option
is forgotten -- an fstab edit, a rescue mount, a recovery boot -- which
is exactly when a degraded raid56 is most likely to be written.
Control the policy with a "stripe_alloc" filesystem property instead,
following the property system's compression precedent: an xattr in the
btrfs.* namespace, user-visible and admin-controlled, on the top-level
subvolume's root directory. Set it once (setfattr -n btrfs.stripe_alloc
-v 1, or btrfs property once btrfs-progs learns the name) and it is
persistent: the kernel applies it when the root directory inode loads
its properties during mount, before any user IO. Deleting the xattr
disables the policy; open runs drain at the next commit's retirement,
which runs unconditionally. Stray copies of the xattr -- a received or
cloned subvolume -- are ignored: only the top-level root carries the
policy, so receiving a stream from a stripe_alloc filesystem cannot flip
the policy on the destination.
The support checks (no space_cache=v1, not zoned, no remap-tree, not
mixed block groups) are enforced both when the property is set and when
it is applied at mount. The mixed block group test matters most here.
btrfs_check_mountopts() refuses mixed for the mount option, and without
the same test this would be a second way in that skips it -- enabling a
policy the series does not support there, after the block group read
that would have warned about uncovered metadata has already run, so the
user is told the filesystem is protected and it is not.
This stays within the series' no-on-disk-format-change constraint, and
that is the compatibility story: an older kernel mounts the filesystem
read-write and simply uses the legacy allocator, which is fully
compatible because the on-disk layout is unchanged. New kernels honor
the property during the feature's long-tail testing period; if no use
case surfaces where stripe-exclusive allocation is worse than the write
hole it protects against, it can eventually become the only allocation
mode and the property a no-op.
Known old-kernel interactions with the btrfs. namespace: existing
kernels list and read the xattr (the btrfs. get path is a plain xattr
read) but refuse to set or remove unknown property names, so the flag
can only be managed from a kernel that knows it. btrfs-progs
interaction (check, property list) with an unrecognized property is a
userspace compatibility item to verify and, if needed, patch.
The mount option is kept for now as a non-persistent override. Later
protection stages with different risk profiles (the log-tree
full-stripe relocation) should be gated by their own property rather
than widening this one, so their testing exposure can be controlled
independently.
The property is also applied before tree-log replay, which allocates and
commits: without the policy its allocations can land in the free sectors
of partially written stripes next to committed data. The xattr item is
read directly rather than through an inode, since replay may rewrite the
root directory's inode item; the root directory's load after replay still
applies the final state.
A remount copies the whole mount_opt word from its fs_context snapshot, so
the current STRIPE_ALLOC value is carried into the context first, or a
property change made after the context was created would be undone
without its enable or disable work.
btrfs: raid56: batch stripe_alloc partial writes into full-stripe writes
A raid56 sub-stripe write pays a read-modify-write: read the rest of the
stripe, recompute parity, write. The plug callback merges rbios submitted
within one plug window and the stripe cache saves re-reads for
back-to-back writes, but measurement shows what survives them: on a
4-device raid5, small-file writeback produces 100% sub-stripe rbios (zero
full-stripe writes) and a mixed 4K-512K workload ~55%, so parity is
recomputed and rewritten once per file that lands in a stripe.
Stripe-exclusive allocation gives raid56 something the allocator never
guaranteed before: the rest of an open run's stripe is either written
before the run closes -- within the current transaction -- or not at all.
That makes holding a partial write a sound bet, so park it: a partial
write rbio whose stripe belongs to a stripe run keeps the stripe
lock but does not start its RMW. Later writes to the stripe merge into
it through the existing lock_stripe_add() path, and the moment its data
bitmap covers the stripe it is submitted as a single full-stripe write:
one parity computation, no reads.
A draining run counts, not just an open one. A run closes the moment it
is fully allocated -- typically milliseconds before its last writes reach
the raid56 layer -- so gating on "open" would leave exactly those tail
writes unable to park and unable to pad, and they are the bulk of the
read-modify-writes this patch exists to remove. A draining run is in
fact the safest thing to park against: its frontier is frozen, so
everything below it is inflight IO the commit drain already waits for
(arrival guaranteed) and everything above it is dead space the pad may
fill.
Completion accounting can never trigger the flush of runts -- parked bios
have not been submitted, and preallocated or discarded allocations
produce no bios at all -- so liveness comes from explicit flushes: the
run retirement paths flush before the commit's IO drain (which waits on
the very bios parked here), ordered-extent waiters flush their range on a
short retry period, and a timer bounds everything else. Writes with a
blocked waiter (REQ_SYNC: fsync- and sync-driven writeback) park with a
~3ms deadline instead of 100ms -- long enough for the rest of one
writeback pass over the stripe to merge in, short enough to stay
invisible in fsync latency -- and a sync bio merging into a parked rbio
pulls the deadline in. Only the bg->open_stripe slots are consulted for
eligibility, and a parked rbio that a racing retirement closed out from
under is bounded by the timer, never by the drain.
Measured (same 4-device raid5, deterministic workloads, classifying every
write rbio at rmw_rbio):
fsync latency is 12ms vs 8ms unpatched (each fsync pays one short park).
Batching also collapses trapped free space in a mixed-size forward fill
from ~87 MiB to ~3 MiB: stripes now fill completely before the commit
closes their run, so the frontier strands almost nothing.
Validated: btrfs selftests; mixed-size + concurrency + churn + balance
workload with trapped-space accounting returning to zero after deleting
everything; scrub and full sha256 read-back over 4005 mixed random files;
and the dm-log-writes write-hole crash matrix (24/24 controls clean, 0/36
armed cases with committed data damage) -- parking does not perturb the
stripe-exclusivity guarantee it rides on.
Parking widens a run's post-close drain from microseconds to the park
deadline, which exposes an assumption of the range-to-run lookups: at most
one run on a block group's list covers any address. A reservation
discarded before anything references it (the cow_file_range() error path
under ENOSPC, the find_free_extent() backout paths) returns its bytes to
the free space cache at once, so a stripe can become fully free while its
old run is still draining and be claimed into a new run.
btrfs_claim_free_stripe_run() therefore trims a candidate to end before
the first live run overlapping it, or rejects it if its head overlaps.
Without that, fill-to-ENOSPC/delete cycles with balance and fsstress on a
4-device raid5 hit an inflight accounting assertion within 15 minutes.
The fast fsync path waits for page writeback without the ordered extent
wait that flushes parked rbios, so it flushes the parked rbios covering its
ordered extents before the wait. A sync rbio leaving a raid_unplug() batch
(a sync(2) or WB_SYNC_ALL sweep, whose whole flush shares one plug) has
nothing more coming and a waiter behind it, so it does not park; unplugged
sync submissions (the fsync flush) still park and merge their sibling bios.
rbio_try_park() keeps reading the rbio after publishing it (to queue the
timer and recheck rbio_is_full()), while a flusher may already have taken,
written and freed it, so the parking thread holds its own reference across
that window. Whatever is still parked at unmount or on a failed mount is
sent down, and the park timer cancelled, before the raid56 workqueue is
destroyed.
btrfs: stripe_alloc: grow the frontier run instead of stranding its tail
When an allocation is larger than every open run's remainder, the
allocator claims fresh fully-free stripes and opens a separate run,
leaving the old run's tail behind. The banded cursors keep that tail
open for a while, but a band collision or the commit eventually closes
it, and the tail -- perfectly usable space that a misfit merely jumped
over -- is stranded in a partially filled stripe. In a forward fill
this happens at every size upshift, and measurement shows it is where
nearly all of stripe_alloc's trapped space comes from: on a mixed-size
fill it strands ~14% of the data written, a figure that band-granularity
tuning moves by at most a tenth and commit frequency does not move at
all.
Grow the frontier run instead. If the freshly claimed stripes directly
follow the end of an open run of the same class, extend that run to
absorb them and place the allocation at the run's old tail, spilling
contiguously into the new stripes. Nothing is stranded, and the extent
is physically contiguous. In a forward fill the by-size claim naturally
returns the stripes adjacent to the frontier, so the single frontier run
just keeps growing -- the allocator packs the way the stock allocator
does, while keeping stripe exclusivity: the grown stripes were claimed
fully free, the run still closes at the next commit and is never
reopened, so every stripe of the run is filled within one open-run
lifetime. A grown run keeps its open_seq; it can only still be in a
band slot if the retire walk for its window has not yet run, which the
open_stripe_lock nesting at the claim site excludes from racing.
Measured on a 4-device raid5 with a deterministic interleaved 4K-512K
fill (trapped space via the stripe_unusable counter, byte-identical data
both sides): 111.1 MiB trapped without growing, 7.2 MiB with -- a 94%
reduction, and usable capacity at ENOSPC within 0.5% of the stock
allocator (1561 vs 1569 MiB, vs 1421 MiB without growing). The benefit
survives churn: with age-correlated deletion the grown runs' contiguous
packing lets whole stripes free together, and steady-state usable
capacity stays at 1486-1561 MiB versus 1284-1421 MiB without growing
across correlated and adversarial small-file deletion patterns. Data
integrity verified by scrub and full sha256 read-back over mixed
random-content files, and trapped-space accounting still returns to
zero when all files are deleted.
The open-stripe selftest is updated: the misfit allocation now grows the
run and lands at the run's old tail, spanning the stripe boundary, and
the relocation-class case doubles as a check that a claim adjacent to
another class's run never grows it.
btrfs: stripe_alloc: keep an open run per size band to trap less
A single open stripe run per class forces every allocation that does not
fit the current run's remainder to close that run -- trapping its
unallocated tail in a now partially filled stripe -- and claim a fresh
fully-free stripe. A workload that interleaves small and large extents
(the common case) therefore strands a tail on every size change, even
though a later small allocation could have filled it.
Keep one open run per power-of-two band of remaining free space instead
(band k holds a run with remaining in [2^k, 2^(k+1)) sectors), per class.
An allocation takes the run in the smallest band that still fits --
segregated best fit -- so a small write lands in an already-open, nearly
full stripe rather than opening a new one, while a large write that no
open run can hold opens a fresh stripe and leaves the smaller runs open
for the small writes that do fit them. After each allocation the run is
re-placed into the band its new remainder falls in; when two runs collide
in a band the fuller one is kept -- it can serve larger future
allocations, and closing it would trap more -- and the other is closed,
its tail returned to the free space cache.
16 bands cover a full stripe of up to 2^16 sectors, far more than any
raid56 geometry (nr_data_stripes * stripe_len / sectorsize is 158 sectors
at 10 data stripes and a 64K stripe). Commit-time retirement and the
read-only / removal paths now walk every band of every class; only the
bg->open_stripe[][] slots can ever hold an open run, so one pass still
suffices, and the obsolete single-slot assertions are dropped.
This does not touch the write-hole guarantee: every run is still a
contiguous fill of fully-free stripes, closed at commit and never
reopened. It only changes which open run an allocation joins, reducing
the free space trapped in partial stripes -- and thus stripe_unusable --
for mixed-size workloads.
The open-stripe selftest is rewritten to the multi-cursor contract: a
misfit allocation now keeps the old run open (its tail is not returned to
the cache) and a later small allocation backfills it, so fewer stripes are
claimed before the free space is exhausted.
btrfs: account stripe_alloc trapped free space for honest statfs
Free space in the partially filled stripes of a stripe_alloc (raid56
write-hole-safe) block group is real free space -- it is in the free
space tree and cache -- but the stripe-exclusive allocator cannot hand it
out until the whole stripe frees. statfs therefore over-reports available
space, promising free space that a later allocation refuses with ENOSPC.
Track this trapped space per block group as stripe_unusable, summed into
space_info->bytes_stripe_unusable. It is derived, so there is no on-disk
format change and the free space tree and cache are left untouched
(preserving the extent-tree/free-space-tree consistency btrfs check
verifies).
stripe_unusable is defined by a scan that accumulates every free byte of
the block group into a per-stripe array and then sums the stripes that are
partially filled. A per-stripe accumulator, rather than a streaming sweep
or a per-stripe cache search, is what makes the result correct regardless
of the order the free space cache yields its ranges -- an offset-sorted
bitmap entry can emit runs that lie past a following extent entry, so the
ranges are not globally monotonic -- and regardless of whether free space
is stored as extents or bitmaps. A debug-build assertion checks that the
scan distributes exactly the cache's free space. Because trapped space
only settles at commit -- when the retire path returns partially filled
stripes and deleted extents are unpinned -- the scan is recomputed at
commit for groups whose free space changed (flagged cheaply on the
allocation and free paths), which suits the timescale of the reclaim it
feeds far better than a running per-extent tally. The space_info total is
recomputed as the sum of the armed groups after those rescans, so it
cannot drift.
statfs subtracts bytes_stripe_unusable from the data f_bavail so df
reports what can actually be allocated, and the counter is exposed at
/sys/fs/btrfs/<uuid>/allocation/data/bytes_stripe_unusable. It is
deliberately not part of btrfs_space_info_used(), so the reservation layer
is unchanged and this carries no ENOSPC-behaviour risk.
The counter is armed once a group's free space cache is loaded, so a group
contributes zero until then and statfs starts optimistic and settles to
honest as groups cache and commit. It is disarmed when a group turns
read-only (its free space is then accounted as read-only and already
excluded from statfs) and re-armed by a rescan on the way back to
read-write, so a balance both recovers trapped space and updates the
counter. A CONFIG_BTRFS_DEBUG-only sysfs trigger, stripe_unusable_rescan,
forces a full recompute for auditing the accounting.
Two deliberate imprecisions, matching existing behaviour rather than
bettering it: statfs subtracts bytes_stripe_unusable but not
bytes_zone_unusable, so stripe_alloc df reports availability before reclaim
while zoned df reports it after; and superblock stripes, permanently
unusable in every profile, are left to the existing bytes_super accounting
rather than separately reported here. Both await a maintainer decision on
a common convention.
inc_block_group_ro() asks the other groups for as much free space as the
group withdraws. The group's trapped bytes are already counted in
btrfs_space_info_used() through bytes_stripe_unusable, so they are
subtracted from that demand rather than counted twice. Otherwise a
fragmented raid56 data group could never be made read-only, and scrub
(which needs raid56 groups read-only) and device replace failed with
"scrub: failed setting block group ro: -28" on a churned filesystem.
The scan walks the group's free space tree, so the entries of an
allocation cluster the legacy allocator set up are returned to the group
before the scan, when arming and at the commit-time rescan.
btrfs: retire stripe runs at commit and gate stripe_alloc
Hook stripe run retirement into the transaction commit, after the wait
for pending ordered extents and before TRANS_STATE_COMMIT_DOING stops
accepting joins. The retirement drain waits out in-flight data writes
into the stripes of the transaction's runs: a pure data-IO wait, never
an ordered extent wait. It must run while the transaction still
accepts joins: a task holding unreported stripe run bytes (a reserved
extent whose bio has not completed) can block on a transaction join,
for example for a chunk allocation while walking a delalloc range, and
the drain waits for those bytes, so retiring after joins are blocked
would deadlock. After the drain, no stripe this transaction references
can ever be written again, so a crash after the superblock write cannot
tear it -- closing the raid56 write hole for stripe-allocated block
groups without requiring flushoncommit. A defensive retirement in
btrfs_free_block_groups() covers the transaction abort path.
Joins are still accepted between the retirement and
TRANS_STATE_COMMIT_DOING. An ordered extent whose stripe run was opened
after the committing transaction's retire point cannot insert its file
extent into that transaction, so btrfs_finish_one_ordered() ends its
handle and waits for the commit's critical section to end. That wait
must not be taken
under the range's extent lock: buffered writers block on the locked
range in lock_and_cleanup_extent_if_need() while holding their
prepared, locked folios, which stops the writeback flusher and
kcompactd behind those folios, which stops reclaim -- and the
committing transaction is itself draining data bios whose submission
path can need memory (dm-crypt bounce pages on the host where this was
caught). Drop the extent lock and the FINISHING_ORDERED tag before the
wait and re-take them before re-joining, preserving the
extent-lock-before-join ordering. The window is safe: the range is
still covered by the pending ordered extent, so a writer that takes the
lock finds it, releases its folios and waits -- which is exactly what
unbinds writeback and reclaim.
Force COW for in-place writes that would land in raid56 data block
groups while stripe_alloc is enabled (nodatacow files and writes into
preallocated extents): a single in-place write could tear a stripe
containing other files' committed extents, making the guarantee
conditional on the whole filesystem's usage. The check sits in
can_nocow_file_extent(), covering buffered and direct IO with one
choke point, and applies per extent so nocow to non-raid56 profiles
keeps working. The data relocation inode is exempt: its extents live
in relocation-class stripe runs that never share stripes with other
data, and relocation depends on in-place writes.
Validate the option at mount: it is refused with space_cache=v1, whose
cache inode is nodatacow, preallocated and nodatasum and is overwritten in
place during commit (the free space tree and no cache at all are both
fine: nothing in the allocator reads either), with the
remap-tree feature (whose relocation writes bypass
the ordered extent accounting) and on zoned filesystems (which have
their own allocator and no write hole).
Mixed data+metadata filesystems are refused too. The immediate symptom
is a hang: a run drains when it is closed and its inflight_bytes reach
zero, and the only things that subtract are ordered extent completion, a
discarded allocation and a freed reservation. Metadata has no ordered
extent -- btrfs_alloc_tree_block() allocates with is_data false and
end_bbio_meta_write() reports nothing back to the run -- so a metadata
allocation raises inflight_bytes and nothing ever lowers it, the run
never drains, and btrfs_retire_open_stripes() waits for a completion
that has no code path to arrive from. The hung task detector stays
quiet, because wait_var_event() sleeps in a state it exempts; the
filesystem just stops. Reproduced deterministically on
mkfs.btrfs -M -d raid5 -m raid5 mounted -o stripe_alloc: the first sync
after a few hundred small files never returns.
That symptom is not why the refusal is permanent: a later patch
(stripe_meta) gives metadata its own completion report, and the drain
then terminates. These are why:
- raid56 deliberately skips csum lookup for mixed block groups, to
avoid recursing into a metadata read while holding the full stripe
lock (see the comment above the map_type test in fill_data_csums()).
That test reads the block group's flags, so in a mixed group it
cannot tell a data stripe from a metadata stripe and disables
verification for every one of them. Stripe-exclusive allocation
could keep the two apart -- allocation classes already never share a
run, so never a stripe -- but nothing durable records which class a
stripe held, and asking the extent tree while holding the full stripe
lock is the recursion the test exists to avoid. The recovery paths
here assume a rebuilt data sector can be checked; in a mixed group it
cannot.
- Data and metadata draw on one space_info and one free space pool.
Data reservations carry a pessimistic whole-stripe margin, metadata
reservations carry none, and a metadata claim that finds no fully
free full stripe returns ENOSPC -- which for metadata aborts the
transaction rather than failing one write. A data fill can starve
metadata into an abort, which separate block groups cannot do.
Mixed block groups are a mkfs-time property of small filesystems, where
raid56 is least appropriate and stripe_alloc's trapped space costs
proportionally most, and the option cannot become applicable later. So
do not carry a half-supported mode.
btrfs_is_stripe_alloc_bg() therefore requires DATA without METADATA as
belt and braces for a block group that somehow reaches the allocator
anyway, and moves out of extent-tree.c's file scope so block-group.c
can share the one copy of the rule. (Metadata and system block groups
never carried the DATA flag, so they were already excluded.)
The cache format can only be converted at mount, so space_cache=v1 is
refused in btrfs_check_mountopts() and checked again after
btrfs_reconfigure() restores the cache options to what is on disk, and
cache_save_setup() declines to set a v1 cache up while stripe_alloc is on,
at the point where the in-place write would be issued. v1 cache inodes
left on disk by an earlier mount are harmless and not refused: nothing
writes them without cache_save_setup().
A range-limited writeback (filemap_fdatawrite_range() before a dedupe or
clone, for example) allocates the whole contiguous delalloc range as
ordered extents but submits only the folios inside its range. The rest
stay allocated and unsubmitted until the flusher reaches the inode, and
the commit's retirement waits for them: commits of 600 s were seen with
vm.dirty_writeback_centisecs=60000. So while stripe_alloc is enabled,
btrfs_run_delalloc_range() takes the zoned path, run_delalloc_cow() with
extent_write_locked_range(), and writes the whole allocated range at once;
run_delalloc_nocow() does the same after a range in which fallback_to_cow()
ran. The cost is the zoned one: a small dedupe or clone of a large dirty
file waits for up to max_extent_size of IO that the allocation had already
committed to.
btrfs: report stripe run data IO through the ordered extent lifecycle
Pair every byte reserved from an open stripe run with exactly one
"write done" report, so commit-time retirement can wait for all data IO
into a window's stripes:
- Ordered extents get a stripe_run pointer, attached at creation by a
range lookup (cheap: gated on the fs having any stripe runs at all,
and a run's block group membership is established before its
allocation returns, so the gate cannot miss). NOCOW and PREALLOC
ordered extents write into previously allocated extents, which can
never lie inside a run claimed from fully-free stripes, and are
skipped. The report fires once at IO completion (the IO_DONE moment
in can_finish_ordered_extent(), before any transaction join, so the
commit-time drain can never deadlock on a blocked join), with a
catch-all when an ordered extent is freed without completing IO.
- Reservations freed without an ordered extent ever owning them (error
paths) report through btrfs_free_reserved_extent(); the one caller
that frees a range an ordered extent did own (the finish-error path)
uses btrfs_free_reserved_extent_ordered() to avoid double reporting.
- Preallocated extents never issue data IO and report at insertion;
this also covers relocation's data inode preallocations.
btrfs: add the stripe_alloc allocation policy for raid56 data
Wire the open stripe run allocator into find_free_extent() as a per-
block-group policy: with the new stripe_alloc mount option, allocations
from raid56 data block groups go through btrfs_alloc_from_open_stripe()
instead of the clustered allocator, and only ever land in fully-free,
stripe-aligned runs. Since the loop's terminal LOOP_NO_EMPTY_SIZE
degradation only applies to the clustered path and stripe block groups
ignore empty_size/empty_cluster, the natural terminal behaviour is:
no fully-free stripe in any block group -> allocate a chunk -> ENOSPC.
The availability hint feeds max_extent_size so callers retry with
smaller allocations instead of failing early.
Stripe runs get an allocation class: relocation overwrites its
preallocated extents in place, so its allocations must never share a
stripe with ordinary cow data. Classes never share a run, and one
block group at a time is softly dedicated to relocation by reusing
fs_info->data_reloc_bg (btrfs_clear_data_reloc_bg() moves from zoned
code to generic code for this; the class tag, not the dedication, is
what carries correctness). The dedication is dropped when relocation
finishes or the dedicated group runs out of stripes.
Setting a block group read-only now retires its stripe runs (in
btrfs_inc_block_group_ro(), covering scrub, relocation and unused
block group deletion), pairing with the allocator's ->ro check under
the stripe_run_lock so no run can survive into or be created in a
read-only group.
A write admitted just before a group was dedicated to relocation may still
need that group's stripes, so the dedication excludes the group only from
the allocator's first passes: a write no other group can serve falls back
to it once the search reaches the chunk-allocation loop. The class tag
already keeps the two sides out of each other's stripes; zoned's hard
exclusion follows from append-only zones, which do not apply here.
An error from the stripe allocation itself (-ENOMEM for a run) skips the
group like any other failure, and is returned instead of -ENOSPC when no
other group has space. It is never taken as a found extent, which would
reserve an offset this group did not set.
On a filesystem that has gone into error, the drain waits for a read-only
group's runs and for the retirement of the open runs end at once: after a
transaction abort the ordered extents holding a run's bytes in flight are
only torn down at unmount, so the waits would never return (a reclaim
worker hung in btrfs_inc_block_group_ro() on a degraded raid6 whose commit
had hit ENOSPC, and the same wait would park the transaction kthread or
wedge close_ctree()). The runs are left as they are, released at teardown,
and a warning says when runs were left in flight.
__btrfs_handle_fs_error() wakes the drain waiters, after the full barrier
wake_up_var() needs when the condition is not set under a lock.
btrfs: add open stripe run tracking for stripe-exclusive allocation
Add the in-memory state and lifecycle for "open stripe runs", the
allocation windows of the raid56 stripe-exclusive allocation policy. A
run is a contiguous stripe-aligned region claimed whole from the free
space cache via btrfs_claim_free_stripe_run() and filled strictly
sequentially by btrfs_alloc_from_open_stripe(). A run closes when it is
exhausted, when an allocation does not fit its remainder, or when the
transaction commit retires it; closed runs are never reopened and their
unallocated tails return to the free space cache, where the fully-free
claim rule makes them unallocatable until the whole stripe frees. This
is what will guarantee that a full stripe only receives writes within
one commit window, closing the raid56 write hole for these block groups.
Each run counts reserved bytes whose data IO has not completed yet,
maintained under the block group lock and reported back through
btrfs_open_stripe_write_done(). btrfs_retire_open_stripes() implements
commit-time retirement: bump the retire sequence, close every run opened
before it, and wait for their inflight bytes to drain. It is a pure
data-IO wait, deliberately not an ordered extent wait: it is designed to
run after the committing transaction stops accepting joins
(TRANS_STATE_COMMIT_DOING with a single writer), where waiting for
ordered extent completion would deadlock on the blocked transaction
join, and where every extent the transaction references already has its
own data on disk. Allocations racing with the commit open runs stamped
with a newer sequence and are neither retired nor waited for; their
extents can only be referenced by the next transaction. Block groups
with runs are tracked on an fs_info list whose membership is
established before an allocation returns, which is what lets the retire
walk rely on the sequence stamp.
Exercised by a new sanity self-test; the raid56 stripe allocation
policy and the commit hook will be the first non-test users.
btrfs: add btrfs_claim_free_stripe_run() for stripe-exclusive allocation
Add a free space cache primitive that finds and removes a contiguous,
fully-free, stripe-aligned run of full stripes from a block group. This
is the building block for a raid56 allocation policy that never issues
sub-stripe writes into stripes containing committed data, closing the
raid56 write hole for datacow writes: because a partially-filled stripe
can never satisfy the fully-free requirement, stripes retired at commit
time become unallocatable without any persistent allocator state.
The search walks the by-size free space index (largest max contiguous
free run first) rather than the by-offset tree: an entry whose largest
contiguous free run is smaller than a full stripe cannot contain a
fully-free stripe, and every following entry is no larger, so the search
stops at the first such entry. Near-full, where the free space
degenerates into many sub-stripe holes, that is an O(1) fast fail
instead of a scan of the whole free space tree on every allocation --
the dominant cost of the known raid56 near-full allocation slowdown.
Full stripe geometry is relative to the block group start and supports
non-power-of-two stripe widths. Unaligned head and tail remainders are
returned to the free space cache with their trim state preserved. Runs
are found within a single free space entry; a fully-free stripe split
across an extent entry and a bitmap neighbour is deliberately not found,
which errs toward missing a claimable stripe, never toward claiming a
non-free byte.
The function is exercised by new sanity self-tests covering aligned and
capped claims, partial-stripe exclusion, head/tail carving, block-group-
relative geometry, bitmap entries, and non-power-of-two stripe widths.
The raid56 stripe allocation policy will be its first non-test user.
Josef Bacik [Tue, 3 Nov 2026 15:05:19 +0000 (15:05 +0000)]
btrfs: don't report success for a skipped log sync
When a commit of the log root tree gives up, because the transaction
needs a full commit or writing the log failed, btrfs_sync_log() still
advances log_transid_committed and hands the result to the contexts on
the list. Unlike a successful commit it does not advance log_transid, so
the log transaction it just retired is still the one new syncs join.
A sync that gets to the log root tree after that adds its context to
the emptied list, sees log_transid_committed >= its log_transid and
returns its log_ret, which is still 0:
fsync A (subvol 1) rename fsync B (subvol 2)
btrfs_need_log_full_commit()
-> false
btrfs_set_log_full_commit()
btrfs_need_log_full_commit()
-> true
btrfs_remove_all_log_ctxs()
log_transid_committed = N
log_transid stays N
log_transid = N, log_ret = 0
log_transid_committed >= N
return 0
btrfs_sync_file() then ends the transaction without committing it, and a
crash before A's transaction commit loses what B was told is durable.
This was reproduced with delays added to btrfs_sync_log().
Fix by advancing log_transid when a commit of the log root tree gives
up, so a later sync joins the next log transaction and either commits it
or gives up itself. The check for an already committed log transaction
can then no longer be true, so warn on debug builds if it is, and force
a full commit.
[zygo: 6.18 backport: 6.18's btrfs_sync_log() has no
trace_btrfs_sync_log_exit() calls and clears log_commit[] with
atomic_set(); context adjusted, no code change]
Josef Bacik [Tue, 3 Nov 2026 15:05:18 +0000 (15:05 +0000)]
btrfs: wait for ordered extent IO in fast fsync
A fast fsync waits for writeback and then checks
BTRFS_INODE_COW_WRITE_ERROR to see whether a COW write failed. If one
did, it must wait for the ordered extents to complete before logging,
since until then the extent maps of the failed range point to extents
that were never written. Logging them puts unwritten extents in place
of the data that was there before when the log is replayed.
The flag can be set too late for that check. end_bbio_data_write()
clears writeback before it finishes the ordered extent:
end_bbio_data_write() fsync
btrfs_folio_clear_writeback()
filemap_fdatawait_range()
BTRFS_INODE_COW_WRITE_ERROR clear
btrfs_log_changed_extents()
btrfs_finish_ordered_extent()
set BTRFS_INODE_COW_WRITE_ERROR
The write paths that fail before submitting a bio, writepage_delalloc()
and submit_write_sector(), clear writeback and finish the ordered
extent through btrfs_mark_ordered_io_finished(), which never sets the
flag at all.
Fix by dropping the flag and having a fast fsync wait for
BTRFS_ORDERED_IO_DONE on the ordered extents it collected for logging.
can_finish_ordered_extent() sets it only after recording any error with
BTRFS_ORDERED_IOERR. The wait costs little, because IO_DONE is set
right after writeback is cleared, not when the ordered extent completes.
[zygo: 6.18 backport: 6.18 takes ordered_tree_lock with IRQs disabled on
the bio completion side (btrfs_finish_ordered_extent() and
btrfs_mark_ordered_io_finished() use spin_lock_irqsave()), so
cow_write_failed() takes it with spin_lock_irq(); btrfs_inode.h and
ordered-data.c context adjusted; IO_DONE is woken on ordered->wait by both
6.18 completion paths (can_finish_ordered_extent(),
btrfs_dec_test_ordered_pending())]
Fixes: f13e01b89daf ("btrfs: ensure fast fsync waits for ordered extents after a write failure") Fixes: a11d6912fdd9 ("btrfs: remove folio parameter from ordered io related functions") CC: stable@vger.kernel.org # 6.10+ Signed-off-by: Josef Bacik <josef@toxicpanda.com> Link: https://lore.kernel.org/r/20261103-b4-btrfs-data-integrity-v2-3-577dc2690061@toxicpanda.com Assisted-by: Claude:claude-opus-5-5
Josef Bacik [Tue, 3 Nov 2026 15:05:17 +0000 (15:05 +0000)]
btrfs: clear dirty blocks after a delalloc failure
If writepage_delalloc() runs the first delalloc ranges of a folio and
then btrfs_run_delalloc_range() fails, the blocks that already got an
ordered extent are finished with an error through
btrfs_mark_ordered_io_finished(), but their dirty bits are left set. The
next writeback then submits blocks whose ordered extent is gone, or is
about to be completed a second time.
Fix by clearing the dirty bits and cycling the writeback state for those
blocks, as submit_write_sector() does for a block it fails to submit.
[zygo: 6.18 backport: the 6.18 loop also clears the folio's ordered flag
(btrfs_folio_clear_ordered()); the new lines go before it]
Fixes: 72dad8e377af ("btrfs: fix double accounting race when btrfs_run_delalloc_range() failed") CC: stable@vger.kernel.org # 6.14+ Reviewed-by: Qu Wenruo <wqu@suse.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Josef Bacik <josef@toxicpanda.com> Link: https://lore.kernel.org/r/20261103-b4-btrfs-data-integrity-v2-2-577dc2690061@toxicpanda.com Assisted-by: Claude:claude-opus-5-5
Qu Wenruo [Fri, 22 May 2026 09:23:53 +0000 (18:53 +0930)]
btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
Recently kernel RAID56 lib is trying to remove the unexpected
single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
btrfs still supports such setup, which means in the long run btrfs has
to handle such corner case by ourselves.
Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
P/Q stripes all match each other, rotation also makes no difference.
This patch will disguise those single-data-RAID56 chunks as
RAID1/RAID1C3 chunks.
This is done at two timings:
- Chunk read
- Chunk allocation
This is done by introducing btrfs_chunk_map::on_disk_type member, which
stores the type read from the on-disk metadata.
Meanwhile btrfs_chunk_map::type is calculated using on_disk_type.
For most profiles @type matches @on_disk_type, but for
single-data-RAID56, the @type will be RAID1/RAID1C3.
This method has a minimal impact on the fs, all other operations like
scrub and read-repair, are all based on the chunk map type, so the
disguise method will require no extra modification to those call sites.
Although there are still some locations that are checking against
block_group->flags, e.g. scrub. Those call sites will still get extra
limits assuming the bg is RAID56. But it should not cause any extra
problem.
Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
(cherry picked from commit 49ba3a3c0d67aa74618a3f8dabea2e2e372102aa)
[6.18 backport: btrfs_chunk_map still has io_align/io_width there.
fill_dummy_bgs() also set bg->flags a second time, from map->type;
that duplicate line is dropped so the on-disk type stays.]
Zygo Blaxell [Thu, 22 Oct 2026 08:42:21 +0000 (04:42 -0400)]
btrfs: back off block group reclaim after repeated ENOSPC
Near a full filesystem every reclaim of a block group can fail with
-ENOSPC, and the worker keeps retrying: a failed group goes back on the
reclaim list at the end of each run (unless periodic reclaim is on) and is
taken again at the next kick. Each attempt makes the group read-only, does
part of the relocation and fails, all while holding the balance exclusive
operation, so a resize, replace or balance cannot start.
This was seen with stripe-exclusive raid56 allocation, whose trapped space
queues groups for reclaim at the fill edge: for over an hour of fsstress
reclaim held the exclusive operation, "error relocating chunk ...: -28"
repeating, with the worker in a metadata reservation (btrfs_block_rsv_refill
under relocate_block_group) because metadata had no room either. Nothing
in the retry loop is specific to that allocator.
After BTRFS_RECLAIM_ENOSPC_STREAK (3) reclaims in a row fail with -ENOSPC
in a space_info, back off: the worker drops that space_info's groups until
10 minutes pass, a chunk is added or removed, or the space_info's free
space (free in its groups plus unallocated) grows by what the last failed
group held. A group that failed with -ENOSPC is not put back on the list
for immediate retry.
/sys/fs/btrfs/<uuid>/allocation/<type>/reclaim_backoffs counts the
backoffs, and the start of one is logged at debug level ("reclaim: ...").
Zygo Blaxell [Tue, 20 Oct 2026 17:04:21 +0000 (13:04 -0400)]
btrfs: log block group reclaim at debug level
The reclaim worker logs only part of what it does. A relocation prints
relocate_block_group()'s "relocating block group ... flags ..." line, but
neither the start of a reclaim (with the group's used and reserved bytes)
nor its outcome is logged; the "reclaiming chunk" info message was removed
as log spam in a507904090d0 ("btrfs: remove redundant auto reclaim log
message"). Only the btrfs_reclaim_block_group tracepoint and the
reclaim_count/reclaim_errors counters in sysfs remain, so a kernel log
taken from a test or a user report cannot show what reclaim decided, or
when.
Log the start of each reclaim (with the group's flags, length, used and
reserved bytes) and its successful end with btrfs_debug(), so they are off
by default and can be enabled with dynamic debug:
echo 'file fs/btrfs/block-group.c format "reclaim: " +p' \
> /sys/kernel/debug/dynamic_debug/control
Errors are still logged with btrfs_err() as before.
btrfs: do not reclaim block groups on an errored filesystem
Block group reclaim makes each group read-only and relocates it, starting
transactions as it goes. Once the filesystem has gone into error (a
transaction abort, say) every transaction it starts fails with -EROFS, so
the work can only fail, while it holds reclaim_bgs_lock and sb_writers and
the balance exclusive operation. Return before taking the groups.
Seen with stripe-exclusive raid56 allocation, whose read-only transition
waits for the group's writes in flight: on a degraded raid6 whose commit
hit ENOSPC and aborted, the reclaim worker sat in that wait for good. The
wait is fixed where it is; the worker has nothing to do there either way.
Reading I/O statistics via sysfs concurrently with tearing down a
frontend device might result in UAF or a NULL dereference, as the
reading of the statistics data is not synchronized with freeing the
data.
In order for the function to legitimately avoid locking/RCU, it needs to
run after all TX/RX processing has finished/stopped: Lookups happen from
xenvif_start_xmit(), while adds/removes happen from xenvif_tx_action()
(called out of xenvif_poll()).
Move the call out of xenvif_disconnect_data() (the call to the function
from connect()'s error path doesn't require any cleaning up there), and
put it past the synchronization point in backend_disconnect().
The network statistic data in the driver is protected by RCU.
Unfortunately when tearing down the queues after an error during
connecting to the frontend (e.g. due to illegal Xenstore data filled
in by the guest), the statistic data is being freed without
synchronizing the RCU state. In case the data is being read via sysfs
in this phase, access to already freed memory or memory being in new
use is possible.
Fix that by calling synchronize_net() before freeing the queue data.
To avoid hard-coding these values in drivers, define constants for them
that drivers can reference. Also, update all instances in bitmap.rs and
id_pool.rs that use these values to use the new constants.