Zygo Blaxell [Sun, 9 Aug 2026 20:04:23 +0000 (16:04 -0400)]
btrfs: stripe_meta: allocate tree blocks by whole stripes
Metadata gets the placement rule data already has: tree blocks are handed
out from runs of fully free stripes, and a stripe is never revisited once
the transaction that filled it has written it. No separate mount option: the
chunk profile already decides it. raid56 metadata has the same write hole
as raid56 data, so it gets the same protection at the same time, and
metadata on any other profile has no write hole to close -- the policy's
RAID56_MASK test simply does not match it, so nothing changes there.
It does cost a stranded stripe tail per commit, and a metadata allocation
that fails aborts the transaction rather than failing one write. Those
are the price of covering raid56 metadata at all rather than a reason to
make it opt-in separately: a filesystem unwilling to pay them does not
want raid56 metadata, and converting it to raid1c3/raid1c4 removes both
the cost and the write hole.
Only system chunks are left uncovered: btrfs_is_stripe_meta_bg() tests the
METADATA flag and a system chunk does not carry it, so a raid56 system
chunk keeps the legacy read-modify-write. The mount-time warning is
narrowed to say exactly that instead of claiming metadata is uncovered,
which it no longer is.
The machinery is the data machinery, including the per-sector liveness
map the previous patch gave it. Metadata needed three things of its own:
- A completion report. A run drains when every byte allocated from it
has been accounted for, and metadata had no equivalent of ordered
extent completion, so its runs could never drain -- the deadlock that
mixed block groups hit. end_bbio_meta_write() now reports, and
btrfs_open_stripe_write_abandoned() covers a tree block freed before
it was ever written (COWed twice in one transaction, say).
- A two-phase closure. btrfs_retire_open_stripes() runs before the
commit writes tree blocks, so it skips metadata runs; they are closed
and drained afterwards by btrfs_retire_meta_stripes(), once every block
is on disk. Closing earlier would strand blocks the commit has not
allocated yet, draining earlier would wait for writes it has not
issued. Metadata runs are also freed there rather than when their
counter reaches zero: a stripe's columns can be written by separate
rbios, and freeing on the first completion leaves the second with no
run to pad or merge against.
- Whole stripes per writeback pass. The opportunistic writeback passes
do not know about stripes and will happily write three tree blocks now
and five later; btrfs_defer_stripe_meta_write() leaves the blocks
dirty for the commit (or a WB_SYNC_ALL waiter) instead, the same shape
as the zoned write-pointer deferral.
Two free paths bypass the usual completion reporting and must report the
block abandoned themselves: btrfs_free_tree_block() can send a
same-transaction unwritten block through the delayed-ref machinery when
check_ref_cleanup() finds its ref head already processed, and
clean_log_buffer() discards unwritten log tree blocks via
btrfs_pin_reserved_extent() without ever visiting btrfs_free_tree_block().
Either miss leaves the block counted as inflight forever: the run never
drains, the block group reference never drops, and unmount asserts. Both
were found by the dm-log-writes replay matrix, whose crash-recovery mounts
exercise both paths routinely.
Zygo Blaxell [Sun, 16 Aug 2026 07:54:18 +0000 (03:54 -0400)]
btrfs: stripe_alloc: cache a failed stripe run scan per block group
On a large aged filesystem (85 TiB, mostly legacy pre-stripe_alloc data,
heavily fragmented free space), the delalloc workers spin at 100% CPU
while commits crawl at KiB/s. NMI backtraces put the time in
btrfs_claim_free_stripe_run() called from btrfs_alloc_from_open_stripe()
for every small compressed extent allocation.
The cost is in find_free_stripe_run_slow(): when the single-entry fast
path misses (always, on fragmented block groups), the slow path walks
candidate stripes from the START of the block group, doing an rbtree
lookup per free-space piece, all under ctl->tree_lock. When the block
group has no claimable run at all - the common case for block groups
filled with legacy data - the scan traverses the entire block group,
returns -ENOSPC, and remembers nothing: the next 4K allocation repeats
the whole scan. Per-allocation cost is O(block group size) in the
fragmented worst case, multiplied by every block group find_free_extent
visits.
The stock allocator avoids exactly this pathology by caching the largest
extent a failed search saw (max_extent_size); the stripe claim path is a
new search primitive built beside it and never inherited the idea. The
per-block-group stripe_claimable counter cannot serve this role: it is a
commit-scan-granular byte estimate that can err high (block groups where
the accounting promises claimable bytes but the scan finds no aligned,
usable run are precisely where the scan storm happens).
Add bg->max_claimable_run, an upper bound learned from the scans
themselves: (u64)-1 unknown, 0 after an exhaustive scan proved no
claimable run exists (find_free_stripe_run_slow() accepts any run of at
least one full stripe, so its failure is that proof). Claims fail in
O(1) while the bound is below one full stripe, or when total free space
in the block group is smaller than a full stripe. Adding free space
resets the bound to unknown in __btrfs_add_free_space(), which covers
all paths that can create a run: frees, unpins, and stripe run returns.
The bound is only trusted and only set once the block group is fully
cached, because cache loading links entries directly and would bypass
the reset.
Zygo Blaxell [Fri, 14 Aug 2026 01:12:04 +0000 (21:12 -0400)]
btrfs: stripe_alloc: gate data admission on claimable whole-stripe supply
Wire stripe_claimable into the DATA reservation path so write() returns -ENOSPC
when there is no fully-free whole stripe to place the write crash-safely --
matching statfs f_bavail, which already subtracts the trapped partial-stripe
space. Add stripe_claimable_admit() and AND it into the two data-admit sites:
__reserve_bytes() and btrfs_try_granting_tickets() (the latter is required, or a
flushed data ticket would be granted by the plain used<=total rule, bypassing the
gate). Scoped to raid56 stripe_alloc DATA (stripe_margin_unit != 0); metadata,
non-raid56 and zoned space_infos are unaffected.
The gate refuses at reservation time, before the range becomes delalloc and later
parks at writeback with nowhere to land -- so ENOSPC is clean and there is no
fill-edge parking collapse.
Phase 1 is pessimistic: it gates on bytes_stripe_claimable alone, which errs LOW,
so it can refuse a write that would have landed in a partially-open stripe
(over-refuses by the open remainder, healed by the commit rescan raising claimable
and the FLUSH_DATA ticket retry). A later change adds bytes_stripe_open to the
bound for statfs-exact behavior.
Zygo Blaxell [Tue, 11 Aug 2026 05:41:53 +0000 (01:41 -0400)]
btrfs: raid56: adaptive park deadline from the parked-rbio backlog
Full-stripe batching parks a sub-stripe write's rbio for a deadline so the
writes that fill the rest of its stripe can merge into it, turning a
read-modify-write into one full-stripe write. The deadline is a fixed
per-filesystem value. It is long enough to catch those merges, but at a
small-file fill to ENOSPC -- where a run's stripes never fill because each
file's neighbours land in the next file's stripe rather than this one --
every partial write waits the deadline out, and then its 10x stuck cap, on
an arrival that never comes before padding to a full stripe and going down.
Lowering the deadline globally fixes that fill but throws away the merges
the deadline exists to catch on ordinary moderate writes.
Feed the deadline back from the parking machinery's own backlog instead of
guessing at the workload. Track stripe_parked_now, the number of
currently-parked rbios, incremented as a partial write parks and
decremented as it unparks. When it runs ahead of the stripe_park_congestion
knob (0 disables it, the default), a new async partial write takes the short
sync deadline through the existing sync-park path rather than the full one.
This is negative feedback: the shorter deadline drains the backlog, so once
it falls back below the knob later writes regain the full deadline and full-
stripe batching. No space-state estimate is needed -- a small-file flood
builds the backlog directly, while a large sequential fill (already full
stripes, never parked) and a moderate steady stream do not.
Measured on an eight-device raid5. A small-file balance-reclaim soak that
times out at the default deadline completes at a congestion of 4; on a
throttled moderate stream, where the backlog stays low, a congestion of 32
never trips, keeps the full deadline, and merges three times as much with
38% fewer RMW reads as the same filesystem under a flat 3ms deadline. The
gauge and a congestion_short counter join the stripe_park_stats sysfs file
so the knob can be set from measurement. Off by default, and no data-path
change: a shortened park still pads or reads exactly as it would have, only
sooner.
Data reservations are admitted against arithmetic -- free space minus
trapped fragments minus open-run remainders minus a per-extent margin --
and under sustained near-full churn the arithmetic and the claim rule
disagree for long enough that admitted buffered writes reach writeback
with nothing claimable left: measured, ~5.2-5.5k reserved writebacks
dropped per fsstress churn run, silently for anything not waiting on
fsync. Stock refuses the same write()s up front. Nothing between
"durable by fsync" and "refused by write()" is acceptable when an
operator accident fills the disk.
First step, accounting only: measure the constraint instead of deriving
it. bg->stripe_claimable counts bytes of fully free whole stripes --
exactly what btrfs_claim_free_stripe_run() can take, the complement of
stripe_unusable within each stripe. The commit rescan derives it from
the same pass that computes trapped bytes (free minus trapped) and is
the sole upward correction; incremental maintenance under
ctl->tree_lock only ever DEBITS -- removals round their decrement OUT
to every touched stripe, claims subtract exactly what they took. The
counter can therefore only under-count between commits, never
over-count.
Crediting freed whole stripes incrementally on the add side is
deliberately not done, having been tried and dropped. Several add
paths re-add free space that a low-level remove never de-credited --
the async discard trim (unlink_free_space / bitmap_clear_bits, then
do_trimming's re-add through __btrfs_add_free_space) and
btrfs_remove_free_space's middle-split tail re-add -- so crediting on
add double-counts and drives the counter ABOVE the authoritative scan.
That is the dangerous direction: once admission gates on this counter,
an over-count admits reservations against phantom supply that writeback
then drops, while an under-count is an early, clean write()-time
ENOSPC. Debit-only makes over-counting structurally impossible,
whatever a future re-add path forgets to de-credit. (Reproduced with
compress+autodefrag+discard=async on a legacy-converted raid5
filesystem, incremental ~= 2x scanned; credit_return attribution
confirmed the phantom entered through do_trimming's re-add, not the
open-stripe allocator returns.)
A WARN_RATELIMIT at the rescan catches the dangerous direction anyway
(incremental above scanned), which under a debit-only rule means a
missed consumption site, and localizes it; the clamp to the scanned
value keeps every rescan authoritative regardless. Rate limited rather
than _ONCE because such a gap recurs on every commit, and reporting
only the first hit hides that it is ongoing.
The space_info aggregate bytes_stripe_claimable follows the
stripe_unusable pattern (incremental between commits, re-totaled from
armed groups at the rescan), shows in the ENOSPC dump and in sysfs.
No admission change yet; that comes once the counter proves accurate
under the fill, churn, balance and reclaim suites. The by-size
fast-fail in the claim keeps its early exit: the point of measuring is
to make such inputs trustworthy, not to search harder around them.
Data block groups only for now; raid56 metadata and mixed block groups
join when the data counter has settled.
Zygo Blaxell [Sun, 9 Aug 2026 03:14:23 +0000 (23:14 -0400)]
btrfs: stripe_alloc: track per-sector liveness and pad from the map
Padding used to ask where the run's allocation frontier was, which
cannot see an allocation abandoned below it -- a reservation released, an
allocation the finder discarded, an extent freed before its write was
ever issued. Runs now carry a bit per sector, set at allocation and
cleared by the new btrfs_open_stripe_write_abandoned() report, and
padding fills exactly the sectors that are dead: this replaces
btrfs_stripe_run_pad_start() with btrfs_stripe_run_pad_mask(). The map
is sized once, with headroom for frontier growth, because the growth
path holds the run lock where nothing may sleep; a run that would
outgrow its map stops growing instead. A closed run's final stripe
overhangs its shrunk end by construction, and those sectors -- the
returned tail, unclaimable while the stripe holds live data -- are dead
and paddable too.
The abandonment report is what closes a writeback ENOSPC race the
fsync-heavy near-full workload hits: an allocation undone without a
report left phantom inflight bytes and phantom-live sectors, and
reservations admitted against that state failed at writeback
(cow_file_range -28) where stock refuses the write() up front. With
the report in place the same workload shows zero writeback failures at
every reservation margin setting, including margin disabled.
Claiming also learns an O(1) fast fail: the by-size index walk is
extracted into find_free_stripe_run(), stopping at the first entry whose
largest contiguous free run cannot hold a full stripe -- near-full, where
free space degenerates into many sub-stripe holes, this replaces a scan
of the whole tree on every allocation.
Moved out of the stripe_meta patch, which needs all of this for tree
blocks but introduced it tangled with the metadata machinery; data wants
it on its own.
Zygo Blaxell [Sat, 8 Aug 2026 15:27:36 +0000 (11:27 -0400)]
btrfs: stripe_alloc: complete parked writes on allocation coverage, not clocks
A parked partial-stripe write goes down its RMW path when its deadline
expires or a retirement flush kicks it, even when every byte it is
missing below the run's frontier belongs to an allocation whose data IO
is already in flight -- allocation happens at writeback submission, and
the commit's retirement drain waits for exactly those arrivals. Timing
out such a park buys nothing and costs a stripe read plus a second write
of the same stripe: measured on an 8-device raid5 buffered fill, the
100ms deadline turns ~2200 parked stripes per 2GB into read-modify-
writes (pad_decline_live), and stretching the clock 20x recovers only
14% -- the clock is the wrong instrument.
Replace the clock with a coverage test. A parked rbio is ready when its
gathered bios cover everything the covering run has allocated inside its
stripe (btrfs_stripe_run_alloc_ceiling); the remainder lies at or past
the frontier, so the existing pad turns it into a single full-stripe
write with no read phase. Merges check readiness as they land, the park
timer holds unready parks instead of expiring them, and run-retirement
flushes leave unready parks parked: the retirement has already closed
the run (freezing the allocated prefix) and its drain waits on the very
arrivals that will complete them. Waiter-driven flushes (fsync's
pre-writeback kick, ordered-extent waits) and sync parks keep today's
forced behaviour: a blocked waiter's latency beats a saved stripe read.
A stuck cap (10x the park deadline) bounds the wait when an arrival can
never come: a writeback error abandoned the allocation, or the stripe
was already written once by a forced sync park. Such parks are forced
down the old path and counted.
New sysfs stripe_park_stats counters: unparked_ready (parks completed by
the coverage test) and stuck (parks forced at the cap).
Zygo Blaxell [Sat, 8 Aug 2026 07:57:22 +0000 (03:57 -0400)]
btrfs: stripe_alloc: trace why padding declined, at the moment it declined
The counters say how often padding refused and broadly why, but not what
the run looked like when it happened, and reconstructing that from the
extent tree afterwards cannot distinguish "the stripe was already
allocated when this write arrived" from "it was allocated shortly after".
Those want different fixes, so record the decision where it is made.
Also count the second refusal, which had no counter at all: the oracle can
allow padding and the sector walk still refuse, because a sector below the
frontier is not covered by this write. That is the case where another
allocation shares the stripe and its data is not in this rbio -- a
different thing from the oracle finding the whole stripe allocated.
The tracepoint carries the stripe, how much of it this write covers, the
refusal reason, and the run's start/end/frontier/inflight, so a trace says
directly whether the frontier had already run past the stripe when the
write showed up.
Zygo Blaxell [Sat, 8 Aug 2026 06:03:40 +0000 (02:03 -0400)]
btrfs: stripe_alloc: tunable park deadlines, and say why padding declined
The deadline constants are meant to be judged "from measurements rather
than taste", but they are compile-time, so every data point costs a kernel
build and a reboot. Measuring a buffered fill -- dd, no fsync, so nothing
kicks the park and the backstop is the only exit -- found 1861 of 3984
parks expiring at the 100ms deadline, which is precisely the no-waiter case
the constant exists for and the one with no numbers behind it.
Expose both deadlines as writable sysfs files, clamped to 60s because a
parked rbio holds its stripe lock, with 0 disabling parking for that class
(useful as an experiment in itself):
Add the counters the stats file cannot supply. rmw_reads counts only
parked writes that still had to read, so it cannot answer "does this
workload read-modify-write at all"; data_rmw counts every data RMW, which
with allow_rmw empty on a covered block group is the number the
stripe-exclusive claim is about, and it should be zero.
And record why padding refused, because the reasons are different defects:
the rest of the stripe is allocated and its data has not arrived yet
(wait longer, or kick when the frontier advances); no open run covers the
stripe, so the run closed before its own write went down; or the frontier
never reached the stripe at all. Attributing them turns "some RMW
remains" into a specific thing to fix.
Zygo Blaxell [Wed, 5 Aug 2026 22:15:06 +0000 (18:15 -0400)]
btrfs: raid56: say whether a metadata read-modify-write is a write hole
meta_rmw counts sub-stripe metadata writes, which is a proxy for exposure
rather than a measurement of it. A stripe modified in place is only a write
hole if it holds data some completed transaction is relying on; a stripe
that two writes of the *same* transaction happen to split costs an extra
read but risks nothing, because a tear loses that whole transaction anyway.
The counter cannot tell those apart, so it cannot answer the only question
that matters.
The rbio can. A read-modify-write has already read every column the write
does not cover, so at the point of the report it is holding the stripe's
on-disk contents. Walk the uncovered tree block positions and read their
headers: a block whose bytenr and fsid match belongs there, and its
generation says which transaction put it there. Compare that against the
generation of the blocks this write is carrying, taken from the same rbio,
so a transaction committing concurrently cannot skew the verdict:
meta_rmw_cur the same transaction's own blocks -- a cost
meta_rmw_old an earlier transaction's -- a write hole
meta_rmw_free no tree block there at all
System block groups are classified too, not just metadata ones. A system
chunk carries no METADATA bit, so a flags test excludes it, but it holds
the chunk tree: the same tree block header, the same generation field, and
a worse consequence if a degraded crash tears it, since without the chunk
tree no logical address can be mapped at all. meta_rmw already counted
system chunks; only the classification skipped them, so their read-modify-
writes were visible as a rate and never as a verdict. Whether a raid56
system chunk ever rewrites parity over a committed tree block is therefore
an open question that this answers by measurement rather than by reasoning
about how the chunk tree is laid out.
This trusts nothing the allocator says about itself. The stripe runs, the
liveness map and the drain accounting are all bookkeeping that could be
wrong in the same way twice; the header in the sector is what a degraded
read would actually have to reconstruct.
On a 3-device raid5 filesystem with raid5 metadata, 6000 small files with
periodic syncs and then a third rewritten, plain stripe_alloc reports 249
sub-stripe metadata writes of which 174 rewrite parity over committed tree
blocks. Adding stripe_meta leaves 37 sub-stripe writes and none of them.
Zygo Blaxell [Tue, 4 Aug 2026 19:51:54 +0000 (15:51 -0400)]
btrfs: stripe_alloc: report read-modify-write of uncovered stripes
The mount-time warning says which block groups stripe_alloc does not
cover. It cannot say whether anything is actually landing there, and on
a filesystem with raid56 metadata that is the interesting question: every
sub-stripe write to an uncovered stripe is a write hole window, where
parity and data reach the disk separately and a crash in between leaves
the stripe unreconstructible.
Report it from the one place that knows the write is going out as a
read-modify-write rather than as a full or padded stripe. Rate limited,
because a raid56-metadata filesystem does this continuously and the point
is to make the exposure visible rather than to fill the log, and paired
with a meta_rmw counter in the existing stripe_park_stats sysfs file so
the rate can be read off without grepping dmesg.
Silent when stripe_alloc is off: there the whole filesystem works this
way and the user has asked for nothing else.
Zygo Blaxell [Tue, 4 Aug 2026 06:43:20 +0000 (02:43 -0400)]
btrfs: stripe_alloc: warn at mount when raid56 metadata is not covered
stripe_alloc closes the write hole for raid56 data. It does nothing for
raid56 metadata, which keeps read-modify-write and keeps the hole, and
on a mixed-block-group filesystem it now does nothing at all. A user
who mounts -o stripe_alloc on -d raid5 -m raid5 has every reason to
believe the filesystem is covered, and finds out otherwise only after a
crash on a degraded array -- with metadata damage, which is worse than
the data damage they were protecting against.
Say it once at mount, after the block groups are read, so the message
reflects what is actually on disk rather than what was asked for. A
warning, not an error: the data guarantee is real and worth having on
its own, and a filesystem can be converted to raid1c3/raid1c4 metadata
without unmounting.
Zygo Blaxell [Fri, 7 Aug 2026 01:35:56 +0000 (21:35 -0400)]
btrfs: stripe_alloc: claim fully-free stripes that span free space entries
The claim fast path searches single entries, so a fully-free full
stripe whose free space spans an entry boundary -- an extent entry
adjoining a bitmap, or two neighbouring bitmap windows -- was
unclaimable. stripe_unusable accounting is entry-blind and counts
exactly those stripes as claimable, so admission reserves data against
them; at writeback the claim finds nothing, cow_file_range() gets
-ENOSPC, and the already-dirtied pages are dropped. full_stripe_len
is not a power of two, so stripe boundaries drift through the fixed
128M bitmap windows and a straddling stripe is a certainty near full,
not a corner case.
Observed live at the raid6 fill edge (rolling-failure, then isolated
by fill-edge-debug with enospc_debug): two 384K stripes, each
straddling a bitmap window boundary, held the space_info accounting
786432 bytes above what the claim could reach, and every writeback
allocation against that phantom margin failed -ENOSPC while ~991MB of
genuinely trapped free space sat in the cache. The reservation margin
cannot absorb this: the gap is per straddling stripe, not per
outstanding extent.
Add an entry-blind slow path: one offset-ordered walk accumulating
contiguous free coverage across entry boundaries, and a piecewise
removal that runs in the same tree_lock critical section as the find,
so a racing claimer cannot see a half-removed run. The fast path is
unchanged and still serves the common case; the slow path runs only
after it fails, which is the near-full case where a stranded stripe
matters most.
Zygo Blaxell [Mon, 3 Aug 2026 07:22:52 +0000 (03:22 -0400)]
btrfs: stripe_alloc: pessimistic data reservation margin
A byte-counted data reservation holds no collateral against the stripe
claim rule. Between admission at write() time and the allocation at
writeback, the trapped-space picture keeps moving: commits close open
stripe runs and trap their sub-stripe tails, and free space returning
mid-transaction (unpinned deletions, same-transaction frees, drained
run tails, freed reservations) lands in the free space cache as
fragments the claim rule can never hand out but that no counter yet
reflects. The counters were honest at every instant for NEW
admissions while OLD admissions were left holding air; delalloc
writeback gets no second chance, so the pages were dropped. Measured
at the fill edge: ~19 MiB of orphaned bytes_may_use with accounting
and allocator in perfect agreement, and in a later round the admitted
may_use+margin exactly equaled the phantom mid-transaction fragments.
Be pessimistic at reservation time and optimistic at allocation time,
the same shape metadata reservations already use:
- bytes_stripe_margin: one full stripe width (fs_info->
stripe_margin_unit, the widest raid56 data full stripe) per
outstanding delalloc extent, charged and released inside
btrfs_mod_outstanding_extents() so it stays exact across delalloc
merges and splits -- including async compression, where the io-tree
split hook grows the margin to one stripe per compressed piece,
matching the true worst case. Counted in btrfs_space_info_used()
so it holds back admissions. The margin exists only for in-flight
dirty data; steady-state capacity and statfs are untouched.
- Admission probes len + margin through the ticketed FLUSH_DATA path
(then immediately re-releases the probe, which the delalloc hooks
re-charge), so a writer that cannot be covered waits for the
flusher -- commits convert pinned deletions into claimable whole
stripes -- and receives an honest ENOSPC at write(2) if flushing
cannot produce cover.
- Every btrfs_add_free_space() into an armed group counts the added
range's sub-stripe head and tail fragments as trapped immediately;
whole stripes fully inside the range are trivially fully free and
stay claimable. A bounded overcount in the safe direction, never
an undercount; the commit rescan remains the authority and
reconciles (and now grants tickets when it lowers the counter,
since admission waiters may be blocked on exactly that headroom).
This closes the retire-to-rescan window the previous patch left
open, and closes it in the one place every returning range passes
through: counting at the individual close/drain call sites instead
misses paths (the drain returns have two) and gets the arithmetic
wrong (round_up() on the non-power-of-two 448K stripe width).
- When a margin-backed data allocation must split across free space
fragments, every non-final piece is kept a whole-stripe multiple
(rounddown; the width is not a power of two), so at most one piece
per delalloc extent can strand a tail -- the margin pre-paid
exactly one.
- A data writeback allocation returning ENOSPC anyway is loudly
warned: the margin should make it impossible, and the pages are
dropped.
With this, the fill-to-ENOSPC harness goes fully clean for the first
time: zero writeback drops, zero warnings, zero allocation failures,
margin drains to zero at idle, and fill capacity is unchanged. A
writeback ENOSPC retry loop that this replaces is dropped entirely.
Known bounds, accepted: compressed writes are probed one unit per
128M range but charged per 128K piece (probe optimistic, charge
honest); direct IO is probed but carries no margin (its reserve-to-
allocate window is one syscall); a remount toggling stripe_alloc
with dirty delalloc drains the counter (charges are gated on the
mount option, releases are not, and the counter clamps at zero).
Zygo Blaxell [Sun, 2 Aug 2026 07:35:36 +0000 (03:35 -0400)]
btrfs: stripe_alloc: count open stripe run remainders against data reservations
Close the remaining reservation-vs-allocation windows the commit-time
stripe_unusable rescan cannot see:
- bytes_stripe_open (new): the sum of open runs' unallocated
remainders, maintained per block group under stripe_run_lock at
every open/alloc/grow/close, synced into the space_info after each
mutation, and counted in btrfs_space_info_used(). Claimed bytes are
invisible to the free space cache and will either be allocated or
become trapped tails at close, so reservations must not be admitted
against them.
A closed run's returned tail is still only counted as trapped by the
commit-time rescan, so reservations can race the retire-to-rescan window
inside a commit and be admitted against freshly trapped tails. The next
patch closes that window from the free space cache side, where every
returning range is seen and no call site can be missed.
Measured on the fill-to-ENOSPC rolling test before this change (with
only the bytes_stripe_unusable accounting): tree 'b' still lost 3.3%
(8532 blocks) and tree 'c' 30% (62371 blocks) to silent writeback
allocation failure as trapping compounded.
btrfs: raid56: skip parking for sync rbios finished by their unplug batch
Parking exists to widen the merge window, and for plugged submissions
the unplug callback is the natural end of that window from the
submitter's side: once raid_unplug() has sorted and merged the batch,
nothing more is coming from it. A sync rbio parked after that point --
a sync(2) or WB_SYNC_ALL sweep, whose whole flush shares one plug --
has a waiter behind it and can only sit out the sync deadline. Mark
rbios leaving an unplug batch and let sync ones skip parking.
Unplugged sync submissions (the fsync flush) still park: their sibling
bios arrive one by one and merge into the parked rbio, and the fsync
path kicks it as soon as they have all been submitted, so parking there
is the merge mechanism and the deadline already never fires.
/sys/fs/btrfs/<uuid>/stripe_park_stats reports lifetime counters for
the parking machinery: rbios parked, bytes merged into parked rbios,
and how each park ended (filled to a full stripe, kicked by a waiter or
a settle/retire flush, or expired at its deadline), plus how many
flushed writes were padded to full stripes versus still needing the
RMW read phase. The filled/kicked/expired split shows directly whether
the deadlines are sized right for a given system -- expired parks that
later reappear as rmw_reads are the batching the window failed to
capture -- so any future retuning (or an adaptive deadline) can argue
from measurements instead of taste.
btrfs: raid56: do not park sync writes into nocow runs
An in-place overwrite of a nodatacow file under stripe_alloc parked
like any partial write, but with a waiter behind it and nothing worth
merging: in-place overwrites arrive one fsync at a time, and unlike
the datacow fsync path nothing kicks the parked rbio before the page
writeback wait, so every fsync ate the full sync park deadline.
Measured against stock nodatacow on the same rig, that deadline was
the bulk of a ~3.7ms per-fsync regression. Skip parking for sync
writes whose stripe belongs to a NOCOW-class run; async writeback
still parks and merges there.
btrfs: stripe_alloc: persist nocow runs across commits and remounts
A nodatacow inode's private stripe run used to close at every
transaction commit like all runs, so a slowly appended nocow file burned
a fresh stripe per commit -- and after a remount its partial stripe's
free tail was abandoned outright. Neither cost buys anything: the run
machinery's commit-time closing exists for invariant I2, and a nocow
stripe holds only the owner's write-hole-waived data, so there is
nothing for I2 to protect.
Keep NOCOW-class runs open across commits: the commit-time retirement
and its drain predicate skip them (their extents insert without the
window-sequence deferral, which is fine -- the data is on disk when the
ordered extent finishes, and later same-stripe writes can tear only the
owner's own data). They still close on forced quiescing (read-only,
removal, unmount) and now on the owning inode's eviction, so a cached
but idle inode cannot pin a claimed tail forever.
Across remounts, re-adopt instead: when a nocow allocation's hint
points into a partial stripe that the committed extent tree proves is
wholly owned by the allocating inode, claim exactly the stripe's free
tail (a new exact-range claim that verifies every byte is free before
removing; nothing else can consume free space inside a partially used
stripe, so verify-then-remove cannot race) and continue the run at the
old frontier. Appends to a nocow file then pack sequentially through
commits, evictions and remounts alike.
The re-adoption lookup must not block. It runs from
btrfs_alloc_from_inode_stripe_run(), i.e. inside find_free_extent(),
which holds space_info->groups_sem for read, and stripe_extents_owned_by()
does a full btrfs_search_slot() on the extent tree. The reverse order is
longstanding upstream -- delayed ref processing holds extent tree locks
and then calls find_free_extent(), which takes groups_sem -- so waiting
here closes a cycle. Both sides only take groups_sem for read, which is
not enough: it is an rw_semaphore and block group creation and removal
take it for write, so a writer queued between the two readers blocks the
second one. The check cannot be hoisted above the lock, since it is a
question about the specific stripe the allocator has just settled on, but
it does not have to block: it is read-only and advisory, and every caller
already treats "not provably ours" as a reason to decline the
optimisation rather than as a fact about the extents. So set path->nowait
for the groups_sem caller and let -EAGAIN fall into the existing error
paths; the cost on contention is a re-adoption that does not happen.
btrfs_stripe_nocow_writable() keeps the blocking search -- it runs from
the nocow check with no groups_sem held, and a spurious "not ours" there
would force COW on an extent that does not need it.
btrfs: stripe_alloc: allow-rmw policy, and write-in-place for isolated nocow extents
Re-enable write-in-place for nodatacow files and preallocated extents
under stripe-exclusive allocation, with the understood caveat that the
write hole cannot be prevented for data that opts out of COW: an
in-place write RMWs its stripe's parity, so a degraded crash can tear
the stripe. What CAN be guaranteed is the blast radius: in-place is
permitted only for extents whose full stripes are isolated to the
writing inode, so such a crash can tear only the writing file's own
data -- the nodatacow contract, no worse.
The gate is per extent, decided where nocow eligibility is already
checked: fast path, the stripe belongs to one of the inode's own
NOCOW-class runs (which only ever held its extents); slow path, the
committed extent tree proves sole ownership (single plain data ref,
count 1, matching root and objectid; anything shared, foreign or
metadata fails). The fully-free claim rule keeps uncommitted foreign
extents out of partially used stripes, so the committed tree is
authoritative. Extents that fail -- anything allocated before stripe
isolation existed, extents shared through reflink or snapshots,
relocated extents -- simply stay force-COWed, and because their rewrite
is steered into the inode's private NOCOW run, the next overwrite of
the same data passes: legacy nocow files migrate themselves to
isolation in one COW generation, with no tool and no flag day.
Log-commit settling skips NOCOW-class runs: nodatacow data gets no
fsync survival guarantee (its own later in-place writes can always
tear it), so closing the run would trap its tail for nothing. The
write-hole debug checker skips groups that hosted NOCOW-class runs,
like relocation-used groups, since isolated in-place writes land in
stripes whose runs have drained.
The interface is a word-list policy naming the cases in which
stripe_alloc may permit the legacy unsafe RMW, each independently:
nodatacow in-place writes for nodatacow files' extents
prealloc in-place writes into preallocated extents
fsync waive the close-at-log-commit guarantee: no settling, no
per-inode LOG steering, no carry-forward; logged stripes
may be extended and RMWed as before those patches
It is "allow_rmw", not "allow_overwrite": the fsync case overwrites
nothing, but all three permit read-modify-write of stripes that a
degraded crash can then tear. The nodatacow and prealloc cases require
the per-extent stripe isolation above (the blast radius stays confined
to the writing file); fsync restores the 3a-era exposure where a
degraded crash may cost just-fsynced data, detectably, in exchange for
none of the log-commit costs.
The policy is persistent as the btrfs.stripe_alloc_allow_rmw property
on the top-level root directory, following stripe_alloc's precedent
(applied when the root inode loads during mount, before any user IO),
with a mount option of the same name as a non-persistent override; the
effective policy is their union. Words are separated by comma, space
or colon -- the mount option form must use colon, since mount splits
options at commas. Everything defaults off: plain stripe_alloc keeps
forcing COW and keeps the full fsync guarantee.
btrfs: stripe_alloc: isolate nodatacow and preallocated extents by stripe
Preallocated extents and nodatacow files' extents are candidates for
write-in-place, which reintroduces the raid56 write hole for every
stripe such a write touches: the RMW recomputes parity that also covers
whatever else shares the stripe. Before write-in-place can be
re-enabled for them (a later change; stripe_alloc still forces COW
today), their placement must guarantee the blast radius: such an extent
must never share a stripe with any other file's data.
Steer them into private per-inode stripe runs of a new NOCOW class,
reusing the log-active inode machinery: runs owned by one inode, never
in the shared band slots, found by owner-and-class lookup. Successive
allocations of the same file pack sequentially into the file's own
stripes; different files, and the datacow/relocation/log classes, never
share a stripe with them. A preallocation signals itself through a new
btrfs_reserve_extent() parameter; nodatacow files are recognized by the
inode flag. Placement remains best effort: when no fully-free stripes
are left for a private run the allocation falls back to the shared
runs, which is safe -- an extent that lands in a shared stripe simply
stays force-COWed when write-in-place arrives.
btrfs: raid56: pad sub-stripe writes to full stripes in open runs
A sub-stripe write does a full RMW: read every untouched data sector of
the stripe, recompute parity, write. Under stripe-exclusive allocation
the read phase is usually pointless: a stripe covered by a live stripe
run has never been written at or past the run's allocation frontier, so
the sectors being read contain nothing.
When every data sector the rbio does not cover lies at or past the
frontier, zero-fill those sectors instead of reading them and write
them out with the stripe -- the zeros must reach the disk, or the
parity (computed over them) would not match what scrub reads back. The
partial write becomes one full-stripe write: no read phase, one parity
pass. A frontier that grows during the attempt is safe: the newer
allocation's write serializes behind this rbio's stripe lock and lands
over the zeros. If any uncovered sector is below the frontier (already
allocated to someone else), fall back to the normal RMW.
Together with kicking parked rbios before the fast fsync's writeback
wait, this removes both stalls the raid56 layer added to fsync under
stripe_alloc: the park deadline and the RMW read round trip.
btrfs: stripe_alloc: kick parked rbios before the fast fsync's writeback wait
The fast fsync path waits for page writeback, which completes only when
the raid56 layer writes the data -- but a partial-stripe rbio parks to
collect merges until its sync deadline, and on this path nothing unparks
it before the wait: the stripes are settled only later, in the logging
itself. The full-sync path does not have this problem because the
ordered extent wait already flushes parked rbios before sleeping.
All of the fsync's writes are submitted before the wait, so nothing more
can merge into its stripes; flush the parked rbios covering the attached
ordered extents instead of sleeping out their deadline. Removes the
sync park timeout from the fast fsync critical path.
btrfs: stripe_alloc: carry an inode's log tail forward at fsync
Closing a log-active inode's private stripe run at each log commit
traps the run's final partial stripe tail every time the inode fsyncs:
a steady fsync stream burns one stripe per log commit until the stripes
free or balance runs.
Reclaim the tails by carrying the partial stripe's live data forward.
At each log commit, the settling walk now computes the closed run's
final partial stripe, and the logging paths record the file ranges of
the logged extents that live inside it on a small per-inode table
(bounded; overflow just means an extent is not carried). At the
inode's next fsync, before its delalloc flush, each recorded range that
still maps to the recorded disk bytenr is re-dirtied, defrag style
(reserve, reset delalloc state, mark the folios dirty; absent folios
are read back, which is a plain read of settled data). The flush then
COWs the carried ranges together with the new data into the inode's
current private run, and the old partial stripe empties and frees
whole.
Everything downstream is the ordinary COW pipeline: new extent maps,
ordered extents, checksums, file extent items and delayed refs, and
the log's modified-extents snapshot -- taken inside btrfs_log_inode
after the flush -- picks up the new addresses by itself. Reflinked
ranges need no special handling: foreign referents keep the old extent
alive in its closed (never again written) stripe, and only the space
reclaim degrades. A range that was rewritten, truncated, punched or
compressed simply fails the mapping check and stays put.
Crash safety does not regress. A carried extent's old copy is dropped
through the normal paths: if it was committed, the free pins until the
transaction commits; if it was logged but never committed, its ADD and
DROP delayed refs cancel in cleanup_ref_head(), which also pins
must_insert_reserved heads -- either way the old blocks cannot be
reallocated before the log that references them is superseded.
The settling walk also learned to report the partial stripe through a
widened btrfs_log_settle_stripes() signature (inode and file range
instead of fs_info); compressed extents pass a zero file length to opt
out of carrying while still settling.
The re-dirty reserves with NO_FLUSH. carry_one_range() takes the folio
locks and the extent range lock before reserving, and a flushing
(ticketed) reservation can sleep in wait_reserve_ticket() until the
flusher makes progress -- but every way forward needs the locks we
hold: FLUSH_DELALLOC has to write back the carried range, which blocks
in __folio_lock() on our locked folio, and a transaction commit (with
flushoncommit) waits for the same writeback. The whole filesystem then
wedges behind the stuck commit. Carrying is best effort by design and
the carried range's data is already durable in its old stripe, so on
ENOSPC just skip the carry and let the old stripe free the slow way.
btrfs: stripe_alloc: private per-inode stripe runs for log-active inodes
Closing the open stripe run covering every logged extent at each log
commit made completed fsyncs crash-safe, but at a placement cost: an
fsync-heavy inode shares the open band runs with every other writer, so
each of its log commits closes a shared run, fragmenting concurrent
write streams and trapping the shared run's tail.
Give log-active inodes their own runs instead. The first fsync of an
inode sets a sticky runtime flag (at btrfs_sync_file entry, before the
fsync flushes its own delalloc, so even that first fsync's allocations
are steered). Datacow allocations for a flagged inode come from a
private LOG-class run: claimed like any run but never placed in the
shared band slots, owned by the inode, and found by owner lookup under
the block group's stripe_run_lock. A per-inode hint seeds the
allocator's search with the run's location; the hint is advisory (a
stale hint costs a lookup miss, never a wrong run, since the owner
match is authoritative). When no fully-free stripes remain for a
private run the allocation falls back to the shared runs, restoring
the previous placement with unchanged safety.
One inode's log commit now settles only its own stripes: other
writers' runs stay open and their streams stay contiguous, and the
only trapped tails are the fsyncing inode's own. Private runs also
give each log-active inode exclusive stripes, which a later change
uses to copy live tails forward and reclaim them without touching
foreign extents.
The commit-time retirement closed runs by walking the band slots,
which private runs never occupy; walk the block group's run list
instead so every open run, slotted or private, is closed and drained
under invariant I2 (no open run survives a transaction commit).
btrfs: stripe_alloc: settle a logged extent's stripes at log commit
Stripe-exclusive allocation bounds degraded-crash damage to the current
transaction, but within that window two log commits can still share a
stripe: fsync 1 writes the head of a stripe, the open run keeps filling
it, and a later write's read-modify-write rewrites the parity that
protects fsync 1's data. A degraded crash during the second write tears
the first -- the write hole's shape, confined to the log window, and the
reason the fsync guarantee has so far been "loss is detectable" rather
than "completed fsyncs survive".
Close it with a case analysis. A logged extent in a full stripe is
already safe: nothing ever writes a full stripe again, since COW never
overwrites and there are no free sectors left to allocate. A logged
extent in a partial stripe is exposed only to future writes into that
stripe's remaining sectors -- so at log commit, close the stripe's open
run (nothing further allocates into it), kick any parked partial writes
for it, and wait for its in-flight data IO before the log super is
written. This is the log-window analogue of invariant I2, which the
commit-time retirement provides for full commits. Both log paths are
hooked: the fast path per extent map in log_one_extent(), and the
full-sync path where copy_items() walks new data extents (old-transaction
extents are skipped there, and are exactly the ones already settled by
their own commit).
The drain is bounded and join-free: run inflight is counted from
allocation, which happens during writeback with submission following in
the same pass, so the wait is bio flight time plus the parked-write
deadline that the flush short-circuits; write_done reporting needs no
transaction join, so waiting under the inode log mutex and a running
transaction handle is safe.
The cost is spatial: every fsync that logs an extent in a partial stripe
retires that stripe early, trapping its unwritten tail like any other
partially filled stripe until it frees or balance reclaims it.
Fsync-heavy workloads therefore burn a stripe tail per touched stripe
per fsync; the follow-up per-inode log runs and copy-forward relocation
exist to reclaim exactly that cost, and are optimizations on top of the
guarantee this patch completes.
btrfs: stripe_alloc: control the policy with a filesystem property
A mount option is an awkward fit for write-hole protection: it occupies
a mount-option bit, and protection silently lapses whenever the option
is forgotten -- an fstab edit, a rescue mount, a recovery boot -- which
is exactly when a degraded raid56 is most likely to be written.
Control the policy with a "stripe_alloc" filesystem property instead,
following the property system's compression precedent: a btrfs.
namespace xattr, user-visible and admin-controlled, on the top-level
subvolume's root directory. Set it once (setfattr -n btrfs.stripe_alloc
-v 1, or btrfs property once btrfs-progs learns the name) and it is
persistent: the kernel applies it when the root directory inode loads
its properties during mount, before any user IO. Deleting the xattr
disables the policy; open runs drain at the next commit's retirement,
which runs unconditionally. Stray copies of the xattr -- a received or
cloned subvolume -- are ignored: only the top-level root carries the
policy, so receiving a stream from a stripe_alloc filesystem cannot
flip the policy on the destination.
The support checks (free space tree, not zoned, no remap-tree, not
mixed block groups) are enforced both when the property is set and when
it is applied at mount. The mixed block group test matters most here.
btrfs_check_mountopts() refuses mixed for the mount option, and without
the same test this would be a second way in that skips it -- enabling a
policy the series does not support there, after the block group read
that would have warned about uncovered metadata has already run, so the
user is told the filesystem is protected and it is not.
This stays within the series' no-on-disk-format-change constraint, and
that is the compatibility story: an older kernel mounts the filesystem
read-write and simply uses the legacy allocator, which is fully
compatible because the on-disk layout is unchanged. New kernels honor
the property during the feature's long-tail testing period; if no use
case surfaces where stripe-exclusive allocation is worse than the write
hole it protects against, it can eventually become the only allocation
mode and the property a no-op.
Known old-kernel interactions with the btrfs. namespace: existing
kernels list and read the xattr (the btrfs. get path is a plain xattr
read) but refuse to set or remove unknown property names, so the flag
can only be managed from a kernel that knows it. btrfs-progs
interaction (check, property list) with an unrecognized property is a
userspace compatibility item to verify and, if needed, patch.
The mount option is kept for now as a non-persistent override. Later
protection stages with different risk profiles (the log-tree
full-stripe relocation) should be gated by their own property rather
than widening this one, so their testing exposure can be controlled
independently.
btrfs: stripe_alloc: never claim stripes covered by a live run
The range-to-run lookups -- attaching an ordered extent to its stripe run
and reporting completed data IO by bytenr -- assume that at most one run
on a block group's list covers any given address. Nothing enforced that.
A reservation that is discarded before anything references it (the
cow_file_range error path under ENOSPC, the find_free_extent backout
paths) returns its bytes to the free space cache immediately, with no
pinning: there is no committed state to protect. When an ENOSPC failure
storm discards every allocation in a stripe, the stripe is fully free
again and the claim rule -- correctly, by its own lights -- hands it out
as part of a new run while the old run object is still on the list
draining its other stripes' IO. Full-stripe write batching widened a
run's post-close drain from microseconds to the parked-write timeout, and
the soak test hit the overlap within minutes: new allocations' ordered
extents attached to the old run (first match by range), their completions
drained the old run's inflight accounting into an assertion failure, and
the new run's accounting never drained, wedging the commit's retire wait.
The overlap is harmless to data -- a stripe can only be re-claimed if
every byte of it is free, and discarded reservations never issued bios --
but the accounting corruption is fatal.
Freed committed extents cannot reproduce this: they return to the free
cache only in the unpin phase at the tail of a commit, and the same
commit's retirement already drained -- and freed -- every run opened
before it. Only the unpinned immediate-free paths race with a draining
run.
Rather than enumerate those paths, enforce the lookups' assumption at the
claim site: btrfs_claim_free_stripe_run() now trims a candidate to end
before the first live run overlapping it, or rejects it if its head
overlaps. Re-claiming such stripes just waits until the old run drains
off the list, which only comes up inside ENOSPC failure storms.
Reproduced with concurrent fill-to-ENOSPC/delete cycles, balance, and
fsstress on a 4-device raid5: the assertion fired within ~15 minutes
unpatched, and ~10 hours of the same load ran clean with this fix.
btrfs: raid56: batch stripe_alloc partial writes into full-stripe writes
A raid56 sub-stripe write pays a read-modify-write: read the rest of the
stripe, recompute parity, write. The plug callback merges rbios submitted
within one plug window and the stripe cache saves re-reads for
back-to-back writes, but measurement shows what survives them: on a
4-device raid5, small-file writeback produces 100% sub-stripe rbios (zero
full-stripe writes) and a mixed 4K-512K workload ~55%, so parity is
recomputed and rewritten once per file that lands in a stripe.
Stripe-exclusive allocation gives raid56 something the allocator never
guaranteed before: the rest of an open run's stripe is either written
before the run closes -- within the current transaction -- or not at all.
That makes holding a partial write a sound bet, so park it: a partial
write rbio whose stripe belongs to a stripe run keeps the stripe
lock but does not start its RMW. Later writes to the stripe merge into
it through the existing lock_stripe_add() path, and the moment its data
bitmap covers the stripe it is submitted as a single full-stripe write:
one parity computation, no reads.
A draining run counts, not just an open one. A run closes the moment it
is fully allocated -- typically milliseconds before its last writes reach
the raid56 layer -- so gating on "open" would leave exactly those tail
writes unable to park and unable to pad, and they are the bulk of the
read-modify-writes this patch exists to remove. A draining run is in
fact the safest thing to park against: its frontier is frozen, so
everything below it is inflight IO the commit drain already waits for
(arrival guaranteed) and everything above it is dead space the pad may
fill.
Completion accounting can never trigger the flush of runts -- parked bios
have not been submitted, and preallocated or discarded allocations
produce no bios at all -- so liveness comes from explicit flushes: the
run retirement paths flush before the commit's IO drain (which waits on
the very bios parked here), ordered-extent waiters flush their range on a
short retry period, and a timer bounds everything else. Writes with a
blocked waiter (REQ_SYNC: fsync- and sync-driven writeback) park with a
~3ms deadline instead of 100ms -- long enough for the rest of one
writeback pass over the stripe to merge in, short enough to stay
invisible in fsync latency -- and a sync bio merging into a parked rbio
pulls the deadline in. Only the bg->open_stripe slots are consulted for
eligibility, and a parked rbio that a racing retirement closed out from
under is bounded by the timer, never by the drain.
Measured (same 4-device raid5, deterministic workloads, classifying every
write rbio at rmw_rbio):
fsync latency is 12ms vs 8ms unpatched (each fsync pays one short park).
Batching also collapses trapped free space in a mixed-size forward fill
from ~87 MiB to ~3 MiB: stripes now fill completely before the commit
closes their run, so the frontier strands almost nothing.
Validated: btrfs selftests; mixed-size + concurrency + churn + balance
workload with trapped-space accounting returning to zero after deleting
everything; scrub and full sha256 read-back over 4005 mixed random files;
and the dm-log-writes write-hole crash matrix (24/24 controls clean, 0/36
armed cases with committed data damage) -- parking does not perturb the
stripe-exclusivity guarantee it rides on.
btrfs: stripe_alloc: grow the frontier run instead of stranding its tail
When an allocation is larger than every open run's remainder, the
allocator claims fresh fully-free stripes and opens a separate run,
leaving the old run's tail behind. The banded cursors keep that tail
open for a while, but a band collision or the commit eventually closes
it, and the tail -- perfectly usable space that a misfit merely jumped
over -- is stranded in a partially filled stripe. In a forward fill
this happens at every size upshift, and measurement shows it is where
nearly all of stripe_alloc's trapped space comes from: on a mixed-size
fill it strands ~14% of the data written, a figure that band-granularity
tuning moves by at most a tenth and commit frequency does not move at
all.
Grow the frontier run instead. If the freshly claimed stripes directly
follow the end of an open run of the same class, extend that run to
absorb them and place the allocation at the run's old tail, spilling
contiguously into the new stripes. Nothing is stranded, and the extent
is physically contiguous. In a forward fill the by-size claim naturally
returns the stripes adjacent to the frontier, so the single frontier run
just keeps growing -- the allocator packs the way the stock allocator
does, while keeping stripe exclusivity: the grown stripes were claimed
fully free, the run still closes at the next commit and is never
reopened, so every stripe of the run is filled within one open-run
lifetime. A grown run keeps its open_seq; it can only still be in a
band slot if the retire walk for its window has not yet run, which the
open_stripe_lock nesting at the claim site excludes from racing.
Measured on a 4-device raid5 with a deterministic interleaved 4K-512K
fill (trapped space via the stripe_unusable counter, byte-identical data
both sides): 111.1 MiB trapped without growing, 7.2 MiB with -- a 94%
reduction, and usable capacity at ENOSPC within 0.5% of the stock
allocator (1561 vs 1569 MiB, vs 1421 MiB without growing). The benefit
survives churn: with age-correlated deletion the grown runs' contiguous
packing lets whole stripes free together, and steady-state usable
capacity stays at 1486-1561 MiB versus 1284-1421 MiB without growing
across correlated and adversarial small-file deletion patterns. Data
integrity verified by scrub and full sha256 read-back over mixed
random-content files, and trapped-space accounting still returns to
zero when all files are deleted.
The open-stripe selftest is updated: the misfit allocation now grows the
run and lands at the run's old tail, spanning the stripe boundary, and
the relocation-class case doubles as a check that a claim adjacent to
another class's run never grows it.
btrfs: stripe_alloc: keep an open run per size band to trap less
A single open stripe run per class forces every allocation that does not
fit the current run's remainder to close that run -- trapping its
unallocated tail in a now partially filled stripe -- and claim a fresh
fully-free stripe. A workload that interleaves small and large extents
(the common case) therefore strands a tail on every size change, even
though a later small allocation could have filled it.
Keep one open run per power-of-two band of remaining free space instead
(band k holds a run with remaining in [2^k, 2^(k+1)) sectors), per class.
An allocation takes the run in the smallest band that still fits --
segregated best fit -- so a small write lands in an already-open, nearly
full stripe rather than opening a new one, while a large write that no
open run can hold opens a fresh stripe and leaves the smaller runs open
for the small writes that do fit them. After each allocation the run is
re-placed into the band its new remainder falls in; when two runs collide
in a band the fuller one is kept -- it can serve larger future
allocations, and closing it would trap more -- and the other is closed,
its tail returned to the free space cache.
16 bands cover a full stripe of up to 2^16 sectors, far more than any
raid56 geometry (nr_data_stripes * stripe_len / sectorsize is 158 sectors
at 10 data stripes and a 64K stripe). Commit-time retirement and the
read-only / removal paths now walk every band of every class; only the
bg->open_stripe[][] slots can ever hold an open run, so one pass still
suffices, and the obsolete single-slot assertions are dropped.
This does not touch the write-hole guarantee: every run is still a
contiguous fill of fully-free stripes, closed at commit and never
reopened. It only changes which open run an allocation joins, reducing
the free space trapped in partial stripes -- and thus stripe_unusable --
for mixed-size workloads.
The open-stripe selftest is rewritten to the multi-cursor contract: a
misfit allocation now keeps the old run open (its tail is not returned to
the cache) and a later small allocation backfills it, so fewer stripes are
claimed before the free space is exhausted.
btrfs: account stripe_alloc trapped free space for honest statfs
Free space in the partially filled stripes of a stripe_alloc (raid56
write-hole-safe) block group is real free space -- it is in the free
space tree and cache -- but the stripe-exclusive allocator cannot hand it
out until the whole stripe frees. statfs therefore over-reports available
space, promising free space that a later allocation refuses with ENOSPC.
Track this trapped space per block group as stripe_unusable, summed into
space_info->bytes_stripe_unusable. It is derived, so there is no on-disk
format change and the free space tree and cache are left untouched
(preserving the extent-tree/free-space-tree consistency btrfs check
verifies).
stripe_unusable is defined by a scan that accumulates every free byte of
the block group into a per-stripe array and then sums the stripes that are
partially filled. A per-stripe accumulator, rather than a streaming sweep
or a per-stripe cache search, is what makes the result correct regardless
of the order the free space cache yields its ranges -- an offset-sorted
bitmap entry can emit runs that lie past a following extent entry, so the
ranges are not globally monotonic -- and regardless of whether free space
is stored as extents or bitmaps. A debug-build assertion checks that the
scan distributes exactly the cache's free space. Because trapped space
only settles at commit -- when the retire path returns partially filled
stripes and deleted extents are unpinned -- the scan is recomputed at
commit for groups whose free space changed (flagged cheaply on the
allocation and free paths), which suits the timescale of the reclaim it
feeds far better than a running per-extent tally. The space_info total is
recomputed as the sum of the armed groups after those rescans, so it
cannot drift.
statfs subtracts bytes_stripe_unusable from the data f_bavail so df
reports what can actually be allocated, and the counter is exposed at
/sys/fs/btrfs/<uuid>/allocation/data/bytes_stripe_unusable. It is
deliberately not part of btrfs_space_info_used(), so the reservation layer
is unchanged and this carries no ENOSPC-behaviour risk.
The counter is armed once a group's free space cache is loaded, so a group
contributes zero until then and statfs starts optimistic and settles to
honest as groups cache and commit. It is disarmed when a group turns
read-only (its free space is then accounted as read-only and already
excluded from statfs) and re-armed by a rescan on the way back to
read-write, so a balance both recovers trapped space and updates the
counter. A CONFIG_BTRFS_DEBUG-only sysfs trigger, stripe_unusable_rescan,
forces a full recompute for auditing the accounting.
Two deliberate imprecisions, matching existing behaviour rather than
bettering it: statfs subtracts bytes_stripe_unusable but not
bytes_zone_unusable, so stripe_alloc df reports availability before reclaim
while zoned df reports it after; and superblock stripes, permanently
unusable in every profile, are left to the existing bytes_super accounting
rather than separately reported here. Both await a maintainer decision on
a common convention.
btrfs: add a write-hole invariant checker to the raid56 write path
With stripe-exclusive allocation, a raid56 data stripe may only be
written while an open or draining stripe run covers it: after its run
retires and drains at a transaction commit, nothing may ever write to
it again, and a write outside any run means an allocation bypassed the
policy. Both cases are the write hole about to happen.
Check the invariant (under CONFIG_BTRFS_DEBUG) for every raid56 write
operation, full-stripe and sub-stripe alike, at rmw_rbio() time. This
turns every upstream violation -- a missed allocation path, a
retirement ordering bug, an accounting leak -- into a deterministic
WARN at the moment of the offending write, instead of silent damage
that needs a crash plus a device failure plus a scrub to observe. The
bios gathered in an rbio have not reported their IO done yet, so their
runs cannot drain under the check: no false positives from completion
races.
Block groups that ever hosted relocation-class runs are skipped
(sticky, debug-only flag): relocation legitimately overwrites its
preallocated extents in place after their runs drain, and the write
path cannot tell those writes from violations.
btrfs: retire stripe runs at commit and gate stripe_alloc
Hook stripe run retirement into the transaction commit, right after
TRANS_STATE_COMMIT_DOING stops accepting joins with a single writer
left. At that point every data extent the transaction references was
inserted by an ordered extent completion that joined earlier, implying
its own data IO is done; the retirement drain waits out in-flight
neighbour writes in the same stripes (a pure data-IO wait, never an
ordered extent wait, which would deadlock on the blocked join). After
the drain, no stripe this transaction references can ever be written
again, so a crash after the superblock write cannot tear it -- closing
the raid56 write hole for stripe-allocated block groups without
requiring flushoncommit. A defensive retirement in
btrfs_free_block_groups() covers the transaction abort path.
An ordered extent whose stripe run was opened after the committing
transaction's retire point cannot insert its file extent into that
transaction, so btrfs_finish_one_ordered() ends its handle and waits
for the commit's critical section to end. That wait must not be taken
under the range's extent lock: buffered writers block on the locked
range in lock_and_cleanup_extent_if_need() while holding their
prepared, locked folios, which stops the writeback flusher and
kcompactd behind those folios, which stops reclaim -- and the
committing transaction is itself draining data bios whose submission
path can need memory (dm-crypt bounce pages on the host where this was
caught). Drop the extent lock and the FINISHING_ORDERED tag before the
wait and re-take them before re-joining, preserving the
extent-lock-before-join ordering. The window is safe: the range is
still covered by the pending ordered extent, so a writer that takes the
lock finds it, releases its folios and waits -- which is exactly what
unbinds writeback and reclaim.
Force COW for in-place writes that would land in raid56 data block
groups while stripe_alloc is enabled (nodatacow files and writes into
preallocated extents): a single in-place write could tear a stripe
containing other files' committed extents, making the guarantee
conditional on the whole filesystem's usage. The check sits in
can_nocow_file_extent(), covering buffered and direct IO with one
choke point, and applies per extent so nocow to non-raid56 profiles
keeps working. The data relocation inode is exempt: its extents live
in relocation-class stripe runs that never share stripes with other
data, and relocation depends on in-place writes.
Validate the option at mount: it requires the free space tree (the v1
space cache overwrites its data in place during commit), and is
refused with the remap-tree feature (whose relocation writes bypass
the ordered extent accounting) and on zoned filesystems (which have
their own allocator and no write hole).
Mixed data+metadata filesystems are refused too. The immediate symptom
is a hang: a run drains when it is closed and its inflight_bytes reach
zero, and the only things that subtract are ordered extent completion, a
discarded allocation and a freed reservation. Metadata has no ordered
extent -- btrfs_alloc_tree_block() allocates with is_data false and
end_bbio_meta_write() reports nothing back to the run -- so a metadata
allocation raises inflight_bytes and nothing ever lowers it, the run
never drains, and btrfs_retire_open_stripes() waits for a completion
that has no code path to arrive from. The hung task detector stays
quiet, because wait_var_event() sleeps in a state it exempts; the
filesystem just stops. Reproduced deterministically on
mkfs.btrfs -M -d raid5 -m raid5 mounted -o stripe_alloc: the first sync
after a few hundred small files never returns.
That symptom is not why the refusal is permanent: a later patch
(stripe_meta) gives metadata its own completion report, and the drain
then terminates. These are why:
- raid56 deliberately skips csum lookup for mixed block groups, to
avoid recursing into a metadata read while holding the full stripe
lock (see the comment above the map_type test in fill_data_csums()).
That test reads the block group's flags, so in a mixed group it
cannot tell a data stripe from a metadata stripe and disables
verification for every one of them. Stripe-exclusive allocation
could keep the two apart -- allocation classes already never share a
run, so never a stripe -- but nothing durable records which class a
stripe held, and asking the extent tree while holding the full stripe
lock is the recursion the test exists to avoid. The recovery paths
here assume a rebuilt data sector can be checked; in a mixed group it
cannot.
- Data and metadata draw on one space_info and one free space pool.
Data reservations carry a pessimistic whole-stripe margin, metadata
reservations carry none, and a metadata claim that finds no fully
free full stripe returns ENOSPC -- which for metadata aborts the
transaction rather than failing one write. A data fill can starve
metadata into an abort, which separate block groups cannot do.
Mixed block groups are a mkfs-time property of small filesystems, where
raid56 is least appropriate and stripe_alloc's trapped space costs
proportionally most, and the option cannot become applicable later. So
do not carry a half-supported mode.
btrfs_is_stripe_alloc_bg() therefore requires DATA without METADATA as
belt and braces for a block group that somehow reaches the allocator
anyway, and moves out of extent-tree.c's file scope so block-group.c
can share the one copy of the rule. (Metadata and system block groups
never carried the DATA flag, so they were already excluded.)
btrfs: report stripe run data IO through the ordered extent lifecycle
Pair every byte reserved from an open stripe run with exactly one
"write done" report, so commit-time retirement can wait for all data IO
into a window's stripes:
- Ordered extents get a stripe_run pointer, attached at creation by a
range lookup (cheap: gated on the fs having any stripe runs at all,
and a run's block group membership is established before its
allocation returns, so the gate cannot miss). NOCOW and PREALLOC
ordered extents write into previously allocated extents, which can
never lie inside a run claimed from fully-free stripes, and are
skipped. The report fires once at IO completion (the IO_DONE moment
in can_finish_ordered_extent(), before any transaction join, so the
commit-time drain can never deadlock on a blocked join), with a
catch-all when an ordered extent is freed without completing IO.
- Reservations freed without an ordered extent ever owning them (error
paths) report through btrfs_free_reserved_extent(); the one caller
that frees a range an ordered extent did own (the finish-error path)
uses btrfs_free_reserved_extent_ordered() to avoid double reporting.
- Preallocated extents never issue data IO and report at insertion;
this also covers relocation's data inode preallocations.
btrfs: add the stripe_alloc allocation policy for raid56 data
Wire the open stripe run allocator into find_free_extent() as a per-
block-group policy: with the new stripe_alloc mount option, allocations
from raid56 data block groups go through btrfs_alloc_from_open_stripe()
instead of the clustered allocator, and only ever land in fully-free,
stripe-aligned runs. Since the loop's terminal LOOP_NO_EMPTY_SIZE
degradation only applies to the clustered path and stripe block groups
ignore empty_size/empty_cluster, the natural terminal behaviour is:
no fully-free stripe in any block group -> allocate a chunk -> ENOSPC.
The availability hint feeds max_extent_size so callers retry with
smaller allocations instead of failing early.
Stripe runs get an allocation class: relocation overwrites its
preallocated extents in place, so its allocations must never share a
stripe with ordinary cow data. Classes never share a run, and one
block group at a time is softly dedicated to relocation by reusing
fs_info->data_reloc_bg (btrfs_clear_data_reloc_bg() moves from zoned
code to generic code for this; the class tag, not the dedication, is
what carries correctness). The dedication is dropped when relocation
finishes or the dedicated group runs out of stripes.
Setting a block group read-only now retires its stripe runs (in
btrfs_inc_block_group_ro(), covering scrub, relocation and unused
block group deletion), pairing with the allocator's ->ro check under
the stripe_run_lock so no run can survive into or be created in a
read-only group.
btrfs: add open stripe run tracking for stripe-exclusive allocation
Add the in-memory state and lifecycle for "open stripe runs", the
allocation windows of the raid56 stripe-exclusive allocation policy. A
run is a contiguous stripe-aligned region claimed whole from the free
space cache via btrfs_claim_free_stripe_run() and filled strictly
sequentially by btrfs_alloc_from_open_stripe(). A run closes when it is
exhausted, when an allocation does not fit its remainder, or when the
transaction commit retires it; closed runs are never reopened and their
unallocated tails return to the free space cache, where the fully-free
claim rule makes them unallocatable until the whole stripe frees. This
is what will guarantee that a full stripe only receives writes within
one commit window, closing the raid56 write hole for these block groups.
Each run counts reserved bytes whose data IO has not completed yet,
maintained under the block group lock and reported back through
btrfs_open_stripe_write_done(). btrfs_retire_open_stripes() implements
commit-time retirement: bump the retire sequence, close every run opened
before it, and wait for their inflight bytes to drain. It is a pure
data-IO wait, deliberately not an ordered extent wait: it is designed to
run after the committing transaction stops accepting joins
(TRANS_STATE_COMMIT_DOING with a single writer), where waiting for
ordered extent completion would deadlock on the blocked transaction
join, and where every extent the transaction references already has its
own data on disk. Allocations racing with the commit open runs stamped
with a newer sequence and are neither retired nor waited for; their
extents can only be referenced by the next transaction. Block groups
with runs are tracked on an fs_info list whose membership is
established before an allocation returns, which is what lets the retire
walk rely on the sequence stamp.
Exercised by a new sanity self-test; the raid56 stripe allocation
policy and the commit hook will be the first non-test users.
btrfs: add btrfs_claim_free_stripe_run() for stripe-exclusive allocation
Add a free space cache primitive that finds and removes a contiguous,
fully-free, stripe-aligned run of full stripes from a block group. This
is the building block for a raid56 allocation policy that never issues
sub-stripe writes into stripes containing committed data, closing the
raid56 write hole for datacow writes: because a partially-filled stripe
can never satisfy the fully-free requirement, stripes retired at commit
time become unallocatable without any persistent allocator state.
The search walks the by-size free space index (largest max contiguous
free run first) rather than the by-offset tree: an entry whose largest
contiguous free run is smaller than a full stripe cannot contain a
fully-free stripe, and every following entry is no larger, so the search
stops at the first such entry. Near-full, where the free space
degenerates into many sub-stripe holes, that is an O(1) fast fail
instead of a scan of the whole free space tree on every allocation --
the dominant cost of the known raid56 near-full allocation slowdown.
Full stripe geometry is relative to the block group start and supports
non-power-of-two stripe widths. Unaligned head and tail remainders are
returned to the free space cache with their trim state preserved. Runs
are found within a single free space entry; a fully-free stripe split
across an extent entry and a bitmap neighbour is deliberately not found,
which errs toward missing a claimable stripe, never toward claiming a
non-free byte.
The function is exercised by new sanity self-tests covering aligned and
capped claims, partial-stripe exclusion, head/tail carving, block-group-
relative geometry, bitmap entries, and non-power-of-two stripe widths.
The raid56 stripe allocation policy will be its first non-test user.
Tal Zussman [Thu, 17 Sep 2026 04:00:13 +0000 (00:00 -0400)]
btrfs: remove TRANS_JOIN_NOLOCK
btrfs_join_transaction_spacecache() was the only user of
TRANS_JOIN_NOLOCK and is gone, so remove the join type, its entries in
the blocked types table, and the special cases in join_transaction() and
start_transaction().
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:12 +0000 (00:00 -0400)]
btrfs: stop reading free space inodes from the commit root
Free space inode data was only read when loading the v1 cache, which is
gone, so btrfs_get_extent() and btrfs_lookup_bio_sums() no longer need
to search the commit root for them.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:11 +0000 (00:00 -0400)]
btrfs: stop special-casing free space inodes in the delalloc accounting
Free space inodes never have delalloc or outstanding extents any more,
so they don't need to be kept off the root's delalloc inode list or out
of the outstanding extents tracepoint.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:10 +0000 (00:00 -0400)]
btrfs: remove the free space inode special cases from the COW paths
Free space inodes are never written anymore, so drop the special cases
for them in cow_file_range(), fallback_to_cow(),
can_nocow_file_extent() and btrfs_finish_one_ordered().
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:09 +0000 (00:00 -0400)]
btrfs: remove the free space inode ordered extent special cases
Free space inodes never have ordered extents anymore. Drop the lockdep
exceptions for them and btrfs_join_transaction_spacecache(), which was
only used to finish their ordered extents during a commit.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:07 +0000 (00:00 -0400)]
btrfs: remove the free space cache trimming ranges
cache_writeout_mutex and trimming_ranges let the v1 cache writer see
ranges that were unlinked from the free space tree while being
discarded. Nothing consumes the list anymore, and the tree itself is
protected by tree_lock, so remove them.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:05 +0000 (00:00 -0400)]
btrfs: remove the SPACE_CACHE mount option flag
Nothing sets BTRFS_MOUNT_SPACE_CACHE anymore, so every test of it is
false. Remove the flag, the checks rejecting the v1 cache on zoned
filesystems and for sector sizes other than the page size, and the
deprecation warning. Show a read-only filesystem that still has an old
cache as nospace_cache, since that's what's in effect. space_cache and
space_cache=v1 keep falling back to no space cache with a warning.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:04 +0000 (00:00 -0400)]
btrfs: remove btrfs_disk_cache_state
With neither the writer nor the loader left, nothing acts on
disk_cache_state. Remove it, the need_clear handling when reading block
groups, and the enum. While at it, drop the unused cache_generation
field from struct btrfs_block_group.
lookup_free_space_inode() converted old style space inodes by clearing
disk_cache_state so the cache would be rewritten with the new inode
flags. Without that it only sets flags on the in-memory inode, which
every remaining caller truncates or deletes right after, so drop the
conversion too.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:03 +0000 (00:00 -0400)]
btrfs: remove the v1 space cache load path
Nothing writes a v1 space cache any more, and since commit 545e560a5b0f
("btrfs: disable v1 space cache") the mount option can't be enabled to
read one either. Remove load_free_space_cache(), its io_ctl helpers and
struct btrfs_io_ctl. Drop the gfp constraint on the inode mapping as
well, it only covered the cache's page cache allocations.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:02 +0000 (00:00 -0400)]
btrfs: drop the transaction handle from the prealloc helpers
The v1 space cache created its inode during the transaction commit, and
btrfs_prealloc_file_range_trans() existed so that preallocation could
reuse the open handle. It was the only caller passing a transaction, so
__btrfs_prealloc_file_range() and insert_prealloc_file_extent() now
always start their own. Fold the wrapper into
btrfs_prealloc_file_range() and drop the parameter.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:01 +0000 (00:00 -0400)]
btrfs: rename cache_write_mutex to dirty_bgs_update_mutex
The v1 space cache writeout is gone, but the mutex is still needed. It
keeps btrfs_remove_block_group() from deleting a block group item while
btrfs_start_dirty_block_groups() is updating it outside the commit
critical section.
Rename it to reflect what it protects, and update the comments around
the dirty block group writeout that still refer to the space cache.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 04:00:00 +0000 (00:00 -0400)]
btrfs: remove the v1 space cache write path
Nothing writes out a v1 space cache any more. Remove the writers and
their io_ctl helpers, along with create_free_space_inode() and
btrfs_prealloc_file_range_trans(), whose only user was the cache inode
creation. The io_list and io_ctl block group fields were only used by
the writers, so remove them too.
btrfs_truncate_free_space_cache() only needed the block group to wait
for and clear in-flight cache IO, so drop that parameter.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 03:59:59 +0000 (23:59 -0400)]
btrfs: remove the free space cache endio workqueue
Free space inodes are no longer written to, so nothing queues ordered
extent completion on endio_freespace_worker. Remove it and always use
endio_write_workers in btrfs_queue_ordered_fn().
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 03:59:58 +0000 (23:59 -0400)]
btrfs: remove the v1 space cache writeout from the transaction commit
Nothing sets SPACE_CACHE anymore, so the dirty block group writers never
have a cache to write out or wait for. Remove cache_save_setup(),
btrfs_setup_space_cache(), the io_list handling, the io_bgs list and
BTRFS_TRANS_CACHE_ENOSPC, and the abort-time cleanup of in-flight cache
IO.
The -ENOENT retry in btrfs_write_dirty_block_groups() handled a free
space endio worker creating a block group during the commit critical
section, so drop it too.
Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Thu, 17 Sep 2026 03:59:57 +0000 (23:59 -0400)]
btrfs: stop enabling the v1 space cache from the on-disk state
Since commit 545e560a5b0f ("btrfs: disable v1 space cache") the mount
options can no longer request the v1 space cache, but a filesystem with
an active v1 cache and no free space tree still enables it from
cache_generation, and remount does the same. Drop both, so SPACE_CACHE
can never be set.
btrfs_start_pre_rw_mount() then sees the on-disk cache as active but
unwanted and cleans it up, as -o nospace_cache does today. That covers
the read-only to read-write remount as well, so drop the toggle in
btrfs_remount_cleanup(), which would otherwise start a transaction on
remounts of a read-only filesystem with an old cache.
The cleanup is now unconditional, and the first read-write mount fails
if it fails, as it did with -o nospace_cache. This also lets an old
filesystem mount without options when the page size is larger than the
sector size, which btrfs_check_features() rejected once SPACE_CACHE was
set from the superblock.
Assisted-by: Claude:claude-fable-5-1 Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Signed-off-by: David Sterba <dsterba@suse.com>
Yang Xiuwei [Wed, 19 Aug 2026 02:54:36 +0000 (10:54 +0800)]
btrfs: drop unused uring encoded IO REISSUE stash helpers
After not keeping state across -EAGAIN, restoring bc->data on REISSUE is
dead. Remove it, stop using the cmd PDU on the write path, and fold the
read -EAGAIN check into the existing error path.
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn> Signed-off-by: David Sterba <dsterba@suse.com>
Yang Xiuwei [Wed, 19 Aug 2026 02:54:35 +0000 (10:54 +0800)]
btrfs: don't stash uring encoded data across -EAGAIN
Returning -EAGAIN while leaving btrfs_uring_encoded_data in the cmd PDU
leaks if the request is cancelled or the ring exits before reissue.
io_uring does not free driver PDU allocations on cleanup.
Write: io_queue_sqe() always issues with IO_URING_F_NONBLOCK first, so
return -EAGAIN before allocating and free data on every exit.
Read: free on nowait -EAGAIN too; only -EIOCBQUEUED keeps the
allocation for btrfs_uring_read_finished().
Fixes: 34310c442e17 ("btrfs: add io_uring command for encoded reads (ENCODED_READ ioctl)") Fixes: e32dcdb0af9f ("btrfs: add io_uring interface for encoded writes") Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn> Signed-off-by: David Sterba <dsterba@suse.com>
Yang Xiuwei [Wed, 19 Aug 2026 02:54:34 +0000 (10:54 +0800)]
btrfs: unlock inode and extent in caller when uring read extent fails
btrfs_uring_read_extent() runs only after btrfs_encoded_read() has
taken the inode shared lock and the extent lock. On failure it used to
unlock in out_fail, and a pages-array allocation failure returned
-ENOMEM without unlocking at all.
Unlock in the caller instead on all failure returns, matching the
copy_to_user() error path. The deferred -EIOCBQUEUED path still unlocks
in btrfs_uring_read_finished().
Fixes: 34310c442e17 ("btrfs: add io_uring command for encoded reads (ENCODED_READ ioctl)") Suggested-by: Qu Wenruo <quwenruo.btrfs@gmx.com> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn> Signed-off-by: David Sterba <dsterba@suse.com>
Yang Xiuwei [Wed, 19 Aug 2026 02:54:33 +0000 (10:54 +0800)]
btrfs: free iov when btrfs_uring_read_extent fails
After btrfs_uring_read_extent(), the caller always jumped to out_acct.
That skips kfree(data->iov), which is only correct for -EIOCBQUEUED
where the deferred path owns the iov. On failure, fall through to
out_free instead.
Fixes: 34310c442e17 ("btrfs: add io_uring command for encoded reads (ENCODED_READ ioctl)") Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn> Signed-off-by: David Sterba <dsterba@suse.com>
Yang Xiuwei [Wed, 19 Aug 2026 02:54:32 +0000 (10:54 +0800)]
btrfs: always return -EIOCBQUEUED after btrfs_uring_read_extent_endio
If all bios finish before btrfs_encoded_read_regular_fill_pages()
returns, it calls btrfs_uring_read_extent_endio() and previously
returned the I/O status. A negative errno then made
btrfs_uring_read_extent() unlock and free while
btrfs_uring_read_finished() did the same again.
Return -EIOCBQUEUED so only the deferred path cleans up.
Reported-by: Yue Sun <samsun1006219@gmail.com> Closes: https://lore.kernel.org/linux-btrfs/20260630091609.3414-1-samsun1006219@gmail.com/ Suggested-by: Jens Axboe <axboe@kernel.dk> Fixes: 34310c442e17 ("btrfs: add io_uring command for encoded reads (ENCODED_READ ioctl)") Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn> Signed-off-by: David Sterba <dsterba@suse.com>
Boris Burkov [Wed, 16 Sep 2026 22:17:24 +0000 (15:17 -0700)]
btrfs: keep unused block groups queued when a pass fails
Once any block_group sets ret!=0 in the main loop of
btrfs_delete_unused_bgs(), the check
if (ret || btrfs_mixed_space_info(space_info)) {
btrfs_put_block_group(block_group);
continue;
}
skips the rest of the unused bgs while unlinking them from
fs_info->unused_bgs. There is no "level triggered" re-queueing of empty
block groups onto fs_info->unused_bgs so it is possible to leak quite a
bit of space this way and unless we happen to get a balance or
re-use/re-empty one of these bgs, they are leaked for good, which can
lead to a spurious enospc later.
While I have observed such leaked blocked groups that are empty but not
on the unused_bgs list on production systems, I have not observed that
it is definitely due to this issue. I also reproduced this behavior by
injecting an ENOSPC error from btrfs_start_trans_remove_block_group
which can also fail with ENOMEM, so this feels like a legitimate
injection point.
To fix it, instead of checking ret in the loop, just break out of the
loop when ret != 0. Also, link the bg to the retry list at the
individual failure sites so that the failing bg is not leaked.
Assisted-by: LLM (reproducer/error injection) Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Boris Burkov <boris@bur.io> Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 16 Sep 2026 16:20:52 +0000 (17:20 +0100)]
btrfs: remove redundant eb generation check in btrfs_buffer_uptodate()
It's pointless to check if the extent buffer's generation does not match
the value of 'parent_transid' because if it does, then we have already
entered the previous if statement and returned from the function.
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 16 Sep 2026 15:49:37 +0000 (16:49 +0100)]
btrfs: check if there is space for chunk item when validating sys chunk array
We checked if have enough remaining space for a key before dereferencing a
key, but we then dereference a chunk item, to get the number of stripes,
without checking if there is space for the item. So add a check to see if
there is enough space for a chunk item before dereferencing the item to
extract the stripe count.
Fixes: 2a9bb78cfd36 ("btrfs: validate system chunk array at btrfs_validate_super()") Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 16 Sep 2026 14:43:41 +0000 (15:43 +0100)]
btrfs: abort transaction on failure to update inode for hole punching and reflinking
If we fail to update the inode we error out without aborting the
transaction, which can result in a persistent inconsistency if after
the failure the transaction is committed, as we have dropped file
extent items from a range and either punched a hole or insert a new file
extent item for that range (for reflinks).
So add the missing transaction abort.
Fixes: 2aaa66558172 ("Btrfs: add hole punching") Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
btrfs: add "/dev/root" exception for device path update
[BEHAVIOR CHANGE]
Since commit 108cc8733989 ("btrfs: fix a lockdep caused by path
resolution during device scan"), users with btrfs rootfs but without an
initramfs are complaining that grub2 can no longer detect the rootfs
device:
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
[CAUSE]
Although using btrfs without an initramfs is not recommended (if a new
device is added to the rootfs, the system can no longer boot, as there
is no way to register all devices), there is still a minority of users
doing this.
If there is no initramfs but the rootfs is on a block-device-based
filesystem, the kernel boot sequence initializes a minimal ramfs/tmpfs,
creates "/dev/root" with the proper device number for the rootfs, and
then invokes mount using "/dev/root".
That's why the end user will get the mount output:
/dev/root on / rw
To be honest, this is a user space problem: no one should trust the
device path shown in mount, only the device number.
E.g. one can even use "/proc/self/fd/*" to mount an fs, and that proc
path will be registered, and no one else can mount that fs using that
path.
Before commit 108cc8733989 ("btrfs: fix a lockdep caused by path
resolution during device scan"), btrfs had an internal path lookup
workaround to address such weird paths, it works by checking if the
existing device path can still resolve to the device number.
But that path resolution is deadlock prone, thus it's replaced by a
simple devt check.
This works fine in most cases, as a btrfs device is registered by udev at
boot time, thus all paths are sane.
However this will not work for systems without an initramfs, causing the
unreachable "/dev/root" path to exist forever without a way to rename
it.
[WORKAROUND]
Despite updating the docs to discourage root btrfs without an initramfs,
add an exception to the device path rename requirement.
If the device has the name "/dev/root", we know it's booted without
an initramfs, and only for that case we allow device path update.
And if someone intentionally created "/dev/root" after boot, the
existing devt checks will reject that weird name as usual.
This should satisfy the minority of users, and still keep most of the
existing guards preventing unexpected/unnecessary device path updates.
But still, I prefer grub2 to implement a more robust device
number based probing, and no one should use btrfs as rootfs without an
initramfs.
Guanghui Yang [Wed, 16 Sep 2026 05:16:38 +0000 (05:16 +0000)]
btrfs: clear free space tree creation state on rebuild failure
btrfs_rebuild_free_space_tree() sets BTRFS_FS_CREATING_FREE_SPACE_TREE
before rebuilding the free space tree. Several error paths return
without clearing this flag.
The transaction restart failure path can leave the flag set on a live
filesystem, causing delayed reference processing to be skipped. Clear it
on all free space tree rebuild failure paths. Keep
BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED set, since a failed rebuild leaves
the free space tree untrusted. Callers must fall back to extent-tree
caching.
Fixes: 882af9f13e83 ("btrfs: handle free space tree rebuild in multiple transactions") CC: stable@vger.kernel.org # 6.14+ Assisted-by: LLM Reviewed-by: Boris Burkov <boris@bur.io> Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Guanghui Yang <3497809730@qq.com> Signed-off-by: David Sterba <dsterba@suse.com>
Daniel Linjama [Wed, 16 Sep 2026 06:15:56 +0000 (09:15 +0300)]
btrfs: handle lack of space when cleaning up verity items
When enable_verity() hits the qgroup limit, rollback_verity() needs its
own metadata reservation. When the qgroup limit or lack of space refuses
the rollback, the whole filesystem is forced read-only even though the
qgroup limit was for one subvolume only. Also orphan cleanup at the next
mount fails the same way, so the leftover items are never removed: with
-EDQUOT the subvolume stays unreachable, and with -ENOSPC on a full
filesystem the next read-write mount fails.
Start transactions with btrfs_start_transaction_fallback_global_rsv() in
btrfs_orphan_cleanup(), drop_verity_items() and rollback_verity(). Those
calls only delete items and free the space in the end, so they may use
the global reserve and skip the qgroup limit, which avoids -ENOSPC and
-EDQUOT.
Fixes: 146054090b08 ("btrfs: initial fsverity support") Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Daniel Linjama <daniel@dev.linjama.com> Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Mon, 14 Sep 2026 17:11:30 +0000 (18:11 +0100)]
btrfs: fix creation of compressed inline extents that don't save space
If the compressed data of an inline extent is larger than or equals to the
size of the uncompressed data, we are still allowing the creation of the
compressed inline extent, which does not result in any benefits, quite the
contrary as we waste metadata space and have to decompress when reading.
This is a recent regression introduced in commit 3eaf5f082c4c ("btrfs:
extract inlined creation into a dedicated delalloc helper").
It happens because we are passing the block size to btrfs_compress_bio(),
so we don't get -E2BIG from the compression code anymore, but we can not
pass i_size either, because if i_size is smaller than sector size, we
end up never creating lzo compressed inline extent for such small i_size
values. So refuse the compressed result at run_delalloc_inline() if
its size is not smaller than the uncompressed size (i_size).
Reported-by: Hanabishi <i.r.e.c.c.a.k.u.n+kernel.org@gmail.com> Link: https://lore.kernel.org/linux-btrfs/c97652a5-ac6b-4de6-aa23-3cdebc01d00b@gmail.com/ Fixes: 3eaf5f082c4c ("btrfs: extract inlined creation into a dedicated delalloc helper") CC: stable@vger.kernel.org # 7.1+ Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
btrfs: simplify heuristic_collect_sample() to handle large folios better
Currently heuristic_collect_sample() is purely page size based, and it
has a lot of extra handling just inside the page.
However we already have large folio support, there is no need to look up
the same folio repeatedly.
Simplify the handling by:
- Use @cur as the iterator instead of page index
- Handle the sample copying on a per-folio basis
Although kmap_local_folio() requires an offset to handle HIGHMEM
page mapping, we have rejected large folios for HIGHMEM systems
completely.
So we can safely handle all sample copying inside the folio in one
go.
- Remove unnecessary unaligned range handling
All the range passed in should be block aligned, thus there is no need
to handle cases where sample crosses the block boundary.
Reviewed-by: Boris Burkov <boris@bur.io> Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
btrfs: fix off-by-one end related to inode_need_compress()
In most cases btrfs uses @end as the inclusive end bytenr for a range,
and this applies to inode_need_compress().
However we have several sites not following the inclusive bytenr:
- run_delalloc_inline()
Which assigned @blocksize as @end for inode_need_compress()
This makes inode_need_compress() always skip the disk_i_size check.
- heuristic_collect_sample()
Which assigned "start + BTRFS_MAX_UNCOMPRESSED" to @end, which is
the exclusive bytenr.
Neither is really causing any real problem, as
heuristic_collect_sample() has proper checks to avoid reading anything
beyond @end, and the sampling read size is 16 bytes, so it has enough
headroom to handle that off-by-one problem.
But still I do not like anything out of the common scheme, so fix the
off-by-one @end for both call sites, and add extra ASSERT()s to catch
such unaligned parameters.
Reviewed-by: Boris Burkov <boris@bur.io> Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
- Remove @size parameter
Since commit b2e743927fdd ("btrfs: make btrfs_do_readpage() to do
block-by-block read"), all callers are passing sectorsize as @size,
so there is no need for such parameter.
Furthermore since we only write one block at a time, there is no need
for a while() loop, nor the advance of various local variables.
- Update the comments on the parameter list
* @disk_bytenr is shared for both read and write
* rename @page to @folio
- Update the return value to return 0 or error
Since we won't queue multiple blocks anyway, there is no point in
returning the queued bytes.
It makes more sense to return an error code, although the only error
code will be -EUCLEAN for writes.
- Rename submit_extent_folio() to submit_one_block()
- Rename submit_one_sector() to submit_write_sector()
- Update the error message to utilize the new returned error code inside
submit_write_sector()
Reviewed-by: Boris Burkov <boris@bur.io> Signed-off-by: Qu Wenruo <wqu@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Fri, 11 Sep 2026 10:56:29 +0000 (11:56 +0100)]
btrfs: tree-checker: cache accessor return value in CHECK_FE_ALIGNED()
We are calling an accessor for a file extent item multiple times in the
CHECK_FE_ALIGNED() macro, even when we don't find a corruption (once in
the if statement's expression and then once again in the expression for
the return value). This adds extra runtime overhead (which is critical
since the tree checker runs against every extent buffer when it's read
or before persisting it) and increases the module's size.
So cache the accessor's return value in a variable and use it, reducing
runtime, object size and making the source code shorter too. Also avoid
repeating twice the IS_ALIGNED() computation.
Before:
$ size fs/btrfs/btrfs.ko
text data bss dec hex filename 2073340 217928 15624 2306892 23334c fs/btrfs/btrfs.ko
After:
$ size fs/btrfs/btrfs.ko
text data bss dec hex filename 2073076 217928 15624 2306628 233244 fs/btrfs/btrfs.ko
Also running the following fsstress test and capturing the runtime of
check_extent_data_item() (the only caller of CHECK_FE_ALIGNED()) in
nanoseconds (using bpftrace), showed the following runtime improvements:
Filipe Manana [Thu, 10 Sep 2026 16:37:48 +0000 (17:37 +0100)]
btrfs: tree-checker: print dev extent offset in error message
If a dev extent's offset is not sector size aligned, the error message is
printing the dev extent's objectid instead of the offset. This is a copy
paste error, as before this check we check the objectid field.
Fixes: 008e2512dc56 ("btrfs: tree-checker: add dev extent item checks") Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 9 Sep 2026 11:26:30 +0000 (12:26 +0100)]
btrfs: increment extent count once when logging extents during fast fsync
In btrfs_log_changed_extents() we increment the extent count twice, and
we then fallback to a transaction commit if the count reaches a threshold
of 32K. However we increment the count twice, which is confusing and
pointless. So increment the count only once and reduce the threshold to
half (16K).
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Wed, 9 Sep 2026 17:10:48 +0000 (13:10 -0400)]
btrfs: use u64 for the page indices in heuristic_collect_sample()
index and index_end are derived from the u64 start and end offsets, and
index is shifted back into a byte offset for offset_in_folio(), which
needs a cast to u64 to be safe on 32-bit. Make them u64 instead so the
cast goes away. They still fit pgoff_t where they are passed to
filemap_get_folio(), as they came from a valid file offset.
Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Mon, 7 Sep 2026 20:20:01 +0000 (16:20 -0400)]
btrfs: use folios for reading super blocks from the block device
btrfs_read_disk_super() and the zoned super block log comparison go
through read_cache_page_gfp() and page_address(), and
btrfs_release_disk_super() recovers the page with virt_to_page(). Use
mapping_read_folio_gfp(), folio_address(), and virt_to_folio() instead.
This removes the last callers of read_cache_page_gfp() and put_page()
in btrfs.
Compute the super block address with offset_in_folio() as
write_dev_supers() does, rather than assuming it is at the start of
the page.
Assisted-by: Claude:claude-fable-5-1 Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Mon, 7 Sep 2026 20:20:00 +0000 (16:20 -0400)]
btrfs: fix stale function references in compression comments
add_ra_bio_pages() was renamed to add_ra_bio_folios(), and
btrfs_compress_filemap_get_folio() wraps filemap_get_folio(), not
find_get_page(). Update the comments accordingly.
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Mon, 7 Sep 2026 20:19:59 +0000 (16:19 -0400)]
btrfs: convert heuristic_collect_sample() to use folios
Convert the sampling loop to folios. This removes the last caller of
find_get_page() in btrfs and saves a call to compound_head() per sampled
page. Document that the lookup is not supposed to fail with an ASSERT().
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Mon, 7 Sep 2026 20:19:58 +0000 (16:19 -0400)]
btrfs: convert btrfs_compr_pool_scan() to use folios
The compression pool holds order-0 folios, but btrfs_compr_pool_scan()
walks it as struct page through page->lru. Walk it as folios, matching
the other compression pool functions. This removes the last use of
page->lru in btrfs and saves a call to compound_head() per freed
folio.
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Mon, 7 Sep 2026 20:19:57 +0000 (16:19 -0400)]
btrfs: tests: use eb folio helpers in extent buffer memory checks
dump_eb_and_memory_contents() and verify_eb_and_memory() hardcode one
page per folio instead of using get_eb_folio_index() and
get_eb_offset_in_folio() like the rest of the extent buffer code. Use
the helpers and folio_address(). This removes the last struct page usage
in the file.
No functional change. The tests only run with sectorsize == PAGE_SIZE,
and the test extent buffers are backed by order-0 folios.
Assisted-by: Claude:claude-fable-5-1 Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Tal Zussman [Mon, 7 Sep 2026 20:19:56 +0000 (16:19 -0400)]
btrfs: tests: convert test_find_delalloc() to use folios
This removes the last btrfs callers of find_or_create_page(),
find_lock_page(), SetPageDirty(), ClearPageDirty(), and get_page(), and
15 calls to compound_head(). The folio lookups return an ERR_PTR instead
of NULL, so adjust the error handling.
Update the comments and test messages accordingly.
The test still works in PAGE_SIZE units, which relies on the test inode
never getting large folios, so assert that the folios are order-0 where
that matters.
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Tal Zussman <tz2294@columbia.edu> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Hongling Zeng [Mon, 31 Aug 2026 05:38:01 +0000 (13:38 +0800)]
btrfs: take commit root semaphore when iterating in mark_block_group_to_copy()
mark_block_group_to_copy() iterates over the commit root with
skip_locking=true. A concurrent transaction commit can swap and free
the commit root during iteration, causing use-after-free when
accessing extent buffers.
Fix it by using path->need_commit_sem to protect the commit root search.
Fixes: 78ce9fc269af ("btrfs: zoned: mark block groups to copy for device-replace") CC: stable@vger.kernel.org Assisted-by: Codex:gpt-5.5 Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Hongling Zeng <zenghongling@kylinos.cn> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
btrfs: use kvmalloc() to allocate compression workspace buffer for zlib and zstd
With the experimental bs > ps support, the workspace buffer for both
zlib and zstd can be as large as 64K, and on 4K page sized systems such
kmalloc() calls have a much higher chance to fail, as that requires
physically contiguous memory to fulfill such allocation.
The same also applies to S390's hardware accelerated path, which
requires a buffer size of 4 pages.
Meanwhile lzo is already using kvmalloc() for its buffer, and there is
no special requirement for any physically contiguous memory anyway.
So change the zlib and zstd workspace buffer allocation to use
kvmalloc() to reduce the chance of memory allocation failure.
Reviewed-by: Daniel Vacek <neelx@suse.com> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Although btrfs doesn't support inlined extents larger than PAGE_SIZE
for bs > ps cases, it's still possible for the experimental bs > ps
support to mount a btrfs created on a system with a much larger page size,
thus can still hit an inlined extent that is way larger than the current
page size.
E.g. a compressed inline extent which has 32K compressed size, is created
on 64K page sized ARM64 with 64K sectorsize, then mounted on x86_64 with
the experimental bs > ps support.
In that case, when reading the compressed inline extent, we need to
allocate a buffer that is the same size as the compressed inline extent
(32K).
That kmalloc() call will request physically contiguous memory for that
32K allocation, and if the system has a very fragmented memory space,
such allocation can fail.
But there is really no reason that we require such buffer to be
physically contiguous, so change it to kvmalloc() to reduce the chance
of allocation failure for bs > ps cases.
And for all bs <= ps cases, the kvmalloc() call will just be fulfilled by
kmalloc() so this will not bring any change to the most common cases.
Only bs > ps will get the benefit of less memory allocation failure.
Reviewed-by: Daniel Vacek <neelx@suse.com> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
btrfs: tree-log: use kvmalloc() for overwrite_item()
The @src_copy buffer utilized inside overwrite_item() can be as large as
the nodesize.
For an existing btrfs with 64KiB nodesize, it means there is a high
chance to fail the kmalloc() call if there is not enough physically
contiguous pages.
Meanwhile there is really no need for such physically contiguous pages,
as we only use that buffer to compare the content of the item.
Use kvmalloc() to replace the kmalloc() call. For most cases that
kvmalloc() call will be easily fulfilled by regular kmalloc(), but for
really large items and large nodes, kvmalloc() will have a much higher
chance to get memory allocated.
Reviewed-by: Daniel Vacek <neelx@suse.com> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a bug report that the kmalloc() call inside split_item() failed
with the following call trace, and triggered a transaction abort:
kworker/u69:8: page allocation failure: order:4, mode:0x40c40(GFP_NOFS|__GFP_COMP), nodemask=(null)
CPU: 3 UID: 0 PID: 1154528 Comm: kworker/u69:8 Not tainted 7.0.2 #1 PREEMPTLAZY
Workqueue: events_unbound btrfs_async_reclaim_metadata_space
Call Trace:
<TASK>
dump_stack_lvl+0x47/0x60
warn_alloc.cold+0x67/0xec
__alloc_pages_slowpath.constprop.0+0x9bf/0xed0
__alloc_frozen_pages_noprof+0x1ac/0x1c0
___kmalloc_large_node+0x9d/0xc0
__kmalloc_noprof+0x17b/0x1f0
split_item+0x9e/0x2e0
btrfs_del_csums+0x285/0x400
__btrfs_free_extent.isra.0+0x6de/0x12b0
__btrfs_run_delayed_refs+0x522/0x10c0
btrfs_run_delayed_refs+0x4d/0x1d0
flush_space+0x34d/0x4e0
do_async_reclaim_metadata_space+0x89/0x1d0
btrfs_async_reclaim_metadata_space+0x44/0x60
process_one_work+0x145/0x230
worker_thread+0x185/0x2e0
kthread+0xca/0x100
ret_from_fork+0x14e/0x200
ret_from_fork_asm+0x11/0x20
</TASK>
BTRFS error (device dm-3 state A): Transaction aborted (error -12)
BTRFS: error (device dm-3 state A) in btrfs_del_csums:1053: errno=-12 Out of memory
BTRFS info (device dm-3 state EA): forced readonly
BTRFS: error (device dm-3 state EA) in do_free_extent_accounting:3168: errno=-12 Out of memory
BTRFS error (device dm-3 state EA): failed to run delayed ref for logical 1202913873920 num_bytes 274432 type 184 action 2 ref_mod 1: -12
BTRFS: error (device dm-3 state EA) in btrfs_run_delayed_refs:2247: errno=-12 Out of memory
[CAUSE]
The kmalloc() call is to allocate a buffer to store the full item.
However as shown in the above call trace, the order can be high (4), and
since we're using GFP_NOFS, it's impossible to reclaim memory by writing
back dirty pages.
When there is no physically contiguous memory left, such high order
allocation can easily fail, and if such kmalloc() happens in a critical
path we can trigger a transaction abort.
[FIX]
Instead of kmalloc(), which requires physically contiguous pages, use
kvmalloc().
There is no special requirement for physically contiguous pages here, we
just want virtually contiguous memory as a buffer.
Reported-by: xavierbachmeyer182 <xavierbachmeyer182@protonmail.com> Link: https://lore.kernel.org/linux-btrfs/250decb0-d940-4fe6-9b54-d06e1b293a1b@suse.com/ Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by: Daniel Vacek <neelx@suse.com> Signed-off-by: Qu Wenruo <wqu@suse.com> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Zhen Ni [Mon, 22 Dec 2025 03:59:42 +0000 (11:59 +0800)]
btrfs: replace is_data_bbio() with is_data_inode() for direct usage
After commit 81cea6cd7041 ("btrfs: remove btrfs_bio::fs_info by
extracting it from btrfs_bio::inode"), the btrfs_bio::inode field is
mandatory for all btrfs_bio allocations. The NULL check is redundant and
can be removed.
As is_data_bbio() would be a trivial wrapper for is_data_bbio() replace
all calls in in bio.c
Usama Arif [Fri, 4 Sep 2026 16:41:48 +0000 (09:41 -0700)]
btrfs: zstd: avoid a copy in zstd_decompress_bio()
zstd_decompress_bio() gives zstd a sectorsize-sized scratch buffer, and
btrfs_decompress_buf2page() then copies the part overlapping the read bio
into the destination folios. Every delivered byte is written twice.
Instead, choose the output buffer per streaming call. zstd_map_dest()
kmaps the current page-bounded segment of the read bio, so zstd writes
into the page cache directly. The scratch buffer is kept only for output
with no destination: the prefix before a read starting inside a
compressed extent, which zstd cannot skip, and gaps left by folios
already in the page cache.
Varying the output buffer across calls is safe: btrfs uses the default
ZSTD_bm_buffered mode, where the sliding window lives in the dstream's
internal buffer and the caller's dst is a pure sink. The read bio's
iterator must still advance by exactly the bytes delivered, since
btrfs_decompress_bio() zero-fills from it; that used to happen inside
btrfs_decompress_buf2page() and is now an explicit bio_advance(), made
only for output that reached a folio.
bio_iter_iovec() exposes at most one base page, so direct output is
page-bounded. Compared to the old sectorsize-sized chunks, this can
increase stream calls when sectorsize exceeds PAGE_SIZE, but eliminates
the extra btrfs copy for output delivered to the read bio; the 64 KiB
sectorsize row below shows the copy still wins there.
Benchmarked the change in 2-vCPU x86-64 KVM guests (4 KiB pages, RAM
disk) using a 64 MiB zstd-compressed file. Results are medians of seven
cold-cache reads in each of six interleaved A/B boot pairs; mincore
confirmed zero resident pages before every run.
Normal sequential reads with readahead produced:
sectorsize base patched reduction
4 KiB 8.678 ms 8.004 ms 7.80%
16 KiB 8.216 ms 7.934 ms 3.64%
64 KiB 7.875 ms 7.344 ms 6.88%
Random 4 KiB preads at 4 KiB sectorsize, means of six interleaved A/B
boot pairs, patched better in all six:
base patched gain
264.33 MB/s 272.67 MB/s 3.2%
Signed-off-by: Usama Arif <usama.arif@linux.dev> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Layton [Tue, 25 Aug 2026 16:04:20 +0000 (12:04 -0400)]
btrfs: pre-allocate delayed dir index for non-overwrite rename
For rename() without an overwrite target, pre-allocate the delayed
dir index before any btree modifications so that ENOMEM can be returned
before the source is unlinked from the old directory.
Add a prealloc parameter to btrfs_add_link() that allows callers to
pass pre-allocated delayed dir index resources. When provided,
btrfs_add_link() takes ownership: it either passes the prealloc to
btrfs_insert_dir_item() (which commits or frees it), or frees it
on early error. All existing callers pass NULL to preserve the current
behavior.
In btrfs_rename(), when new_inode is NULL (no overwrite), call
btrfs_prealloc_delayed_dir_index() before the first btree modification
and pass the result through to btrfs_add_link(). If the prealloc fails,
-ENOMEM is returned before any btree state has changed. The local
prealloc pointer is cleared once ownership passes to btrfs_add_link(),
so the out_fail path only frees one we still own.
For overwrite rename (new_inode != NULL), the transaction still aborts
on ENOMEM since earlier unlink operations have already made irreversible
btree modifications.
Assisted-by: LLM Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Layton [Tue, 25 Aug 2026 16:04:19 +0000 (12:04 -0400)]
btrfs: handle ENOMEM from btrfs_insert_dir_item() without aborting
Now that btrfs_insert_dir_item() returns -ENOMEM before modifying the
btree (thanks to delayed dir index pre-allocation), callers can handle
ENOMEM gracefully instead of aborting the transaction.
- btrfs_add_link(): add -ENOMEM to the recoverable errors alongside
-EEXIST and -EOVERFLOW.
- btrfs_create_new_inode(): on -ENOMEM from btrfs_add_link(), orphan the
newly-created inode instead of aborting. The inode item was already
written with nlink 1, and discard_new_inode() marks it bad so eviction
won't delete it. So clear_nlink() alone is not enough: persist nlink 0
via btrfs_update_inode(), otherwise orphan cleanup would see nlink > 0,
drop the orphan item, and leak the inode. Fall back to aborting only if
that update also fails.
This turns a filesystem-killing abort into a graceful -ENOMEM return for
create(), mkdir(), mknod(), symlink(), and link() under memory pressure.
Assisted-by: LLM Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Layton [Tue, 25 Aug 2026 16:04:18 +0000 (12:04 -0400)]
btrfs: pre-allocate delayed dir index before btree modification
Move the delayed dir index allocation in btrfs_insert_dir_item() before
the insert_with_overflow() call that modifies the btree. Previously, the
allocations happened after the DIR_ITEM was already inserted, meaning an
ENOMEM failure left the btree in a partially-modified state that could
only be resolved by aborting the transaction.
Add an optional caller-provided btrfs_dir_index_prealloc parameter to
btrfs_insert_dir_item(). When non-NULL, ownership of the prealloc
transfers to btrfs_insert_dir_item(). When NULL, it allocates internally.
All existing callers pass NULL to preserve the current behavior.
Since ownership transfers, btrfs_insert_dir_item() must free the prealloc
on every path that does not commit it. Route all such exits (including
the early path allocation failure) through a common out_free_prealloc
label, rather than keying cleanup on need_delayed_index.
Remove the btrfs_insert_delayed_dir_index() wrapper, as there are no
more callers.
Assisted-by: LLM Suggested-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Layton [Tue, 25 Aug 2026 16:04:17 +0000 (12:04 -0400)]
btrfs: split btrfs_insert_delayed_dir_index() into prealloc and commit phases
Split btrfs_insert_delayed_dir_index() into three functions using a new
btrfs_dir_index_prealloc struct to bundle the pre-allocated resources:
- btrfs_prealloc_delayed_dir_index(): allocates the struct and performs
the two GFP_NOFS allocations (delayed node + delayed item) that can
fail with -ENOMEM. Returns the struct, or ERR_PTR on failure.
- btrfs_insert_delayed_dir_index_prealloc(): populates the item data,
inserts into the rb-tree, and reserves metadata space. Cannot fail
with -ENOMEM since all allocations were done in the prealloc step.
- btrfs_free_delayed_dir_index_prealloc(): frees pre-allocated
resources when the caller's btree insertion fails. Tolerates NULL.
The prealloc is returned as a pointer rather than filled into a
caller-provided struct, so that a plain NULL means "no prealloc" and
callers do not need a separate flag to track whether one exists. It is
consumed (and freed) by either the commit or the free helper, so
ownership is unambiguous.
The original btrfs_insert_delayed_dir_index() is refactored into a thin
wrapper that calls the prealloc and commit functions.
This split allows callers to move the fallible memory allocations before
the point of no return (the DIR_ITEM btree insertion), so that -ENOMEM
can be returned cleanly without aborting the transaction.
Assisted-by: LLM Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Daniel Vacek [Wed, 2 Sep 2026 13:56:27 +0000 (15:56 +0200)]
btrfs: use bio::remaining for async checksumming synchronization
We can use bio::remaining counter to sync the offloaded checksumming.
As a result we can slim down the btrfs_bio structure by 24 bytes
and simplify the code a bit.
Moreover this will allow us enabling async checksumming with encryption
where we need to checksum the bounce bio instead of our regular one
embedded in btrfs_bio. And so we need to extend it's lifetime. This is
the preferred way to do so.
This also fixes a bug in experimental build where the async checksumming
was using the system workqueue instead of fs_info::endio_workers.
Fixes: dd57c78aec39 ("btrfs: introduce btrfs_bio::async_csum") Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Daniel Vacek <neelx@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
Daniel Vacek [Wed, 2 Sep 2026 09:20:24 +0000 (11:20 +0200)]
btrfs: consume given iter directly instead of copying in csum_one_bio()
Avoid copying the iter twice in async case. We already have a copy
csum_one_bio() can consume directly. No need to copy it again the second
time. We can use this copy also in the sync case and get rid of the
parameter.
Reviewed-by: Qu Wenruo <wqu@suse.com> Signed-off-by: Daniel Vacek <neelx@suse.com> Signed-off-by: David Sterba <dsterba@suse.com>
projects / linux / log