Data reservations are admitted against arithmetic -- free space minus
trapped fragments minus open-run remainders minus a per-extent margin --
and under sustained near-full churn the arithmetic and the claim rule
disagree for long enough that admitted buffered writes reach writeback
with nothing claimable left: measured, ~5.2-5.5k reserved writebacks
dropped per fsstress churn run, silently for anything not waiting on
fsync. Stock refuses the same write()s up front. Nothing between
"durable by fsync" and "refused by write()" is acceptable when an
operator accident fills the disk.
First step, accounting only: measure the constraint instead of deriving
it. bg->stripe_claimable counts bytes of fully free whole stripes --
exactly what btrfs_claim_free_stripe_run() can take, the complement of
stripe_unusable within each stripe. The commit rescan derives it from
the same pass that computes trapped bytes (free minus trapped) and is
the sole upward correction; incremental maintenance under
ctl->tree_lock only ever DEBITS -- removals round their decrement OUT
to every touched stripe, claims subtract exactly what they took. The
counter can therefore only under-count between commits, never
over-count.
Crediting freed whole stripes incrementally on the add side is
deliberately not done, having been tried and dropped. Several add
paths re-add free space that a low-level remove never de-credited --
the async discard trim (unlink_free_space / bitmap_clear_bits, then
do_trimming's re-add through __btrfs_add_free_space) and
btrfs_remove_free_space's middle-split tail re-add -- so crediting on
add double-counts and drives the counter ABOVE the authoritative scan.
That is the dangerous direction: once admission gates on this counter,
an over-count admits reservations against phantom supply that writeback
then drops, while an under-count is an early, clean write()-time
ENOSPC. Debit-only makes over-counting structurally impossible,
whatever a future re-add path forgets to de-credit. (Reproduced with
compress+autodefrag+discard=async on a legacy-converted raid5
filesystem, incremental ~= 2x scanned; credit_return attribution
confirmed the phantom entered through do_trimming's re-add, not the
open-stripe allocator returns.)
A WARN_RATELIMIT at the rescan catches the dangerous direction anyway
(incremental above scanned), which under a debit-only rule means a
missed consumption site, and localizes it; the clamp to the scanned
value keeps every rescan authoritative regardless. Rate limited rather
than _ONCE because such a gap recurs on every commit, and reporting
only the first hit hides that it is ongoing.
The space_info aggregate bytes_stripe_claimable follows the
stripe_unusable pattern (incremental between commits, re-totaled from
armed groups at the rescan), shows in the ENOSPC dump and in sysfs.
No admission change yet; that comes once the counter proves accurate
under the fill, churn, balance and reclaim suites. The by-size
fast-fail in the claim keeps its early exit: the point of measuring is
to make such inputs trustworthy, not to search harder around them.
Data block groups only for now; raid56 metadata and mixed block groups
join when the data counter has settled.
A trim takes a free range out of the tree, drops tree_lock for the
discard and adds the range back, and a rescan in that window would find
fewer claimable stripes than the incremental count (seen as "incremental
stripe_claimable ... above scanned" with discard=async). Each step
changes exactly the stripes the range belongs to, so the unlink debits
every one of them that was claimable (wholly free, not inside a live run:
the scan's rule) and the re-add credits every one that is claimable now,
in the tree_lock section that unlinks or links the range. Rounding
instead would over-debit a stripe per trimmed sub-stripe hole, and async
discard trims thousands after a delete. The wholly-free test falls back
to the bitmap lookup at offset_to_bitmap() as the other callers do, since
an extent entry starting inside a bitmap's window hides the bitmap from a
non-fuzzy search.
The nocow tail claim, btrfs_claim_stripe_tail(), now counts and removes
the tail in one ctl->tree_lock section, with claim_stripe_run_pieces()
doing the removal as for any claimed run, and fails with -ENOSPC unless
the whole tail is in the cache, debiting the
claimable count for what it removes in the same lock section. It counted under the lock and then
called btrfs_remove_free_space(), which returns 0 when the range is not
in the cache at all: two allocations for the same inode (two direct
writers, writeback racing fallocate) or a trim holding an entry out could
both "remove" the same tail and hand out the same extent. A self-test
claims the same tail twice.
[zygo: 7.3 port: 7.3 still tracks trimming ranges under
cache_writeout_mutex (v1 space cache); kept around the converted calls,
as on 6.18]