btrfs: stripe_alloc: do not count trapped free space twice when making a block group...

inc_block_group_ro() refuses to make a data block group read-only
unless the other groups of the space_info have at least as much free
space as this group is withdrawing: a read-only group's free space
leaves the pool writers allocate from, data never overcommits, and the
check (sinfo_used + num_bytes <= total_bytes) keeps concurrent writers
from failing allocations they could otherwise have been placed in while
the group is read-only.  Scrub takes the same path as balance, since
the space is demanded for those writers, not for moving data.

With stripe_alloc, the part of this group's free space that is trapped
in partially used stripes is already counted in btrfs_space_info_used()
through bytes_stripe_unusable, and it is also part of num_bytes, so the
check demands room elsewhere for those bytes twice -- although once the
group is read-only its trapped bytes leave the aggregate anyway, and
writers never had them to begin with.  A fragmented raid56 data group
could therefore never be made read-only.

RAID56 groups must be read-only for scrub: scrub keeps its own copy of
the stripe, which a concurrent read-modify-write would corrupt, so
unlike other profiles it cannot proceed without the flag.  On a
filesystem whose data groups had been churned (2.98 GiB group, 51%
used, 1.47 GiB of the remainder trapped) every device's scrub failed
with

  BTRFS warning: scrub: failed setting block group ro: -28

while the same filesystem without stripe_alloc scrubbed at 100% used.
Device replace takes the same path.  Freeing space made scrub work
again.

Subtract the group's own trapped bytes from the space it needs to find
elsewhere.  The counter may lag the space_info total between commits;
the check is a buffer heuristic, and the bound by num_bytes keeps it
from going negative.

Assisted-by: Claude:claude-fable-5
Assisted-by: Claude:claude-fable-5-1

diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 9bc162eec70b284f87764fe6fc1cd0947077a2c1..56cc847ca62affd46a4f4c36d5945bb25cd2d539 100644 (file)
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3775,12 +3775,26 @@ static int inc_block_group_ro(struct btrfs_block_group *cache, bool force)
                ret = 0;
        } else if (sinfo->flags & BTRFS_BLOCK_GROUP_DATA) {
                u64 sinfo_used = btrfs_space_info_used(sinfo, true);
+               u64 trapped = 0;
+
+               /*
+                * The free space of a stripe_alloc group that is trapped in
+                * partially used stripes is already counted as used in the
+                * space_info (bytes_stripe_unusable), and that contribution is
+                * dropped once the group is read-only (see the disarm below).
+                * Asking the rest of the filesystem to absorb it a second time
+                * means a fragmented raid56 data group can never be made
+                * read-only: scrub and device replace, which require it for
+                * raid56, then fail with ENOSPC on a half-full filesystem.
+                */
+               if (READ_ONCE(cache->stripe_unusable_ready))
+                       trapped = min(READ_ONCE(cache->stripe_unusable), num_bytes);
 
                /*
                 * Here we make sure if we mark this bg RO, we still have enough
                 * free space as buffer.
                 */
-               if (sinfo_used + num_bytes <= sinfo->total_bytes)
+               if (sinfo_used - trapped + num_bytes <= sinfo->total_bytes)
                        ret = 0;
        } else {
                /*