btrfs: stripe_alloc: do not count trapped free space twice when making a block group...

inc_block_group_ro() asks the rest of the space_info to absorb a data
block group's free space before marking the group read-only.  With
stripe_alloc, the part of that free space that is trapped in partially
used stripes is already counted as used (bytes_stripe_unusable), and
the group's contribution is dropped again once it is read-only.  The
check therefore demanded room for those bytes twice, and a fragmented
raid56 data group could never be made read-only.

RAID56 groups must be read-only for scrub, so on a filesystem whose
data groups had been churned (2.98 GiB group, 51% used, 1.47 GiB of
the remainder trapped) every device's scrub failed with

  BTRFS warning: scrub: failed setting block group ro: -28

while the same filesystem without stripe_alloc scrubbed at 100% used.
Device replace takes the same path.  Freeing space made scrub work
again.

Subtract the group's own trapped bytes from the space it needs to
find elsewhere.  The counter may lag the space_info total between
commits; the check is a buffer heuristic, and the bound by num_bytes
keeps it from going negative.

Assisted-by: Claude:claude-fable-5

diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index c18116cfbdbe1536f7a14c1c4979572600e63d4b..7e14d460fa7a025957695197cecbb5b2788c170b 100644 (file)
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3739,12 +3739,26 @@ static int inc_block_group_ro(struct btrfs_block_group *cache, bool force)
                ret = 0;
        } else if (sinfo->flags & BTRFS_BLOCK_GROUP_DATA) {
                u64 sinfo_used = btrfs_space_info_used(sinfo, true);
+               u64 trapped = 0;
+
+               /*
+                * The free space of a stripe_alloc group that is trapped in
+                * partially used stripes is already counted as used in the
+                * space_info (bytes_stripe_unusable), and that contribution is
+                * dropped once the group is read-only (see the disarm below).
+                * Asking the rest of the filesystem to absorb it a second time
+                * means a fragmented raid56 data group can never be made
+                * read-only: scrub and device replace, which require it for
+                * raid56, then fail with ENOSPC on a half-full filesystem.
+                */
+               if (READ_ONCE(cache->stripe_unusable_ready))
+                       trapped = min(READ_ONCE(cache->stripe_unusable), num_bytes);
 
                /*
                 * Here we make sure if we mark this bg RO, we still have enough
                 * free space as buffer.
                 */
-               if (sinfo_used + num_bytes <= sinfo->total_bytes)
+               if (sinfo_used - trapped + num_bytes <= sinfo->total_bytes)
                        ret = 0;
        } else {
                /*