Subject: [PATCH 0/3] btrfs: balance resume selects what the original request selected
To: linux-btrfs@vger.kernel.org
-A paused or interrupted balance resumes today with a usage=90 filter
-that the kernel adds on its own (update_balance_args()). It was meant
-to avoid re-balancing the block groups the run had already packed, but
-it misses in both directions:
-
- - It applies to every block group the request has not reached yet,
- except for conversions (those resume as soft converts instead): any
- of those more than 90% full is silently skipped, and the resumed
- balance finishes having done less than the original request asked
- for. A full balance after adding a device, run to restripe every
- block group across it, leaves behind exactly the full block groups
- that most need to move, without saying so.
-
- - It does not exclude the block groups the interrupted run created as
- relocation targets: any of those less than 90% full is selected
- again, and its data is relocated a second time.
-
-As a result, balance resume was only useful for a subset of balance use
-cases. All other use cases required the user to cancel the balance and
-manually run it again with updated filters to exclude work already done.
-
-Patch 2 replaces the heuristic with a bound fixed when the balance
-starts. Before the balance item is written, the vrange filter of each
-chunk type is limited to the addresses below the next chunk to be
-allocated (a 0..end range when the request had none, or a lower end on
-the one it had). Every block group that existed when the balance
-started stays selected until the balance relocates it. New chunks are
-normally allocated after the last chunk, but relocating the highest
-chunk into free space lower down deletes it without allocating
+When a paused or interrupted balance is resumed, the kernel alters the
+balance parameters in surprising ways: adding a `usage=90` filter to
+balances with neither convert nor usage filters, or adding a `soft`
+filter to a converting balance. This can cause the resumed balance
+to repeat work the interrupted balance already did, or skip work that
+the interrupted balance intended to do.
+
+Historically, we have advised users to avoid relying on balance
+resume, and instead mount with `-o skip_balance`, cancel any paused
+balance unconditionally after a mount, and start a new balance using
+an idempotent filter, or manage the entire balancing process with a
+userspace controller.
+
+Since balance works from highest block group bytenr to lowest, and new
+block groups are (usually) created with higher bytenrs than the highest
+existing block group, all balance filters can be made idempotent by
+adding a `vrange` filter.
+
+Patch 2 in this series removes the surprising heuristics and replaces
+them with a `vrange` filter, leaving the remaining filter parameters
+unmodified. Before the balance item is written, the vrange filter of
+each chunk type is limited to the addresses below the next chunk to be
+allocated (a 0..end range when the request had no vrange, or a lower end
+on the vrange supplied by userspace). Every block group that existed
+when the balance started stays selected until the balance relocates it.
+
+New chunks are normally allocated after the last chunk, but relocating the
+highest chunk into free space lower down deletes it without allocating
anything, and the next chunk would then reuse its address below the
-bound, where a resumed balance would select it again. So while a
-balance item exists, the chunk allocator never allocates below the
-bound: an in-memory floor that find_next_chunk() observes, the same way
-the chunk allocator already observes balance state during conversion.
-The bound never moves, so there is no progress to track or persist: no
-new on-disk state, no writes during the balance, and a resume after an
-unmount or a crash needs nothing more than the balance item that is
-already there. The bound is visible as a vrange in `btrfs balance
-status -v` and in the "balance: start" and "balance: resume" log lines.
-
-On resume, the walk revisits only the chunks it left in place above the
-point where it stopped (filtered out, ENOSPC, active swapfile) and checks
-them against the filters again; nothing is skipped.
+bound, where a resumed balance would select it again. To prevent this,
+while a balance item exists, find_next_chunk() never allocates below
+the vrange upper bound.
+
+The upper bound never moves, so there is no progress to track or persist.
+There's no new on-disk state and no balance item updates during the
+balance. A resume after an unmount or a crash needs nothing more than
+the balance item that is already there.
+
+On resume, the balance loop revisits only the chunks it left in place
+above the point where it stopped (filtered out, ENOSPC, active swapfile)
+and checks them against the filters again. Nothing is skipped, and
+nothing is repeated except the lightweight filter check.
The vrange filter has been present in btrfs since balance filters were
introduced in 2012, so there is no compatibility issue arising from
because nothing is done to a queued chunk until balance_remap_chunks(),
which undoes its own state on failure.
-Patch 3 is optional, offered for reviewers to reject; patches 1-2 do not
-depend on it, and my preference is to drop it. A balance item written
-by an older kernel has no vrange, so with patch 2, a balance that starts
-on an old kernel and resumes on a new kernel restarts from the
-beginning, redoing much of the work already done. That behavior may be
-an unpleasant surprise for some users, so patch 3 resumes exactly those
-items with the old usage=90 heuristic instead. The cost is keeping a
-layer of surprising behavior and code in the kernel to avoid a one-time
-issue that affects few users, is relatively easy to recover from, and
-will become irrelevant once the last users running old kernels upgrade.
+Patch 3 is optional. I offer it for completeness, but there's a tradeoff
+that might not be worth the cost. It can be dropped without breaking
+the rest of the series.
+
+A balance item written by an older kernel has no vrange, so with patch
+2, a balance that starts on an old kernel and resumes on a new kernel
+restarts from the beginning, redoing much of the work already done.
+That behavior may be an unpleasant surprise for some users, so when there
+is no `vrange` filter, patch 3 resumes exactly those items with the old
+usage=90 and convert-soft heuristics. The cost is keeping a layer of
+surprising behavior and code in the kernel to avoid a one-time issue
+that affects few users, is relatively easy to recover from, and will
+become irrelevant forever once the last users running old kernels upgrade.
Testing (KASAN kernels, misc-next and 6.18):