Since balance works from highest block group bytenr to lowest, and new
block groups are (usually) created with higher bytenrs than the highest
-existing block group, all balance filters can be made idempotent by
-adding a `vrange` filter or limiting the end of an existing vrange,
-without changing any other filter parameter. This eliminates the need
-for guesswork, and with the floor described below, this guarantees a
+existing block group, a `vrange` filter can be added, or the end of an
+existing vrange limited, without changing any other filter parameter.
+Adding vrange to any balance filter makes it idempotent, except for the
+limit filter which is handled in the second patch. This eliminates the
+need for guesswork, and with the floor described below, this guarantees a
correct balance resume result.
Patch 1 in this series removes the surprising heuristics and replaces
below the vrange upper bound.
The upper bound never moves, so there is no progress to track or persist.
-There's no new on-disk state and no balance item updates during the
-balance. A resume after an unmount or a crash needs nothing more than
+There's no new on-disk state. There are no balance item updates during
+the balance unless the limit filter is used. A resume after an unmount or a crash needs nothing more than
the balance item that is already there.
On resume, the balance loop revisits only the chunks it left in place
introduced in 2012, so there is no compatibility issue arising from
its use.
+Patch 2 handles the limit filter. A limit counts chunks instead of
+selecting them by address, so no vrange can make it idempotent. Balance
+counts the limit down in memory as it selects chunks, and a resume after
+a pause continues with what is left, but the balance item keeps the
+original values: a resume after an unmount or a crash starts counting
+again, and can relocate up to the full limit a second time. A
+`limit=min..max` range has a second problem: the resumed balance counts
+only the chunks that are left, which can be fewer than min, and then it
+relocates nothing. Patch 2 counts each chunk the balance processes,
+relocated or left in place by ENOSPC or an active swapfile (as the
+in-memory count always did), against the limit, or against both ends of
+a range (min stops at 0), and writes the remaining limits to the balance
+item after each such chunk. Only chunk types with a limit filter write
+the item, so a balance without limits makes no more writes than before.
+A resume after an unmount or a crash then continues from the remaining
+limits, like a resume after a pause, and min is the remaining minimum
+in both cases (in memory it was never decremented, so a range resumed
+after a pause had the same problem). The update joins the running
+transaction after the relocated chunk has been removed, so it commits
+with the removal or after it, never before. A crash between the two
+can make the resumed balance relocate one chunk more than requested,
+never fewer.
+
Patch 3 is optional. I offer it for completeness, but there's a tradeoff
that might not be worth the cost. It can be dropped without breaking
the rest of the series.
bound). On a kernel that still adds soft on resume, the dup chunk is
skipped and the test fails.
+ - balance-resume-limit-test.sh: data balances with `limit=N`,
+ `limit=M..M` and `limit=1..M-2` (M data chunks), paused after the
+ first chunk. The balance item read from the unmounted device, and the
+ recovered balance after a skip_balance mount, must show the remaining
+ limits, and after the resume the total number of chunks relocated must
+ be what the original request selected. The same for a resume on the
+ same mount, without an unmount. A balance without limits must not
+ update the balance item, and a limited one must update it once per
+ chunk. A limited balance is also recorded with dm-log-writes and
+ replayed to every flush point: in every replayed state with a balance
+ item, the chunks already relocated plus the remaining limit must equal
+ the request, or exceed it by one. Without patch 2, the resume after an
+ unmount relocates up to the full limit again, and the `limit=M..M`
+ resumes relocate nothing.
+
- remap-balance-resume-test.sh (remap-tree filesystems): a
METADATA_REMAP chunk placed above slow data chunks, a full -d -m
balance paused after the walk passed it, unmount, resume; the remap