When the copy of a chunk stops part way through, scrub_enumerate_chunks()
still moves cursor_left to the end of the chunk and marks the
dev_replace item for writeback. If the replace is being suspended, the
next transaction commit writes that cursor to disk, and the replace
resumed at the next mount starts after the chunk. The part of the chunk
that the scrub had not reached yet is never copied to the target:
handle_ops_on_dev_replace() duplicates only new writes, and nothing else
copies existing extents. The replace finishes without reporting any
error, and when the source device goes away, the data that was only on
it is lost.
This happens on unmount: close_ctree() suspends a running replace with
btrfs_dev_replace_suspend_for_unmount() and cancels its scrub, and
btrfs_commit_super() then commits the running transaction, if there is
one, with the advanced cursor. Since commit
cab0d8623fb4 ("btrfs: avoid
create and commit empty transaction when committing super") an idle
unmount commits nothing there, and the cursor on disk stays at the
start of the chunk, but any transaction still running at that point
exposes the bug, e.g. one left by the cleaner dropping a deleted
subvolume. Before that commit, every unmount during a replace was
affected.
With 1 GiB data chunks, a filesystem with data profile single was
unmounted while a replace was copying a chunk and a subvolume was being
dropped. After the resumed replace finished and the source device was
removed, 530 MiB of file data from that chunk read back as -EIO. With
data profile raid1, scrub found the same blocks with csum errors on the
new device.
Move cursor_left to the end of the chunk only when scrub_chunk() returns
0. When it fails or is canceled, cursor_left stays at the start of the
chunk, where it was set before the copy started, and a resumed replace
copies the whole chunk again. A replace that fails is canceled, not
resumed, so for it the cursor only changes the reported progress.
Commit
1a1a8b732c7e ("Btrfs: fix race setting block group back to RW
mode during device replace") moved the cursor update before the error
checks so that it would happen before the block group goes back to RW,
because at the time writes were duplicated to the target only below
cursor_left. Commit
22ab04e814f4 ("Btrfs: fix race between device
replace and chunk allocation") made the duplication of writes
unconditional, and since then the cursor is only used to report
progress and to resume the replace. The update stays before
btrfs_dec_block_group_ro() anyway.
Fixes: 1a1a8b732c7e9 ("Btrfs: fix race setting block group back to RW mode during device replace")
Assisted-by: Claude:claude-opus-5-5
cache, found_key.offset))
ro_set = 0;
- down_write(&dev_replace->rwsem);
- dev_replace->cursor_left = dev_replace->cursor_right;
- dev_replace->item_needs_writeback = 1;
- up_write(&dev_replace->rwsem);
+ /*
+ * Move the cursor past this chunk only when all of it has been
+ * copied. If the copy was canceled or failed part way through,
+ * the cursor stays at the start of the chunk, so that a replace
+ * suspended by unmount copies the whole chunk again when it is
+ * resumed. Nothing else copies the extents the scrub did not
+ * reach: only new writes are duplicated to the target.
+ */
+ if (ret == 0) {
+ down_write(&dev_replace->rwsem);
+ dev_replace->cursor_left = dev_replace->cursor_right;
+ dev_replace->item_needs_writeback = 1;
+ up_write(&dev_replace->rwsem);
+ }
if (ro_set)
btrfs_dec_block_group_ro(cache);