Zygo Blaxell [Sun, 29 Mar 2015 16:38:34 +0000 (12:38 -0400)]
zygo: cherry-picked up to e56a951 (extra args linus/master --remotes=btrfs-next/* --remotes=mason/* --after=now - 90 days -- fs/btrfs)
Commits included:
---- 83fe27e rcu: Make SRCU optional by using CONFIG_SRCU
---- b4caecd fs: introduce f_op->mmap_capabilities for nommu mmap support
---- de1414a fs: export inode_to_bdi and use it in favor of mapping->backing_dev_info
---- b83ae6d fs: remove mapping->backing_dev_info
---- df0ce26 fs: remove default_backing_dev_info
---- d83a08d mm: drop vm_ops->remap_pages and generic_file_remap_pages() stub
---- 8d38633 page_writeback: put account_page_redirty() after set_page_dirty()
---- b7a0365 Btrfs: ctree: reduce args where only fs_info used
---- a585e94 Btrfs: delayed-inode: replace root args iff only fs_info used
---- 01d5847 Btrfs: disk-io: replace root args iff only fs_info used
---- f749303 Btrfs: switch to kvfree() helper
---- e8c9f18 btrfs: constify structs with op functions or static definitions
---- b9fd47c btrfs: cleanup: remove no-used alloc_chunk in btrfs_check_data_free_space()
---- a688a04 btrfs: remove unused chunk_tree argument in several functions
---- 08da757 btrfs: cleanup: use for() loop in btrfs_map_bio()
---- 41d6b13 btrfs: fix sizeof format specifier in btrfs_check_super_valid()
---- 9eaed21 btrfs: remove unused fs_info arg from btrfs_close_extra_devices()
---- 0489234 btrfs: consistently use fs_info in close_ctree()
---- 638aa7e btrfs: factor btrfs_init_scrub() out of open_ctree()
---- 779a65a btrfs: factor btrfs_init_balance() out of open_ctree()
---- f37938e btrfs: factor btrfs_init_btree_inode() out of open_ctree()
---- ad61836 btrfs: factor btrfs_init_dev_replace_locks() out of open_ctree()
---- f9e92e4 btrfs: factor btrfs_init_qgroup() out of open_ctree()
---- 2a45819 btrfs: factor btrfs_init_workqueues() out of open_ctree()
---- 63443bf btrfs: factor btrfs_replay_log() out of open_ctree()
---- 4bbcaa6 btrfs: factor btrfs_read_roots() out of open_ctree()
---- 6f01105 btrfs: use correct type for workqueue flags
---- a4f3d2c btrfs: cleanup, reduce temporary variables in btrfs_read_roots
---- e36cb0b VFS: (Scripted) Convert S_ISLNK/DIR/REG(dentry->d_inode) to d_is_*(dentry)
---- 853d8ec btrfs: need_resched not needed with cond_resched
---- 351810c btrfs: use cond_resched_lock where possible
---- f8c269d btrfs: cleanup 64bit/32bit divs, compile time constants
---- 31e818f btrfs: cleanup, use kmalloc_array/kcalloc array helpers
---- f64c7b1 btrfs: remove shadowing variables in __btrfs_buffered_write
---- 3284da7 btrfs: use explicit initializer for seq_elem
---- b8b93ad btrfs: cleanup 64bit/32bit divs, provably bounded values
---- 47c5713 btrfs: replace remaining do_div calls with div_u64 variants
---- 9d644a6 btrfs: cleanup, use correct type in div_u64_rem
---- 093adbc btrfs: switch helper macros to static inlines in sysfs.h
---- 258ece0 btrfs: remove shadowing variables in __btrfs_map_block
---- 9266ff8 Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)
---- aa4661f Btrfs: prepare block group cache before writing
---- 2e0adf0 fix qgroups
---- 777414a Btrfs: fix merge delalloc logic
---- 3075d58 Btrfs: account for the correct number of extents for delalloc reservations
---- 2fdbf36 Btrfs: just free dummy extent buffers
---- 464919a Btrfs: add sanity test for outstanding_extents accounting
++++ 4f764e5 Btrfs: remove deleted xattrs on fsync log replay
++++ 5cc2b17 Btrfs: send, add missing check for dead clone root
++++ 2f1f465 Btrfs: send, don't leave without decrementing clone root's send_progress
++++ 8996a48 Btrfs: incremental send, clear name from cache after orphanization
++++ 5f806c3 Btrfs: incremental send, remove dead code
++++ 3d850dd Btrfs: add missing inode item update in fallocate()
++++ bf69196 Btrfs: change the insertion criteria for the qgroup operations rbtree
++++ 2f2ff0e Btrfs: fix metadata inconsistencies after directory fsync
++++ dcc82f4 Btrfs: fix log tree corruption when fs mounted with -o discard
++++ 3c3b04d btrfs: don't accept bare namespace as a valid xattr
++++ 26e726a btrfs: incorrect handling for fiemap_fill_next_extent return
++++ d862095 btrfs: explicitly set control file's private_data
++++ 727b978 btrfs: cleanup orphans while looking up default subvolume
++++ e56a951 Btrfs: Remove the check for old-style mkfs
Liu Bo [Tue, 17 Mar 2015 06:34:16 +0000 (14:34 +0800)]
Btrfs: Remove the check for old-style mkfs
This was used to make sure that a fresh btrfs from an older mkfs.btrfs,
but it also allows us to mount a buggy btrfs if this btrfs has the right
superblock head part but has something wrong with chunk tree part[1], and
after that we can hit BUG_ON()s set in the code to prevent something
impossible.
Since David has released "Btrfs progs v3.19-rc2", just remove the check,
if anyone who wants to make a fresh btrfs, please use the latest one.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com> Reviewed-by: Omar Sandoval <osandov@osandov.com> Reviewed-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit e56a951e01bf55f49533c47ad2ce61dbd613a3f3)
Jeff Mahoney [Fri, 20 Mar 2015 18:02:09 +0000 (14:02 -0400)]
btrfs: cleanup orphans while looking up default subvolume
Orphans in the fs tree are cleaned up via open_ctree and subvolume
orphans are cleaned via btrfs_lookup_dentry -- except when a default
subvolume is in use. The name for the default subvolume uses a manual
lookup that doesn't trigger orphan cleanup and needs to trigger it
manually as well. This doesn't apply to the remount case since the
subvolumes are cleaned up by walking the root radix tree.
Signed-off-by: Jeff Mahoney <jeffm@suse.com> Reviewed-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 727b9784b6085c99c2f836bf4fcc2848dc9cf904)
Tom Van Braeckel [Tue, 24 Mar 2015 15:35:49 +0000 (16:35 +0100)]
btrfs: explicitly set control file's private_data
The private_data member of the Btrfs control device file
(/dev/btrfs-control) is used to hold the current transaction and needs
to be initialized to NULL to signify that no transaction is in progress.
We explicitly set the control file's private_data to NULL to be
independent of whatever value the misc subsystem initializes it to.
Backstory:
----------
The misc subsystem (which is used by /dev/btrfs-control) initializes
a file's private_data to point to the misc device when a driver has
registered a custom open file operation and initializes it to NULL
when a custom open file operation has *not* been provided.
This subtle quirk is confusing, to the point where kernel code registers
*empty* file open operations to have private_data point to the misc
device structure.
And it leads to bugs, where the addition or removal of a custom open
file operation surprisingly changes the initial contents of a file's
private_data structure.
To simplify things in the misc subsystem, a patch [1] has been proposed
to *always* set private_data to point to the misc device instead of
only doing this when a custom open file operation has been registered.
But before we can fix this in the misc subsystem itself, we need to
modify the (few) drivers that rely on this very subtle behavior.
[1] https://lkml.org/lkml/2014/12/4/939
Signed-off-by: Martin Kepplinger <martink@posteo.de> Signed-off-by: Tom Van Braeckel <tomvanbraeckel@gmail.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit d8620958296e4fa61afde421f1de16a5c2234b28)
Chengyu Song [Tue, 24 Mar 2015 22:12:56 +0000 (18:12 -0400)]
btrfs: incorrect handling for fiemap_fill_next_extent return
fiemap_fill_next_extent returns 0 on success, -errno on error, 1 if this was
the last extent that will fit in user array. If 1 is returned, the return
value may eventually returned to user space, which should not happen, according
to manpage of ioctl.
Signed-off-by: Chengyu Song <csong84@gatech.edu> Reviewed-by: David Sterba <dsterba@suse.cz> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 26e726afe01c1c82072cf23a5ed89ce25f39d9f2)
ie. the missing attribute name after the namespace.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=94291 Reported-by: William Douglas <william.douglas@intel.com> CC: <stable@vger.kernel.org> # 2.6.29+ Signed-off-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 3c3b04d10ff1811a27f86684ccd2f5ba6983211d)
Filipe Manana [Mon, 23 Mar 2015 14:07:40 +0000 (14:07 +0000)]
Btrfs: fix log tree corruption when fs mounted with -o discard
While committing a transaction we free the log roots before we write the
new super block. Freeing the log roots implies marking the disk location
of every node/leaf (metadata extent) as pinned before the new super block
is written. This is to prevent the disk location of log metadata extents
from being reused before the new super block is written, otherwise we
would have a corrupted log tree if before the new super block is written
a crash/reboot happens and the location of any log tree metadata extent
ended up being reused and rewritten.
Even though we pinned the log tree's metadata extents, we were issuing a
discard against them if the fs was mounted with the -o discard option,
resulting in corruption of the log tree if a crash/reboot happened before
writing the new super block - the next time the fs was mounted, during
the log replay process we would find nodes/leafs of the log btree with
a content full of zeroes, causing the process to fail and require the
use of the tool btrfs-zero-log to wipeout the log tree (and all data
previously fsynced becoming lost forever).
Fix this by not doing a discard when pinning an extent. The discard will
be done later when it's safe (after the new super block is committed) at
extent-tree.c:btrfs_finish_extent_commit().
Fixes: e688b7252f78 (Btrfs: fix extent pinning bugs in the tree log) CC: <stable@vger.kernel.org> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit dcc82f4783ad91d4ab654f89f37ae9291cdc846a)
Filipe Manana [Fri, 20 Mar 2015 17:19:46 +0000 (17:19 +0000)]
Btrfs: fix metadata inconsistencies after directory fsync
We can get into inconsistency between inodes and directory entries
after fsyncing a directory. The issue is that while a directory gets
the new dentries persisted in the fsync log and replayed at mount time,
the link count of the inode that directory entries point to doesn't
get updated, staying with an incorrect link count (smaller then the
correct value). This later leads to stale file handle errors when
accessing (including attempt to delete) some of the links if all the
other ones are removed, which also implies impossibility to delete the
parent directories, since the dentries can not be removed.
Another issue is that (unlike ext3/4, xfs, f2fs, reiserfs, nilfs2),
when fsyncing a directory, new files aren't logged (their metadata and
dentries) nor any child directories. So this patch fixes this issue too,
since it has the same resolution as the incorrect inode link count issue
mentioned before.
This is very easy to reproduce, and the following excerpt from my test
case for xfstests shows how:
# Create our main test file and directory.
$XFS_IO_PROG -f -c "pwrite -S 0xaa 0 8K" $SCRATCH_MNT/foo | _filter_xfs_io
mkdir $SCRATCH_MNT/mydir
# Make sure all metadata and data are durably persisted.
sync
# Add a hard link to 'foo' inside our test directory and fsync only the
# directory. The btrfs fsync implementation had a bug that caused the new
# directory entry to be visible after the fsync log replay but, the inode
# of our file remained with a link count of 1.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/mydir/foo_2
# Add a few more links and new files.
# This is just to verify nothing breaks or gives incorrect results after the
# fsync log is replayed.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/mydir/foo_3
$XFS_IO_PROG -f -c "pwrite -S 0xff 0 64K" $SCRATCH_MNT/hello | _filter_xfs_io
ln $SCRATCH_MNT/hello $SCRATCH_MNT/mydir/hello_2
# Add some subdirectories and new files and links to them. This is to verify
# that after fsyncing our top level directory 'mydir', all the subdirectories
# and their files/links are registered in the fsync log and exist after the
# fsync log is replayed.
mkdir -p $SCRATCH_MNT/mydir/x/y/z
ln $SCRATCH_MNT/foo $SCRATCH_MNT/mydir/x/y/foo_y_link
ln $SCRATCH_MNT/foo $SCRATCH_MNT/mydir/x/y/z/foo_z_link
touch $SCRATCH_MNT/mydir/x/y/z/qwerty
# Now fsync only our top directory.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/mydir
# And fsync now our new file named 'hello', just to verify later that it has
# the expected content and that the previous fsync on the directory 'mydir' had
# no bad influence on this fsync.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/hello
# Simulate a crash/power loss.
_load_flakey_table $FLAKEY_DROP_WRITES
_unmount_flakey
# Verify the content of our file 'foo' remains the same as before, 8192 bytes,
# all with the value 0xaa.
echo "File 'foo' content after log replay:"
od -t x1 $SCRATCH_MNT/foo
# Remove the first name of our inode. Because of the directory fsync bug, the
# inode's link count was 1 instead of 5, so removing the 'foo' name ended up
# deleting the inode and the other names became stale directory entries (still
# visible to applications). Attempting to remove or access the remaining
# dentries pointing to that inode resulted in stale file handle errors and
# made it impossible to remove the parent directories since it was impossible
# for them to become empty.
echo "file 'foo' link count after log replay: $(stat -c %h $SCRATCH_MNT/foo)"
rm -f $SCRATCH_MNT/foo
# Now verify that all files, links and directories created before fsyncing our
# directory exist after the fsync log was replayed.
[ -f $SCRATCH_MNT/mydir/foo_2 ] || echo "Link mydir/foo_2 is missing"
[ -f $SCRATCH_MNT/mydir/foo_3 ] || echo "Link mydir/foo_3 is missing"
[ -f $SCRATCH_MNT/hello ] || echo "File hello is missing"
[ -f $SCRATCH_MNT/mydir/hello_2 ] || echo "Link mydir/hello_2 is missing"
[ -f $SCRATCH_MNT/mydir/x/y/foo_y_link ] || \
echo "Link mydir/x/y/foo_y_link is missing"
[ -f $SCRATCH_MNT/mydir/x/y/z/foo_z_link ] || \
echo "Link mydir/x/y/z/foo_z_link is missing"
[ -f $SCRATCH_MNT/mydir/x/y/z/qwerty ] || \
echo "File mydir/x/y/z/qwerty is missing"
# We expect our file here to have a size of 64Kb and all the bytes having the
# value 0xff.
echo "file 'hello' content after log replay:"
od -t x1 $SCRATCH_MNT/hello
# Now remove all files/links, under our test directory 'mydir', and verify we
# can remove all the directories.
rm -f $SCRATCH_MNT/mydir/x/y/z/*
rmdir $SCRATCH_MNT/mydir/x/y/z
rm -f $SCRATCH_MNT/mydir/x/y/*
rmdir $SCRATCH_MNT/mydir/x/y
rmdir $SCRATCH_MNT/mydir/x
rm -f $SCRATCH_MNT/mydir/*
rmdir $SCRATCH_MNT/mydir
# An fsck, run by the fstests framework everytime a test finishes, also detected
# the inconsistency and printed the following error message:
#
# root 5 inode 257 errors 2001, no inode item, link count wrong
# unresolved ref dir 258 index 2 namelen 5 name foo_2 filetype 1 errors 4, no inode ref
# unresolved ref dir 258 index 3 namelen 5 name foo_3 filetype 1 errors 4, no inode ref
status=0
exit
The expected golden output for the test is:
wrote 8192/8192 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
File 'foo' content after log replay: 0000000 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
* 0020000
file 'foo' link count after log replay: 5
file 'hello' content after log replay: 0000000 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
* 0200000
Which is the output after this patch and when running the test against
ext3/4, xfs, f2fs, reiserfs or nilfs2. Without this patch, the test's
output is:
wrote 8192/8192 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 65536/65536 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
File 'foo' content after log replay: 0000000 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
* 0020000
file 'foo' link count after log replay: 1
Link mydir/foo_2 is missing
Link mydir/foo_3 is missing
Link mydir/x/y/foo_y_link is missing
Link mydir/x/y/z/foo_z_link is missing
File mydir/x/y/z/qwerty is missing
file 'hello' content after log replay: 0000000 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
* 0200000
rmdir: failed to remove '/home/fdmanana/btrfs-tests/scratch_1/mydir/x/y/z': No such file or directory
rmdir: failed to remove '/home/fdmanana/btrfs-tests/scratch_1/mydir/x/y': No such file or directory
rmdir: failed to remove '/home/fdmanana/btrfs-tests/scratch_1/mydir/x': No such file or directory
rm: cannot remove '/home/fdmanana/btrfs-tests/scratch_1/mydir/foo_2': Stale file handle
rm: cannot remove '/home/fdmanana/btrfs-tests/scratch_1/mydir/foo_3': Stale file handle
rmdir: failed to remove '/home/fdmanana/btrfs-tests/scratch_1/mydir': Directory not empty
Fsck, without this fix, also complains about the wrong link count:
root 5 inode 257 errors 2001, no inode item, link count wrong
unresolved ref dir 258 index 2 namelen 5 name foo_2 filetype 1 errors 4, no inode ref
unresolved ref dir 258 index 3 namelen 5 name foo_3 filetype 1 errors 4, no inode ref
So fix this by logging the inodes that the dentries point to when
fsyncing a directory.
Filipe Manana [Sat, 14 Mar 2015 07:03:27 +0000 (07:03 +0000)]
Btrfs: change the insertion criteria for the qgroup operations rbtree
After looking at Liu Bo's recent patch (titled
"Btrfs: fix comp_oper to get right order") I realized the search made by
qgroup_oper_exists() was buggy because its rbtree navigation comparison
function, comp_oper_exist(), only looks at the fields bytenr and ref_root
of a tree node, ignoring the seq field completely. This was wrong because
when we insert a node into the rbtree we use comp_oper(), which takes a
decision based first on bytenr, then on seq and then on the ref_root field.
That means qgroup_oper_exists() could miss the fact that at least one
operation with given bytenr and ref_root exists.
Consider the following simple example of a 3 nodes qgroup operations
rbtree (created using comp_oper before this patch), where each node's key
is a tuple with the shape (bytenr, seq, ref_root, op):
[ (4096, 2, 20, op X) ]
/ \
/ \
[ (4096, 1, 5, op Y) ] [ (4096, 3, 10, op Z) ]
qgroup_oper_exists() when called to search for an existing operation for
bytenr 4096 and ref root 10 wouldn't find anything because it would go to
the left subtree instead of the right subtree, since comp_oper_exits()
ignores the seq field completely.
Fix this by changing the insertion navigation function to use the ref_root
field right after using the bytenr field and before using the seq field,
so that qgroup_oper_exists() / comp_oper_exist() work as expected.
This patch applies on top of the patch mentioned above from Liu.
Filipe Manana [Thu, 12 Mar 2015 23:23:13 +0000 (23:23 +0000)]
Btrfs: add missing inode item update in fallocate()
If we fallocate(), without the keep size flag, into an area already covered
by an extent previously fallocated, we were updating the inode's i_size but
we weren't updating the inode item in the fs/subvol tree. A following umount
+ mount would result in a loss of the inode's size (and an fsync would miss
too the fact that the inode changed).
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 3d850dd44889d3aa67d0b8007c2cdd259bff7da4)
Filipe Manana [Thu, 12 Mar 2015 16:04:50 +0000 (16:04 +0000)]
Btrfs: incremental send, remove dead code
The logic to detect path loops when attempting to apply a pending
directory rename, introduced in commit f959492fc15b (Btrfs: send, fix more issues related to directory renames)
is no longer needed, and the respective fstests test case for that commit,
btrfs/045, now passes without this code (as well as all the other test
cases for send/receive).
Filipe Manana [Thu, 12 Mar 2015 15:16:20 +0000 (15:16 +0000)]
Btrfs: incremental send, clear name from cache after orphanization
If a directory's reference ends up being orphanized, because the inode
currently being processed has a new path that matches that directory's
path, make sure we evict the name of the directory from the name cache.
This is because there might be descendent inodes (either directories or
regular files) that will be orphanized later too, and therefore the
orphan name of the ancestor must be used, otherwise we send issue rename
operations with a wrong path in the send stream.
$ mkfs.btrfs -f /dev/sdc
$ mount /dev/sdc /mnt2
$ btrfs receive /mnt2 -f /tmp/1.send
$ btrfs receive /mnt2 -f /tmp/2.send
ERROR: rename data/p1/p2 -> data/n4/p1/p2 failed. no such file or directory
Directories data/p1 (inode 263) and data/p1/p2 (inode 264) in the parent
snapshot are both orphanized during the incremental send, and as soon as
data/p1 is orphanized, we must make sure that when orphanizing data/p1/p2
we use a source path of o263-6-o/p2 for the rename operation instead of
the old path data/p1/p2 (the one before the orphanization of inode 263).
A test case for xfstests follows soon.
Reported-by: Robbie Ko <robbieko@synology.com> Signed-off-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 8996a48c0a8ed01c55f52e6794625c95a9977ee6)
Filipe Manana [Mon, 2 Mar 2015 20:53:53 +0000 (20:53 +0000)]
Btrfs: send, don't leave without decrementing clone root's send_progress
If the clone root was not readonly or the dead flag was set on it, we were
leaving without decrementing the root's send_progress counter (and before
we just incremented it). If a concurrent snapshot deletion was in progress
and ended up being aborted, it would be impossible to later attempt to
delete again the snapshot, since the root's send_in_progress counter could
never go back to 0.
We were also setting clone_sources_to_rollback to i + 1 too early - if we
bailed out because the clone root we got is not readonly or flagged as dead
we ended up later derreferencing a null pointer because we didn't assign
the clone root to sctx->clone_roots[i].root:
for (i = 0; sctx && i < clone_sources_to_rollback; i++)
btrfs_root_dec_send_in_progress(
sctx->clone_roots[i].root);
So just don't increment the send_in_progress counter if the root is readonly
or flagged as dead.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 2f1f465ae6da244099af55c066e5355abd8ff620)
Filipe Manana [Mon, 2 Mar 2015 20:53:52 +0000 (20:53 +0000)]
Btrfs: send, add missing check for dead clone root
After we locked the root's root item, a concurrent snapshot deletion
call might have set the dead flag on it. So check if the dead flag
is set and abort if it is, just like we do for the parent root.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 5cc2b17e80cf5770f2e585c2d90fd8af1b901258)
Filipe Manana [Mon, 23 Feb 2015 19:53:35 +0000 (19:53 +0000)]
Btrfs: remove deleted xattrs on fsync log replay
If we deleted xattrs from a file and fsynced the file, after a log replay
the xattrs would remain associated to the file. This was an unexpected
behaviour and differs from what other filesystems do, such as for example
xfs and ext3/4.
Fix this by, on fsync log replay, check if every xattr in the fs/subvol
tree (that belongs to a logged inode) has a matching xattr in the log,
and if it does not, delete it from the fs/subvol tree. This is a similar
approach to what we do for dentries when we replay a directory from the
fsync log.
This issue is trivial to reproduce, and the following excerpt from my
test for xfstests triggers the issue:
# Create out test file and add 3 xattrs to it.
touch $SCRATCH_MNT/foobar
$SETFATTR_PROG -n user.attr1 -v val1 $SCRATCH_MNT/foobar
$SETFATTR_PROG -n user.attr2 -v val2 $SCRATCH_MNT/foobar
$SETFATTR_PROG -n user.attr3 -v val3 $SCRATCH_MNT/foobar
# Make sure everything is durably persisted.
sync
# Now delete the second xattr and fsync the inode.
$SETFATTR_PROG -x user.attr2 $SCRATCH_MNT/foobar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foobar
_crash_and_mount
# After the fsync log is replayed, the file should have only 2 xattrs, the ones
# named user.attr1 and user.attr3. The btrfs fsync log replay bug left the file
# with the 3 xattrs that we had before deleting the second one and fsyncing the
# file.
echo "xattr names and values after first fsync log replay:"
$GETFATTR_PROG --absolute-names --dump $SCRATCH_MNT/foobar | _filter_scratch
# Now write some data to our file, fsync it, remove the first xattr, add a new
# hard link to our file and commit the fsync log by fsyncing some other new
# file. This is to verify that after log replay our first xattr does not exist
# anymore.
echo "hello world!" >> $SCRATCH_MNT/foobar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foobar
$SETFATTR_PROG -x user.attr1 $SCRATCH_MNT/foobar
ln $SCRATCH_MNT/foobar $SCRATCH_MNT/foobar_link
touch $SCRATCH_MNT/qwerty
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/qwerty
_crash_and_mount
# Now only the xattr with name user.attr3 should be set in our file.
echo "xattr names and values after second fsync log replay:"
$GETFATTR_PROG --absolute-names --dump $SCRATCH_MNT/foobar | _filter_scratch
status=0
exit
The expected golden output, which is produced with this patch applied or
when testing against xfs or ext3/4, is:
xattr names and values after first fsync log replay:
# file: SCRATCH_MNT/foobar
user.attr1="val1"
user.attr3="val3"
xattr names and values after second fsync log replay:
# file: SCRATCH_MNT/foobar
user.attr3="val3"
Without this patch applied, the output is:
xattr names and values after first fsync log replay:
# file: SCRATCH_MNT/foobar
user.attr1="val1"
user.attr2="val2"
user.attr3="val3"
xattr names and values after second fsync log replay:
# file: SCRATCH_MNT/foobar
user.attr1="val1"
user.attr2="val2"
user.attr3="val3"
A patch with a test case for xfstests follows soon.
Zygo Blaxell [Fri, 27 Mar 2015 03:34:36 +0000 (23:34 -0400)]
zygo: cherry-picked up to e1cbbfa (extra args linus/master --remotes=btrfs-next/* --remotes=mason/* --after=now - 90 days -- fs/btrfs)
Commits included:
++++ 83fe27e rcu: Make SRCU optional by using CONFIG_SRCU
---- 381cf65 btrfs: fix leak of path in btrfs_find_item
++++ 14692cc btrfs: cleanup, remove inode_item_info helper
++++ c234a24 btrfs: cleanup, remove inode_ref_info helper
++++ 9c4f61f btrfs: simplify insert_orphan_item
++++ 1d4c08e btrfs: expand btrfs_find_item if found_key is NULL
---- b4caecd fs: introduce f_op->mmap_capabilities for nommu mmap support
++++ de1414a fs: export inode_to_bdi and use it in favor of mapping->backing_dev_info
---- b83ae6d fs: remove mapping->backing_dev_info
---- df0ce26 fs: remove default_backing_dev_info
++++ e7070be Btrfs: change how we track dirty roots
++++ ce93ec5 Btrfs: track dirty block groups on their own list
++++ a8df6fe Btrfs: fix setup_leaf_for_split() to avoid leaf corruption
++++ 68b663d btrfs: update message levels for errors
++++ aa8ee31 btrfs: update message levels during failed mount
++++ f0954c6 btrfs: update message levels after checksum errors
---- 5efa049 btrfs: set proper message level for skinny metadata
++++ 9ee49a04 btrfs: switch extent_state state to unsigned
++++ 730a78c btrfs: remove a no-op unfreeze superbock callback
++++ 6219872 Btrfs: lookup for block group only if needed when freeing a tree block
++++ d36808e Btrfs: fix directory inconsistency after fsync log replay
++++ 2c2c452 Btrfs: fix fsync when extend references are added to an inode
++++ df8d116 Btrfs: fix fsync log replay for inodes with a mix of regular refs and extrefs
++++ e34c330 Btrfs: fix a out-of-bound access of raid_map
++++ cc7539e Btrfs: sort raid_map before adding tgtdev stripes
++++ 8e5cfb5 Btrfs: Make raid_map array be inlined in btrfs_bio structure
++++ 6e9606d Btrfs: add ref_count and free function for btrfs_bio
++++ b25c94c Btrfs: Fix a jump typo of nodatasum_case to avoid wrong WARN_ON()
++++ 114ab50 Btrfs: Remove noneed force_write in scrub_write_block_to_dev_replace
++++ 09dd7a0 Btrfs: Cleanup btrfs_bio_counter_inc_blocked()
++++ 7653947 Btrfs: btrfs_rm_dev_replace_blocked(): Use wait_event()
++++ dc5f7a3 Btrfs: Break loop when reach BTRFS_MAX_MIRRORS in scrub_setup_recheck_block()
++++ 8d6738c Btrfs: Separate finding-right-mirror and writing-to-target's process in scrub_handle_errored_block()
++++ b968fed Btrfs: Combine per-page recover in dev-replace and scrub
++++ be50a8d Btrfs: Simplify scrub_setup_recheck_block()'s argument
++++ 10f1190 Btrfs: Include map_type in raid_bio
++++ ffe2d20 Btrfs: Introduce BTRFS_BLOCK_GROUP_RAID56_MASK to check raid56 simply
++++ 5701934 Rename all ref_count to refs in struct
++++ 26455d3 Btrfs: cleanup unused run_most
++++ 0ee13fe btrfs: qgroup: move WARN_ON() to the correct location.
++++ 78f55e5 Btrfs: fix unused members in struct btrfs_root
++++ 95449a1 Btrfs: insert_new_root: Fix lock type of the extent buffer.
++++ a937b97 btrfs: kill btrfs_inode_*time helpers
++++ 9cc97d6 Btrfs: Add code to support file creation time
++++ 75d6ad3 btrfs: more superblock checks, lower bounds on devices and sectorsize/nodesize
++++ ce7fca5 btrfs: add checks for sys_chunk_array sizes
++++ 1ffb22c btrfs: cleanup, rename a few variables in btrfs_read_sys_array
++++ e3540ea btrfs: add more checks to btrfs_read_sys_array
++++ d4b450c Btrfs: fix race between transaction commit and empty block group removal
++++ 001a648 Btrfs: add missing cleanup on sysfs init failure
++++ de554a4 Btrfs: fix scrub race leading to use-after-free
++++ 289454a btrfs: clear bio reference after submit_one_bio()
++++ 2f08108 btrfs: delete chunk allocation attemp when setting block group ro
++++ b76808f btrfs: cleanup init for list in free-space-cache
---- d83a08d mm: drop vm_ops->remap_pages and generic_file_remap_pages() stub
++++ 8d38633 page_writeback: put account_page_redirty() after set_page_dirty()
++++ 575849e Btrfs: fix scheduler warning when syncing log
++++ f55985f Btrfs: scrub, fix sleep in atomic context
++++ 13212b5 btrfs: Fix out-of-space bug
++++ 3e05bde Btrfs: only adjust outstanding_extents when we do a short write
++++ 3266789 Btrfs: don't set and clear delalloc for O_DIRECT writes
++++ dcab6a3 Btrfs: account for large extents with enospc
++++ 3d84be7 Btrfs: fix BUG_ON in btrfs_orphan_add() when delete unused block group
---- 1a4bcf4 Btrfs: fix fsync data loss after adding hard link to inode
++++ a742994 Btrfs: don't remove extents and xattrs when logging new names
++++ b7a0365 Btrfs: ctree: reduce args where only fs_info used
++++ a585e94 Btrfs: delayed-inode: replace root args iff only fs_info used
++++ 01d5847 Btrfs: disk-io: replace root args iff only fs_info used
++++ f749303 Btrfs: switch to kvfree() helper
++++ e8c9f18 btrfs: constify structs with op functions or static definitions
++++ b9fd47c btrfs: cleanup: remove no-used alloc_chunk in btrfs_check_data_free_space()
++++ a688a04 btrfs: remove unused chunk_tree argument in several functions
++++ 08da757 btrfs: cleanup: use for() loop in btrfs_map_bio()
++++ 41d6b13 btrfs: fix sizeof format specifier in btrfs_check_super_valid()
++++ 9eaed21 btrfs: remove unused fs_info arg from btrfs_close_extra_devices()
++++ 0489234 btrfs: consistently use fs_info in close_ctree()
++++ 638aa7e btrfs: factor btrfs_init_scrub() out of open_ctree()
++++ 779a65a btrfs: factor btrfs_init_balance() out of open_ctree()
++++ f37938e btrfs: factor btrfs_init_btree_inode() out of open_ctree()
++++ ad61836 btrfs: factor btrfs_init_dev_replace_locks() out of open_ctree()
++++ f9e92e4 btrfs: factor btrfs_init_qgroup() out of open_ctree()
++++ 2a45819 btrfs: factor btrfs_init_workqueues() out of open_ctree()
++++ 63443bf btrfs: factor btrfs_replay_log() out of open_ctree()
++++ 4bbcaa6 btrfs: factor btrfs_read_roots() out of open_ctree()
++++ 6f01105 btrfs: use correct type for workqueue flags
++++ a4f3d2c btrfs: cleanup, reduce temporary variables in btrfs_read_roots
++++ e57cf21 Btrfs: fix allocation size calculations in alloc_btrfs_bio
---- e36cb0b VFS: (Scripted) Convert S_ISLNK/DIR/REG(dentry->d_inode) to d_is_*(dentry)
---- 4d884fc Btrfs: fix fsync race leading to ordered extent memory leaks
++++ 0c0ef4b Btrfs: abort the transaction if we fail to update the free space cache inode
++++ e8c1c76 Btrfs: add missing inode update when punching hole
++++ 5dfe2be Btrfs: fix off-by-one logic error in btrfs_realloc_node
++++ 5cdf83e Btrfs: do not ignore errors from btrfs_lookup_xattr in do_setxattr
---- 1932b7b btrfs: fix lost return value due to variable shadowing
++++ 84471e2 Btrfs: incremental send, don't rename a directory too soon
++++ 853d8ec btrfs: need_resched not needed with cond_resched
++++ 351810c btrfs: use cond_resched_lock where possible
++++ f8c269d btrfs: cleanup 64bit/32bit divs, compile time constants
++++ 31e818f btrfs: cleanup, use kmalloc_array/kcalloc array helpers
++++ f64c7b1 btrfs: remove shadowing variables in __btrfs_buffered_write
++++ 3284da7 btrfs: use explicit initializer for seq_elem
++++ b8b93ad btrfs: cleanup 64bit/32bit divs, provably bounded values
++++ 47c5713 btrfs: replace remaining do_div calls with div_u64 variants
++++ 9d644a6 btrfs: cleanup, use correct type in div_u64_rem
++++ 093adbc btrfs: switch helper macros to static inlines in sysfs.h
++++ 258ece0 btrfs: remove shadowing variables in __btrfs_map_block
++++ f5c0a12 Btrfs: remove extra run_delayed_refs in update_cowonly_root
---- 3a8b36f Btrfs: fix data loss in the fast fsync path
---- dd9ef13 Btrfs:__add_inode_ref: out of bounds memory read when looking for extended ref.
++++ 9266ff8 Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)
++++ aa4661f Btrfs: prepare block group cache before writing
---- 2e0adf0 fix qgroups
---- d220712 btrfs: fix sizeof format specifier in btrfs_check_super_valid()
++++ b4924a0 Btrfs: catch transaction abortion after waiting for it
++++ 48da5f0 Btrfs: fix comp_oper to get right order
++++ 8461a3d Btrfs: fix merge delalloc logic
++++ 6a41dd0 Btrfs: account for the correct number of extents for delalloc reservations
---- ea526d1 Btrfs: fix ASSERT(list_empty(&cur_trans->dirty_bgs_list)
---- 777414a Btrfs: fix merge delalloc logic
---- 3075d58 Btrfs: account for the correct number of extents for delalloc reservations
++++ 2fdbf36 Btrfs: just free dummy extent buffers
++++ 464919a Btrfs: add sanity test for outstanding_extents accounting
---- dcdf7f6 Btrfs: prepare block group cache before writing
++++ ba11721 Btrfs: account merges/splits properly
---- bcb7e44 Btrfs: just free dummy extent buffers
---- 6a3891c Btrfs: add sanity test for outstanding_extents accounting
++++ e1cbbfa Btrfs: fix outstanding_extents accounting in DIO
Josef Bacik [Tue, 17 Mar 2015 14:52:28 +0000 (10:52 -0400)]
Btrfs: fix outstanding_extents accounting in DIO
We are keeping track of how many extents we need to reserve properly based on
the amount we want to write, but we were still incrementing outstanding_extents
if we wrote less than what we requested. This isn't quite right since we will
be limited to our max extent size. So instead lets do something horrible! Keep
track of how many outstanding_extents we reserved, and decrement each time we
allocate an extent. If we use our entire reserve make sure to jack up
outstanding_extents on the inode so the accounting works out properly. Thanks,
Josef Bacik [Fri, 13 Mar 2015 19:01:24 +0000 (15:01 -0400)]
Btrfs: account merges/splits properly
My fix
Btrfs: fix merge delalloc logic
only fixed half of the problems, it didn't fix the case where we have two large
extents on either side and then join them together with a new small extent. We
need to instead keep track of how many extents we have accounted for with each
side of the new extent, and then see how many extents we need for the new large
extent. If they match then we know we need to keep our reservation, otherwise
we need to drop our reservation. This shows up with a case like this
Previously the logic would have said that the number extents required for the
new size (3) is larger than the number of extents required for the largest side
(2) therefore we need to keep our reservation. But this isn't the case, since
both sides require a reservation of 2 which leads to 4 for the whole range
currently reserved, but we only need 3, so we need to drop one of the
reservations. The same problem existed for splits, we'd think we only need 3
extents when creating the hole but in reality we need 4. Thanks,
Josef Bacik [Mon, 16 Mar 2015 21:38:52 +0000 (17:38 -0400)]
Btrfs: add sanity test for outstanding_extents accounting
I introduced a regression wrt outstanding_extents accounting. These are tricky
areas that aren't easily covered by xfstests as we could change MAX_EXTENT_SIZE
at any time. So add sanity tests to cover the various conditions that are
tricky in order to make sure we don't introduce regressions in the future.
Thanks,
Josef Bacik [Mon, 16 Mar 2015 21:38:02 +0000 (17:38 -0400)]
Btrfs: just free dummy extent buffers
If we fail during our sanity tests we could get NULL deref's because we unload
the module before the dummy extent buffers are free'd via RCU. So check for
this case and just free the things directly. Thanks,
Josef Bacik [Fri, 13 Mar 2015 19:12:23 +0000 (15:12 -0400)]
Btrfs: account for the correct number of extents for delalloc reservations
Direct IO can easily pass in an buffer that is greater than
BTRFS_MAX_EXTENT_SIZE, so take this into account when reserving extents in the
delalloc reservation code. Thanks,
Josef Bacik [Fri, 13 Mar 2015 19:12:08 +0000 (15:12 -0400)]
Btrfs: fix merge delalloc logic
My patch to properly count outstanding extents wrt MAX_EXTENT_SIZE introduced a
regression when re-dirtying already dirty areas. We have logic in split to make
sure we are taking the largest space into account but didn't have it for merge,
so it was sometimes making us think we were turning a tiny extent into a huge
extent, when in reality we already had a huge extent and needed to use the other
side in our logic. This fixes the regression that was reported by a user on
list. Thanks,
Reported-by: Markus Trippelsdorf <markus@trippelsdorf.de> Signed-off-by: Josef Bacik <jbacik@fb.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 8461a3de770477a9a7b8eeaebcc4804dbc26ca38)
Liu Bo [Fri, 13 Mar 2015 06:24:38 +0000 (14:24 +0800)]
Btrfs: fix comp_oper to get right order
Case (oper1->seq > oper2->seq) should differ with case (oper1->seq < oper2->seq).
Signed-off-by: Liu Bo <bo.li.liu@oracle.com> Reviewed-by: David Sterba <dsterba@suse.cz> Reviewed-by: Filipe Manana <fdmanana@suse.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 48da5f0a4cb2b6b44579f5737e8be888c0d02526)
Liu Bo [Fri, 6 Mar 2015 12:23:44 +0000 (20:23 +0800)]
Btrfs: catch transaction abortion after waiting for it
This problem is uncovered by a test case: http://patchwork.ozlabs.org/patch/244297.
Fsync() can report success when it actually doesn't. When we
have several threads running fsync() at the same tiem and in one fsync() we
get a transaction abortion due to some problems(in the test case it's disk
failures), and other fsync()s may return successfully which makes userspace
programs think that data is now safely flushed into disk.
It's because that after fsyncs() fail btrfs_sync_log() due to disk failures,
they get to try btrfs_commit_transaction() where it finds that there is
already a transaction being committed, and they'll just call wait_for_commit()
and return. Note that we actually check "trans->aborted" in btrfs_end_transaction,
but it's likely that the error message is still not yet throwed out and only after
wait_for_commit() we're sure whether the transaction is committed successfully.
This add the necessary check and it now passes the test.
Josef Bacik [Mon, 2 Mar 2015 21:37:31 +0000 (16:37 -0500)]
Btrfs: prepare block group cache before writing
Writing the block group cache will modify the extent tree quite a bit because it
truncates the old space cache and pre-allocates new stuff. To try and cut down
on the churn lets do the setup dance first, then later on hopefully we can avoid
looping with newly dirtied roots. Thanks,
Dave could hit this assert consistently running btrfs/078. This is because
when we update the block groups we could truncate the free space, which would
try to delete the csums for that range and dirty the csum root. For this to
happen we have to have already written out the csum root so it's kind of hard to
hit this case. This patch fixes this by changing the logic to only write the
dirty block groups if the dirty_cowonly_roots list is empty. This will get us
the same effect as before since we add the extent root last, and will cover the
case that we dirty some other root again but not the extent root. Thanks,
David Sterba [Tue, 24 Feb 2015 18:40:41 +0000 (19:40 +0100)]
btrfs: switch helper macros to static inlines in sysfs.h
The conversion macros use nested container_of that leads to a warning
fs/btrfs/sysfs.c: In function 'btrfs_feature_visible':
fs/btrfs/sysfs.c:183:8: warning: declaration of '__mptr' shadows a previous local
fs/btrfs/sysfs.c:183:8: warning: shadowed declaration is here
David Sterba [Wed, 25 Feb 2015 14:47:32 +0000 (15:47 +0100)]
btrfs: use explicit initializer for seq_elem
Using {} as initializer for struct seq_elem does not properly initialize
the list_head member, but it currently works because it gets set through
btrfs_get_tree_mod_seq if 'seq' is 0.
David Sterba [Fri, 16 Jan 2015 16:21:12 +0000 (17:21 +0100)]
btrfs: cleanup 64bit/32bit divs, compile time constants
Switch to div_u64 if the divisor is a numeric constant or sum of
sizeof()s. We can remove a few instances of do_div that has the hidden
semtantics of changing the 1st argument.
Small power-of-two divisors are converted to bitshifts, large values are
kept intact for clarity.
David Sterba [Thu, 8 Jan 2015 14:20:54 +0000 (15:20 +0100)]
btrfs: use cond_resched_lock where possible
Clean the opencoded variant, cond_resched_lock also checks the lock for
contention so it might help in some cases that were not covered by
simple need_resched().
Filipe Manana [Sat, 28 Feb 2015 22:29:22 +0000 (22:29 +0000)]
Btrfs: incremental send, don't rename a directory too soon
There's one more case where we can't issue a rename operation for a
directory as soon as we process it. We used to delay directory renames
only if they have some ancestor directory with a higher inode number
that got renamed too, but there's another case where we need to delay
the rename too - when a directory A is renamed to the old name of a
directory B but that directory B has its rename delayed because it
has now (in the send root) an ancestor with a higher inode number that
was renamed. If we don't delay the directory rename in this case, the
receiving end of the send stream will attempt to rename A to the old
name of B before B got renamed to its new name, which results in a
"directory not empty" error. So fix this by delaying directory renames
for this case too.
Filipe Manana [Mon, 23 Feb 2015 19:50:49 +0000 (19:50 +0000)]
Btrfs: do not ignore errors from btrfs_lookup_xattr in do_setxattr
The return value from btrfs_lookup_xattr() can be a pointer encoding an
error, therefore deal with it. This fixes commit 5f5bc6b1e2d5
("Btrfs: make xattr replace operations atomic").
Filipe Manana [Mon, 23 Feb 2015 19:48:52 +0000 (19:48 +0000)]
Btrfs: fix off-by-one logic error in btrfs_realloc_node
The end_slot variable actually matches the number of pointers in the
node and not the last slot (which is 'nritems - 1'). Therefore in order
to check that the current slot in the for loop doesn't match the last
one, the correct logic is to check if 'i' is less than 'end_slot - 1'
and not 'end_slot - 2'.
Fix this and set end_slot to be 'nritems - 1', as it's less confusing
since the variable name implies it's inclusive rather then exclusive.
Filipe Manana [Sun, 15 Feb 2015 22:38:54 +0000 (22:38 +0000)]
Btrfs: add missing inode update when punching hole
When punching a file hole if we endup only zeroing parts of a page,
because the start offset isn't a multiple of the sector size or the
start offset and length fall within the same page, we were not updating
the inode item. This prevented an fsync from doing anything, if no other
file changes happened in the current transaction, because the fields
in btrfs_inode used to check if the inode needs to be fsync'ed weren't
updated.
This issue is easy to reproduce and the following excerpt from the
xfstest case I made shows how to trigger it:
# Fsync the file, this makes btrfs update some btrfs inode specific fields
# that are used to track if the inode needs to be written/updated to the fsync
# log or not. After this fsync, the new values for those fields indicate that
# a subsequent fsync does not need to touch the fsync log.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
# Force a commit of the current transaction. After this point, any operation
# that modifies the data or metadata of our file, should update those fields in
# the btrfs inode with values that make the next fsync operation write to the
# fsync log.
sync
# Punch a hole in our file. This small range affects only 1 page.
# This made the btrfs hole punching implementation write only some zeroes in
# one page, but it did not update the btrfs inode fields used to determine if
# the next fsync needs to write to the fsync log.
$XFS_IO_PROG -c "fpunch 8000 4K" $SCRATCH_MNT/foo
# Another variation of the previously mentioned case.
$XFS_IO_PROG -c "fpunch 15000 100" $SCRATCH_MNT/foo
# Now fsync the file. This was a no-operation because the previous hole punch
# operation didn't update the inode's fields mentioned before, so they remained
# with the values they had after the first fsync - that is, they indicate that
# it is not needed to write to fsync log.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
echo "File content before:"
od -t x1 $SCRATCH_MNT/foo
# Simulate a crash/power loss.
_load_flakey_table $FLAKEY_DROP_WRITES
_unmount_flakey
# Enable writes and mount the fs. This makes the fsync log replay code run.
_load_flakey_table $FLAKEY_ALLOW_WRITES
_mount_flakey
# Because the last fsync didn't do anything, here the file content matched what
# it was after the first fsync, before the holes were punched, and not what it
# was after the holes were punched.
echo "File content after:"
od -t x1 $SCRATCH_MNT/foo
This issue has been around since 2012, when the punch hole implementation
was added, commit 2aaa66558172 ("Btrfs: add hole punching").
A test case for xfstests follows soon.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit e8c1c76e804b18120e6977fc092769c043876212)
Josef Bacik [Thu, 12 Feb 2015 14:43:51 +0000 (09:43 -0500)]
Btrfs: abort the transaction if we fail to update the free space cache inode
Our gluster boxes were hitting a problem where they'd run out of space when
updating the block group cache and therefore wouldn't be able to update the free
space inode. This is a problem because this is how we invalidate the cache and
protect ourselves from errors further down the stack, so if this fails we have
to abort the transaction so we make sure we don't end up with stale free space
cache. Thanks,
Chris Mason [Fri, 20 Feb 2015 01:51:39 +0000 (17:51 -0800)]
Btrfs: fix allocation size calculations in alloc_btrfs_bio
Since commit 8e5cfb55d3f (Btrfs: Make raid_map array be inlined in
btrfs_bio structure), the raid map array is allocated along with the
btrfs bio in alloc_btrfs_bio. The calculation used to decide how much
we need to allocate was using the wrong parameter passed into the
allocation function.
The passed in real_stripes will be zero if a target replace operation
is not currently running. We want to use total_stripes instead.
Signed-off-by: Chris Mason <clm@fb.com> Reported-by: David Sterba <dsterba@suse.cz> Tested-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit e57cf21e9787c081db4db6afa02e6e70112ee410)
Eric Sandeen [Fri, 1 Aug 2014 23:12:45 +0000 (18:12 -0500)]
btrfs: factor btrfs_read_roots() out of open_ctree()
Also, remove the two local variables create_uuid_tree
and check_uuid_tree; we can use the existence of
the uuid root and/or the RESCAN_UUID_TREE flag to
determine what action to take.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 4bbcaa648d23470f04285c52a27d87c8fe906203)
Eric Sandeen [Fri, 1 Aug 2014 23:12:46 +0000 (18:12 -0500)]
btrfs: factor btrfs_replay_log() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 63443bf54a746fada8ef2829148a29f28f07f7af)
Eric Sandeen [Mon, 16 Feb 2015 15:29:26 +0000 (16:29 +0100)]
btrfs: factor btrfs_init_workqueues() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed to btrfs_init_workqueues] Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 2a4581983f90d829d9aeaa4cb881031950f54937)
Eric Sandeen [Fri, 1 Aug 2014 23:12:42 +0000 (18:12 -0500)]
btrfs: factor btrfs_init_qgroup() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed to btrfs_init_qgroup] Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit f9e92e40b517bfe94d9e961ff484ab885348f322)
Eric Sandeen [Fri, 1 Aug 2014 23:12:41 +0000 (18:12 -0500)]
btrfs: factor btrfs_init_dev_replace_locks() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed to btrfs_init_dev_replace_locks] Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit ad6183680c0845ff8052a919b97c71b4fec8f33a)
Eric Sandeen [Fri, 1 Aug 2014 23:12:40 +0000 (18:12 -0500)]
btrfs: factor btrfs_init_btree_inode() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed to btrfs_init_btree_inode] Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit f37938e0e22f90740cfbfe74ecd9aad5e8369b9d)
Eric Sandeen [Fri, 1 Aug 2014 23:12:39 +0000 (18:12 -0500)]
btrfs: factor btrfs_init_balance() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed to btrfs_init_balance] Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 779a65a495b76aee7fae500ac7ae360dc8fd81ff)
Eric Sandeen [Fri, 1 Aug 2014 23:12:38 +0000 (18:12 -0500)]
btrfs: factor btrfs_init_scrub() out of open_ctree()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed to btrfs_init_scrub] Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 638aa7ed4667f63ec45b264d1b6e259a79638733)
Eric Sandeen [Fri, 1 Aug 2014 23:12:36 +0000 (18:12 -0500)]
btrfs: consistently use fs_info in close_ctree()
close_ctree() has a local fs_info var for convienience;
use it consistently.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 04892340777484351473f1756e6f870500c00a48)
Eric Sandeen [Fri, 1 Aug 2014 23:12:35 +0000 (18:12 -0500)]
btrfs: remove unused fs_info arg from btrfs_close_extra_devices()
The commit: 8dabb74 Btrfs: change core code of btrfs to support the
device replace operations
added the fs_info argument, but never used it -
just remove it again.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 9eaed21ef94b5da30eaf1b2503df1d41c2025175)
Fabian Frederick [Sat, 14 Feb 2015 12:10:22 +0000 (13:10 +0100)]
btrfs: fix sizeof format specifier in btrfs_check_super_valid()
This patch fixes mips compilation warning:
fs/btrfs/disk-io.c: In function 'btrfs_check_super_valid':
fs/btrfs/disk-io.c:3927:21: warning: format '%lu' expects argument
of type 'long unsigned int', but argument 3 has type 'unsigned int' [-Wformat]
Signed-off-by: Fabian Frederick <fabf@skynet.be> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 41d6b13ea7b459cd51d4446c9b264128a52db993)
Zhao Lei [Thu, 12 Feb 2015 07:42:16 +0000 (15:42 +0800)]
btrfs: cleanup: use for() loop in btrfs_map_bio()
for() is obviously better in these code block, and remove noused
init-value to reduce about 6 bytes binary size.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Reviewed-by: David Sterba <dsterba@suse.cz> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 08da757d3f29e9d33480ae1c44b557a2514d70c5)
Zhao Lei [Mon, 9 Feb 2015 12:31:44 +0000 (20:31 +0800)]
btrfs: remove unused chunk_tree argument in several functions
There functions include unused chunk_tree argument from the begining,
it is time to remove them and clean up relative code to prepare value
of this argument in caller.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit a688a04aab72a775e07fe460f563b46f7791cfc1)
Zhao Lei [Mon, 9 Feb 2015 06:40:20 +0000 (14:40 +0800)]
btrfs: cleanup: remove no-used alloc_chunk in btrfs_check_data_free_space()
int alloc_chunk is never used in this function, remove it.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit b9fd47cde5dd9725ca37769861b7424f40828492)
Wang Shilong [Sat, 22 Nov 2014 13:13:10 +0000 (21:13 +0800)]
Btrfs: switch to kvfree() helper
A new helper kvfree() in mm/utils.c will do this.
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit f749303bda20392e1d4a10adda84524365afd0e6)
Daniel Dressler [Fri, 21 Nov 2014 08:15:07 +0000 (17:15 +0900)]
Btrfs: disk-io: replace root args iff only fs_info used
This is the 3rd independent patch of a larger project to cleanup btrfs's
internal usage of btrfs_root. Many functions take btrfs_root only to
grab the fs_info struct.
By requiring a root these functions cause programmer overhead. That
these functions can accept any valid root is not obvious until
inspection.
This patch reduces the specificity of such functions to accept the
fs_info directly.
These patches can be applied independently and thus are not being
submitted as a patch series. There should be about 26 patches by the
project's completion. Each patch will cleanup between 1 and 34 functions
apiece. Each patch covers a single file's functions.
This patch affects the following function(s):
1) csum_tree_block
2) csum_dirty_buffer
3) check_tree_block_fsid
4) btrfs_find_tree_block
5) clean_tree_block
Signed-off-by: Daniel Dressler <danieru.dressler@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit 01d58472a887cba61da7b4e6dc251b5170b57e2e)
Daniel Dressler [Mon, 17 Nov 2014 13:05:02 +0000 (22:05 +0900)]
Btrfs: delayed-inode: replace root args iff only fs_info used
This is the second independent patch of a larger project to cleanup
btrfs's internal usage of btrfs_root. Many functions take btrfs_root
only to grab the fs_info struct.
By requiring a root these functions cause programmer overhead. That
these functions can accept any valid root is not obvious until
inspection.
This patch reduces the specificity of such functions to accept the
fs_info directly.
These patches can be applied independently and thus are not being
submitted as a patch series. There should be about 26 patches by the
project's completion. Each patch will cleanup between 1 and 34 functions
apiece. Each patch covers a single file's functions.
This patch affects the following function(s):
1) btrfs_wq_run_delayed_node
Signed-off-by: Daniel Dressler <danieru.dressler@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit a585e94895ed8ab2930bfd5186ffd20301b5c703)
Daniel Dressler [Wed, 12 Nov 2014 04:43:09 +0000 (13:43 +0900)]
Btrfs: ctree: reduce args where only fs_info used
This patch is part of a larger project to cleanup btrfs's internal usage
of struct btrfs_root. Many functions take btrfs_root only to grab a
pointer to fs_info.
This causes programmers to ponder which root can be passed. Since only
the fs_info is read affected functions can accept any root, except this
is only obvious upon inspection.
This patch reduces the specificty of such functions to accept the
fs_info directly.
This patch does not address the two functions in ctree.c (insert_ptr,
and split_item) which only use root for BUG_ONs in ctree.c
This patch affects the following functions:
1) fixup_low_keys
2) btrfs_set_item_key_safe
Signed-off-by: Daniel Dressler <danieru.dressler@gmail.com> Signed-off-by: David Sterba <dsterba@suse.cz>
(cherry picked from commit b7a0365ec7a0fb1d39113846fd34038af68ebd01)
Filipe Manana [Fri, 13 Feb 2015 16:56:14 +0000 (16:56 +0000)]
Btrfs: don't remove extents and xattrs when logging new names
If we are recording in the tree log that an inode has new names (new hard
links were added), we would drop items, belonging to the inode, that we
shouldn't:
1) When the flag BTRFS_INODE_COPY_EVERYTHING is set in the inode's runtime
flags, we ended up dropping all the extent and xattr items that were
previously logged. This was done only in memory, since logging a new
name doesn't imply syncing the log;
2) When the flag BTRFS_INODE_COPY_EVERYTHING is set in the inode's runtime
flags, we ended up dropping all the xattr items that were previously
logged. Like the case before, this was done only in memory because
logging a new name doesn't imply syncing the log.
This led to some surprises in scenarios such as the following:
1) write some extents to an inode;
2) fsync the inode;
3) truncate the inode or delete/modify some of its xattrs
4) add a new hard link for that inode
5) fsync some other file, to force the log tree to be durably persisted
6) power failure happens
The next time the fs is mounted, the fsync log replay code is executed,
and the resulting file doesn't have the content it had when the last fsync
against it was performed, instead if has a content matching what it had
when the last transaction commit happened.
So change the behaviour such that when a new name is logged, only the inode
item and reference items are processed.
This is easy to reproduce with the test I just made for xfstests, whose
main body is:
# Create our test file with some data.
$XFS_IO_PROG -f -c "pwrite -S 0xaa -b 8K 0 8K" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Make sure the file is durably persisted.
sync
# Append some data to our file, to increase its size.
$XFS_IO_PROG -f -c "pwrite -S 0xcc -b 4K 8K 4K" \
$SCRATCH_MNT/foo | _filter_xfs_io
# Fsync the file, so from this point on if a crash/power failure happens, our
# new data is guaranteed to be there next time the fs is mounted.
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo
# Now shrink our file to 5000 bytes.
$XFS_IO_PROG -c "truncate 5000" $SCRATCH_MNT/foo
# Now do an expanding truncate to a size larger than what we had when we last
# fsync'ed our file. This is just to verify that after power failure and
# replaying the fsync log, our file matches what it was when we last fsync'ed
# it - 12Kb size, first 8Kb of data had a value of 0xaa and the last 4Kb of
# data had a value of 0xcc.
$XFS_IO_PROG -c "truncate 32K" $SCRATCH_MNT/foo
# Add one hard link to our file. This made btrfs drop all of our file's
# metadata from the fsync log, including the metadata relative to the
# extent we just wrote and fsync'ed. This change was made only to the fsync
# log in memory, so adding the hard link alone doesn't change the persisted
# fsync log. This happened because the previous truncates set the runtime
# flag BTRFS_INODE_NEEDS_FULL_SYNC in the btrfs inode structure.
ln $SCRATCH_MNT/foo $SCRATCH_MNT/foo_link
# Now make sure the in memory fsync log is durably persisted.
# Creating and fsync'ing another file will do it.
# After this our persisted fsync log will no longer have metadata for our file
# foo that points to the extent we wrote and fsync'ed before.
touch $SCRATCH_MNT/bar
$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar
# As expected, before the crash/power failure, we should be able to see a file
# with a size of 32Kb, with its first 5000 bytes having the value 0xaa and all
# the remaining bytes with value 0x00.
echo "File content before:"
od -t x1 $SCRATCH_MNT/foo
# Simulate a crash/power loss.
_load_flakey_table $FLAKEY_DROP_WRITES
_unmount_flakey
# After mounting the fs again, the fsync log was replayed.
# The expected result is to see a file with a size of 12Kb, with its first 8Kb
# of data having the value 0xaa and its last 4Kb of data having a value of 0xcc.
# The btrfs bug used to leave the file as it used te be as of the last
# transaction commit - that is, with a size of 8Kb with all bytes having a
# value of 0xaa.
echo "File content after:"
od -t x1 $SCRATCH_MNT/foo
Forrest Liu [Wed, 11 Feb 2015 06:24:12 +0000 (14:24 +0800)]
Btrfs: fix BUG_ON in btrfs_orphan_add() when delete unused block group
Removing large amount of block group in a transaction may encounters
BUG_ON() in btrfs_orphan_add(). That is because btrfs_orphan_reserve_metadata()
will grab metadata reservation from transaction handle, and
btrfs_delete_unused_bgs() didn't reserve metadata for trnasaction handle when
delete unused block group.
Josef Bacik [Wed, 11 Feb 2015 20:08:59 +0000 (15:08 -0500)]
Btrfs: account for large extents with enospc
On our gluster boxes we stream large tar balls of backups onto our fses. With
160gb of ram this means we get really large contiguous ranges of dirty data, but
the way our ENOSPC stuff works is that as long as it's contiguous we only hold
metadata reservation for one extent. The problem is we limit our extents to
128mb, so we'll end up with at least 800 extents so our enospc accounting is
quite a bit lower than what we need. To keep track of this make sure we
increase outstanding_extents for every multiple of the max extent size so we can
be sure to have enough reserved metadata space. Thanks,
Josef Bacik [Wed, 11 Feb 2015 20:08:58 +0000 (15:08 -0500)]
Btrfs: don't set and clear delalloc for O_DIRECT writes
We do this to get the space accounting, but this is just needless churn on the
io_tree, so just drop setting/clearing delalloc and just drop the reserved data
space when we have a successfull allocation. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 3266789f9d08b27275bae5ab1dcd27d1bbf15e79)
Josef Bacik [Wed, 11 Feb 2015 20:08:57 +0000 (15:08 -0500)]
Btrfs: only adjust outstanding_extents when we do a short write
We have this weird dance where we always inc outstanding_extents when we do a
O_DIRECT write, even if we allocate the entire range. To get around this we
also drop the metadata space if we successfully write. This is an unnecessary
dance, we only need to jack up outstanding_extents if we don't satisfy the
entire range request in get_blocks_direct, otherwise we are good using our
original reservation. So drop the unconditional inc and the drop of the
metadata space that we have for the unconditional inc. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 3e05bde8c3c2dd761da4d52944a087907955a53c)
Zhao Lei [Thu, 12 Feb 2015 06:18:17 +0000 (14:18 +0800)]
btrfs: Fix out-of-space bug
Btrfs will report NO_SPACE when we create and remove files for several times,
and we can't write to filesystem until mount it again.
Steps to reproduce:
1: Create a single-dev btrfs fs with default option
2: Write a file into it to take up most fs space
3: Delete above file
4: Wait about 100s to let chunk removed
5: goto 2
Script is like following:
#!/bin/bash
# Recommend 1.2G space, too large disk will make test slow
DEV="/dev/sda16"
MNT="/mnt/tmp"
for ((i = 0; i < 10; i++)); do
df "$MNT" | tail -1
sleep 10
done
done
Reason:
It is triggered by commit: 47ab2a6c689913db23ccae38349714edf8365e0a
which is used to remove empty block groups automatically, but the
reason is not in that patch. Code before works well because btrfs
don't need to create and delete chunks so many times with high
complexity.
Above bug is caused by many reason, any of them can trigger it.
Reason1:
When we remove some continuous chunks but leave other chunks after,
these disk space should be used by chunk-recreating, but in current
code, only first create will successed.
Fixed by Forrest Liu <forrestl@synology.com> in:
Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole
Reason2:
contains_pending_extent() return wrong value in calculation.
Fixed by Forrest Liu <forrestl@synology.com> in:
Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole
Reason3:
btrfs_check_data_free_space() try to commit transaction and retry
allocating chunk when the first allocating failed, but space_info->full
is set in first allocating, and prevent second allocating in retry.
Fixed in this patch by clear space_info->full in commit transaction.
Tested for severial times by above script.
Changelog v3->v4:
use light weight int instead of atomic_t to record have_remove_bgs in
transaction, suggested by:
Josef Bacik <jbacik@fb.com>
Changelog v2->v3:
v2 fixed the bug by adding more commit-transaction, but we
only need to reclaim space when we are really have no space for
new chunk, noticed by:
Filipe David Manana <fdmanana@gmail.com>
Actually, our code already have this type of commit-and-retry,
we only need to make it working with removed-bgs.
v3 fixed the bug with above way.
Changelog v1->v2:
v1 will introduce a new bug when delete and create chunk in same disk
space in same transaction, noticed by:
Filipe David Manana <fdmanana@gmail.com>
V2 fix this bug by commit transaction after remove block grops.
Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com> Suggested-by: Filipe David Manana <fdmanana@gmail.com> Suggested-by: Josef Bacik <jbacik@fb.com> Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 13212b54d18d5235fb97fbdcba8ae453fd2a3a51)
Filipe Manana [Mon, 9 Feb 2015 21:14:24 +0000 (21:14 +0000)]
Btrfs: scrub, fix sleep in atomic context
My previous patch "Btrfs: fix scrub race leading to use-after-free"
introduced the possibility to sleep in an atomic context, which happens
when the scrub_lock mutex is held at the time scrub_pending_bio_dec()
is called - this function can be called under an atomic context.
Chris ran into this in a debug kernel which gave the following trace:
Fix this my acquiring the mutex after calling finish_wait(), which sets the
task's state to TASK_RUNNING.
Signed-off-by: Filipe Manana <fdmanana@suse.com> Reviewed-by: Liu Bo <bo.li.liu@oracle.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 575849ecf5d103ca9bbf0a6b9e89eba335d4e750)
page_writeback: put account_page_redirty() after set_page_dirty()
Helper account_page_redirty() fixes dirty pages counter for redirtied
pages. This patch puts it after dirtying and prevents temporary
underflows of dirtied pages counters on zone/bdi and current->nr_dirtied.
Signed-off-by: Konstantin Khebnikov <khlebnikov@yandex-team.ru> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit 8d38633c3b4093aca7524945f1e9249d7d3a44da)
Shaohua Li [Fri, 9 Jan 2015 18:40:15 +0000 (10:40 -0800)]
btrfs: delete chunk allocation attemp when setting block group ro
Below test will fail currently:
mkfs.ext4 -F /dev/sda
btrfs-convert /dev/sda
mount /dev/sda /mnt
btrfs device add -f /dev/sdb /mnt
btrfs balance start -v -dconvert=raid1 -mconvert=raid1 /mnt
The reason is there are some block groups with usage 0, but the whole
disk hasn't free space to allocate new chunk, so we even can't set such
block group readonly. This patch deletes the chunk allocation when
setting block group ro. For META, we already have reserve. But for
SYSTEM, we don't have, so the check_system_chunk is still required.
Naohiro Aota [Mon, 5 Jan 2015 16:01:03 +0000 (01:01 +0900)]
btrfs: clear bio reference after submit_one_bio()
After submit_one_bio(), `bio' can go away. However submit_extent_page()
leave `bio' referable if submit_one_bio() failed (e.g. -ENOMEM on OOM).
It will cause invalid paging request when submit_extent_page() is called
next time.
I reproduced ENOMEM case with the following script (need
CONFIG_FAIL_PAGE_ALLOC, and CONFIG_FAULT_INJECTION_DEBUG_FS).
Filipe Manana [Fri, 23 Jan 2015 18:27:00 +0000 (18:27 +0000)]
Btrfs: add missing cleanup on sysfs init failure
If we failed during initialization of sysfs, we weren't unregistering the
top level btrfs sysfs entry nor the debugfs stuff.
Not unregistering the top level sysfs entry makes future attempts to reload
the btrfs module impossible and the following is reported in dmesg:
Filipe Manana [Thu, 29 Jan 2015 19:18:25 +0000 (19:18 +0000)]
Btrfs: fix race between transaction commit and empty block group removal
Committing a transaction can race with automatic removal of empty block
groups (cleaner kthread), leading to a BUG_ON() in the transaction
commit code while running btrfs_finish_extent_commit(). The following
sequence diagram shows how it can happen:
CPU 1 CPU 2
btrfs_commit_transaction()
fs_info->running_transaction = NULL
btrfs_finish_extent_commit()
find_first_extent_bit()
-> found range for block group X
in fs_info->freed_extents[]
btrfs_delete_unused_bgs()
-> found block group X
Removed block group X's range
from fs_info->freed_extents[]
Fix this by ensuring that unpining the block group's range in
btrfs_finish_extent_commit() is done in a synchronized fashion
with removing the block group's range from freed_extents[]
in btrfs_delete_unused_bgs()
David Sterba [Fri, 31 Oct 2014 16:18:08 +0000 (17:18 +0100)]
btrfs: more superblock checks, lower bounds on devices and sectorsize/nodesize
I received a few crafted images from Jiri, all got through the recently
added superblock checks. The lower bounds checks for num_devices and
sector/node -sizes were missing and caused a crash during mount.
Tools for symbolic code execution were used to prepare the images
contents.
Reported-by: Jiri Slaby <jslaby@suse.cz> Signed-off-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 75d6ad382bb91f363452119d34238e156589ca2d)
Yang Dongsheng [Tue, 6 Jan 2015 12:54:42 +0000 (20:54 +0800)]
btrfs: qgroup: move WARN_ON() to the correct location.
In function qgroup_excl_accounting(), we need to WARN when
qg->excl is less than what we want to free, same to child
and parents. But currently, for parent qgroup, the WARN_ON()
is located after freeing qg->excl. It will WARN out even we
free it normally.
This patch move this WARN_ON() before freeing qg->excl.
Signed-off-by: Dongsheng Yang <yangds.fnst@cn.fujitsu.com> Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 0ee13fe28ce387864c0d2b1e8c52b64abe2fcd02)
Liu Bo [Wed, 17 Dec 2014 08:14:09 +0000 (16:14 +0800)]
Btrfs: cleanup unused run_most
"run_most" is not used anymore.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com> Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 26455d3318a1e2a38f783db07981e3ed67de40ed)
Zhao Lei [Tue, 20 Jan 2015 07:11:45 +0000 (15:11 +0800)]
Rename all ref_count to refs in struct
refs is better than ref_count to record a struct's ref count.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Suggested-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 570193454a5841a9fa477ce1bcc6c30ca6fcf552)
Zhao Lei [Tue, 20 Jan 2015 07:11:44 +0000 (15:11 +0800)]
Btrfs: Introduce BTRFS_BLOCK_GROUP_RAID56_MASK to check raid56 simply
So we can check raid56 with:
(map->type & BTRFS_BLOCK_GROUP_RAID56_MASK)
instead of long:
(map->type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6))
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit ffe2d2034bbb34f49f76c808550fdfbea2ea1659)
scrub_setup_recheck_block() have many arguments but most of them
can be get from one of them, we can remove them to make code clean.
Some other cleanup for that function also included in this patch.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit be50a8ddaae1d07135fd7e1c7017c1611075a560)
Zhao Lei [Tue, 20 Jan 2015 07:11:41 +0000 (15:11 +0800)]
Btrfs: Combine per-page recover in dev-replace and scrub
The code are similar, combine them to make code clean and easy to maintenance.
Some lost condition are also completed with benefit of this combination.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit b968fed1c3810a0a0d575cab8a72431fbc807615)
Zhao Lei [Tue, 20 Jan 2015 07:11:40 +0000 (15:11 +0800)]
Btrfs: Separate finding-right-mirror and writing-to-target's process in scrub_handle_errored_block()
In corrent code, code of finding-right-mirror and writing-to-target
are mixed in logic, if we find a right mirror but failed in writing
to target, it will treat as "hadn't found right block", and fill the
target with sblock_bad.
Actually, "failed in writing to target" does not mean "source
block is wrong", this patch separate above two condition in logic,
and do some cleanup to make code clean.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com> Signed-off-by: Miao Xie <miaox@cn.fujitsu.com> Signed-off-by: Chris Mason <clm@fb.com>
(cherry picked from commit 8d6738c1bd74a27ff6a5043c5211c7bff7745420)
projects / linux / log