From: Zygo Blaxell Date: Sat, 30 Sep 2023 06:47:08 +0000 (-0400) Subject: btrfs: allow changing system dev_extent size via sysfs X-Git-Url: http://git.hungrycats.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=refs%2Fheads%2Ftopics%2Fdev-extent-defrag;p=linux btrfs: allow changing system dev_extent size via sysfs When a filesystem using striped profiles such as raid0, raid10, raid5, or raid6 is repeatedly resized and balanced, severe and disruptive dev_extent fragmentation can occur when dev_extents are allowed to have different sizes. On filesystems over 50 GiB in size, data and metadata dev_extents are both 1 GiB each, but the system dev_extents are always 32 MiB regardless of filesystem size. This can leave some devices with odd-sized dev_extent holes on filesystems with more devices than the metadata's stripe count. Striped profile allocators will try to allocate the widest block groups possible, so as the devices fill up, eventually block groups are allocated that have dev_extents shorter than 1 GiB, and these will be allocated on all devices because that's what striping profiles do. As storage is expanded, these block groups are balanced and removed, leaving behind small free space holes. These holes limit the size of future block group allocations, causing small dev_extents to multiply. This repeats until the filesystem has hundreds of thousands of dev_extents at the minimum 1 MiB size, instead of hundreds of extents at the normal 1 GiB size. Block groups made from these small dev_extents cannot hold the largest btrfs extents (128 MiB), making their space effectively unusable. The btrfs allocator has several functions that are O(n) in the number of block groups, which start to slow down significantly as the dev_extents fragment. This fragmentation is avoidable by: 1. round down the device sizes to an exact multiple of 1 GiB + 1 MiB. (The 1 MiB is reserved by btrfs at the beginning of each device, so this means 1 GiB dev_extents will always fit exactly on the device with no fragmentation.) 2. set all block group dev_extent sizes to 1 GiB, so that all dev_extents and all free space holes are exactly 1 GiB in length. 3. balance existing block groups so that they are all aligned to 1 GiB boundaries (offset by the reserved 1 MiB at the start). Of these, only #2 requires a kernel change. Signed-off-by: Zygo Blaxell --- diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index 0d14570c8bc29..4b9c80c0868f0 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs/sysfs.c @@ -760,8 +760,8 @@ static ssize_t btrfs_chunk_size_show(struct kobject *kobj, * Store new chunk size in space info. Can be called on a read-only filesystem. * * If the new chunk size value is larger than 10% of free space it is reduced - * to match that limit. Alignment must be to 256M and the system chunk size - * cannot be set. + * to match that limit. Alignment must be to 256M for data and metadata chunks, + * 32M for system chunks. */ static ssize_t btrfs_chunk_size_store(struct kobject *kobj, struct kobj_attribute *a, @@ -771,6 +771,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj, struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj)); char *retptr; u64 val; + u64 min_size; if (!capable(CAP_SYS_ADMIN)) return -EPERM; @@ -781,9 +782,12 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj, if (btrfs_is_zoned(fs_info)) return -EINVAL; - /* System block type must not be changed. */ - if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM) - return -EPERM; + /* System block type must not be smaller than minimum. */ + if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM) { + min_size = SZ_32M; + } else { + min_size = SZ_256M; + } val = memparse(buf, &retptr); /* There could be trailing '\n', also catch any typos after the value */ @@ -796,11 +800,11 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj, /* Limit stripe size to 10% of available space. */ val = min(mult_perc(fs_info->fs_devices->total_rw_bytes, 10), val); - /* Must be multiple of 256M. */ - val &= ~((u64)SZ_256M - 1); + /* Must be multiple of min size. */ + val &= ~((u64)min_size - 1); - /* Must be at least 256M. */ - if (val < SZ_256M) + /* Must be at least 256M for data, 32M for system. */ + if (val < min_size) return -EINVAL; btrfs_update_space_info_chunk_size(space_info, val);