lib: add legacy-free bees_readahead_raw

The legacy extent scanning code suffered from a performance issue:
read calls were issued 4K at a time, sometimes in reverse order.
The kernel's dedupe ioctl implementation has a loop which reads 4K
from each file, comparing them one page at a time.  These IO sizes
are terrible for performance, especially on spinning drives.

To work around the small IO size issue, explicit readahead calls were
introduced so that these 4K reads can be satisfied from VFS cache,
while the VFS cache was populated by fadvise() or pread() into larger
buffer sizes.

Two optimizations were then added on top:  For spinning drives, extra
performance can be achieved by serializing read requests, so a mutex
was added to serialize all reads.  For non-spinning drives, the extra
kernel-to-userspace data copy slowed down read performance, so a
set tracks which data has been read recently, and skips readahead if
a naive model of VFS cache contents expects the data to already be
in the VFS cache.

The scan_next code generates larger read requests, so it doesn't require
explicit readahead for scan performance; however, the kernel dedupe
ioctl still requires explicit readahead, and requires it unconditionally.
Add a separate entry point to the readahead emulation so that scan_next
can issue the readahead unconditionally from BeesContext::dedup().

Signed-off-by: Zygo Blaxell <bees@furryterror.org>

diff --git a/src/bees-lib.cc b/src/bees-lib.cc
index 866ebc0ee646008d465919e7341c67e7984b38c2..3f26c4c8315a2ff78420a02195b1c2b2438d7ace 100644 (file)
--- a/src/bees-lib.cc
+++ b/src/bees-lib.cc
@@ -109,11 +109,9 @@ bees_readahead_check(int const fd, off_t const offset, size_t const size)
        return rv.second;
 }
 
-static
 void
-bees_readahead_nolock(int const fd, const off_t offset, const size_t size)
+bees_readahead_raw(int const fd, const off_t offset, const size_t size)
 {
-       if (!bees_readahead_check(fd, offset, size)) return;
        Timer readahead_timer;
        BEESNOTE("readahead " << name_fd(fd) << " offset " << to_hex(offset) << " len " << pretty(size));
        BEESTOOLONG("readahead " << name_fd(fd) << " offset " << to_hex(offset) << " len " << pretty(size));
@@ -136,6 +134,14 @@ bees_readahead_nolock(int const fd, const off_t offset, const size_t size)
        BEESCOUNTADD(readahead_ms, readahead_timer.age() * 1000);
 }
 
+static
+void
+bees_readahead_nolock(int const fd, const off_t offset, const size_t size)
+{
+       if (!bees_readahead_check(fd, offset, size)) return;
+       bees_readahead_raw(fd, offset, size);
+}
+
 static mutex s_only_one;
 
 void

diff --git a/src/bees-lib.h b/src/bees-lib.h
index e3b9986fdc89b226e29c19268803997c141cb2a1..073c99c84d07badd0119b8cc4dd1738bce3f282c 100644 (file)
--- a/src/bees-lib.h
+++ b/src/bees-lib.h
@@ -38,12 +38,22 @@ int bees_openat2(int parent_fd, const char *pathname, uint64_t flags);
 /// FADV_WILLNEED submits reads at iopriority 3, which are preempted by bees's own
 /// reader threads and may never execute; using pread() forces the data into the page
 /// cache immediately under normal thread scheduling.
+/// This function implements a number of workarounds intended to improve
+/// the behavior of legacy v0.11 scan_one_extent code.  It should not be
+/// used for new code.
 void bees_readahead(int fd, off_t offset, size_t size);
 
 /// Emulate POSIX_FADV_WILLNEED for two ranges simultaneously via pread().
-/// @see bees_readahead
+/// This function is used for the legacy v0.11 match extension code.
+/// It should not be used for new code.
 void bees_readahead_pair(int fd, off_t offset, size_t size, int fd2, off_t offset2, size_t size2);
 
+/// Emulate POSIX_FADV_WILLNEED with no legacy baggage.
+/// This function is called to work around specific kernel dedupe
+/// performance issues.  All other bees_readahead_* functions eventually
+/// call this one to perform the emulation.
+void bees_readahead_raw(int fd, off_t offset, size_t size);
+
 /// Issue POSIX_FADV_DONTNEED to release cached pages for the given range.
 void bees_unreadahead(int fd, off_t offset, size_t size);