aio: add missing smp_rmb() in read_events_ring

commit 2ff396be602f10b5eab8e73b24f20348fa2de159 upstream.

We ran into a case on ppc64 running mariadb where io_getevents would
return zeroed out I/O events.  After adding instrumentation, it became
clear that there was some missing synchronization between reading the
tail pointer and the events themselves.  This small patch fixes the
problem in testing.

Thanks to Zach for helping to look into this, and suggesting the fix.

Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Benjamin LaHaise <bcrl@kvack.org>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>

diff --git a/fs/aio.c b/fs/aio.c
index 6d68e01dc7caaf300139e7e0fc0b9a21a44f63a5..b732a9c3204278062ae7898b00eb27b6b20ad390 100644 (file)
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -1065,6 +1065,12 @@ static long aio_read_events_ring(struct kioctx *ctx,
        tail = ring->tail;
        kunmap_atomic(ring);
 
+       /*
+        * Ensure that once we've read the current tail pointer, that
+        * we also see the events that were stored up to the tail.
+        */
+       smp_rmb();
+
        pr_debug("h%u t%u m%u\n", head, tail, ctx->nr_events);
 
        if (head == tail)