]> git.hungrycats.org Git - linux/commit
fs/pipe: pre-allocate pages outside pipe->mutex in anon_pipe_write
authorBreno Leitao <leitao@debian.org>
Sun, 24 May 2026 14:44:58 +0000 (07:44 -0700)
committerChristian Brauner <brauner@kernel.org>
Thu, 28 May 2026 12:33:24 +0000 (14:33 +0200)
commit212ed884a1aeefda22d87c270d082e4b0a95821f
tree40b8457c9675e2a26f615a8cd638a0d2e1facc99
parentd24576fddf645a84e190a8991985458e40fbe424
fs/pipe: pre-allocate pages outside pipe->mutex in anon_pipe_write

anon_pipe_write() takes pipe->mutex (aka "mutex protecting the whole
thing") and then, from the per-iteration anon_pipe_get_page() helper,
used to call alloc_page(GFP_HIGHUSER | __GFP_ACCOUNT) once per page
while still holding it.

That allocation can sleep doing direct reclaim and/or runs memcg
charging, which extends the critical section and stalls a concurrent
reader on the very same mutex.

Just pre-alloc the required pages before the lock in an array and just pop
them inside the lock.

This can improve the pipe throughput up to 48% and reduce the
latency in 33%, easily seen when there is memory pressure and direct
reclaim.

Reviewed-by: Mateusz Guzik <mjguzik@gmail.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Link: https://patch.msgid.link/20260524-fix_pipe-v3-1-bb4a75d23a90@debian.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
fs/pipe.c