kernel_optimize_test/arch/x86
Michel Lespinasse d065bd810b mm: retry page fault when blocking on disk transfer
This change reduces mmap_sem hold times that are caused by waiting for
disk transfers when accessing file mapped VMAs.

It introduces the VM_FAULT_ALLOW_RETRY flag, which indicates that the call
site wants mmap_sem to be released if blocking on a pending disk transfer.
In that case, filemap_fault() returns the VM_FAULT_RETRY status bit and
do_page_fault() will then re-acquire mmap_sem and retry the page fault.

It is expected that the retry will hit the same page which will now be
cached, and thus it will complete with a low mmap_sem hold time.

Tests:

- microbenchmark: thread A mmaps a large file and does random read accesses
  to the mmaped area - achieves about 55 iterations/s. Thread B does
  mmap/munmap in a loop at a separate location - achieves 55 iterations/s
  before, 15000 iterations/s after.

- We are seeing related effects in some applications in house, which show
  significant performance regressions when running without this change.

[akpm@linux-foundation.org: fix warning & crash]
Signed-off-by: Michel Lespinasse <walken@google.com>
Acked-by: Rik van Riel <riel@redhat.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Reviewed-by: Wu Fengguang <fengguang.wu@intel.com>
Cc: Ying Han <yinghan@google.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: "H. Peter Anvin" <hpa@zytor.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-10-26 16:52:09 -07:00
..
boot x86, setup: Use string copy operation to optimze copy in kernel compression 2010-10-07 21:23:09 -07:00
configs defconfig reduction 2010-08-14 22:26:53 +02:00
crypto Merge git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6 2010-05-03 11:28:58 +08:00
ia32 Don't dump task struct in a.out core-dumps 2010-10-14 10:57:40 -07:00
include/asm mm: remove pte_*map_nested() 2010-10-26 16:52:08 -07:00
kernel perf, x86: Fix up kmap_atomic() type 2010-10-26 16:52:08 -07:00
kvm Merge branch 'kvm-updates/2.6.37' of git://git.kernel.org/pub/scm/virt/kvm/kvm 2010-10-24 12:47:25 -07:00
lguest x86: lguest: Use new irq allocator 2010-10-12 16:53:45 +02:00
lib x86, mem: Optimize memmove for small size and unaligned cases 2010-09-24 18:57:11 -07:00
math-emu x86, fpu: Unbreak FPU emulation 2010-05-10 13:37:16 -07:00
mm mm: retry page fault when blocking on disk transfer 2010-10-26 16:52:09 -07:00
oprofile Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-10-21 14:11:46 -07:00
pci x86, olpc: Only enable PCI configuration type override on XO-1 2010-09-23 11:14:18 -07:00
power x86, tsc, sched: Recompute cyc2ns_offset's during resume from sleep states 2010-08-20 14:59:02 +02:00
tools
vdso Merge branches 'x86-cleanups-for-linus', 'x86-vmware-for-linus', 'x86-mtrr-for-linus', 'x86-apic-for-linus', 'x86-fpu-for-linus' and 'x86-vdso-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-08-06 16:22:59 -07:00
video
xen Merge branch 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:56 -07:00
.gitignore add random binaries to .gitignore 2010-04-08 11:34:34 +02:00
Kbuild
Kconfig Merge branch 'core-memblock-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-10-21 18:52:11 -07:00
Kconfig.cpu Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 08:58:16 -07:00
Kconfig.debug Merge branch 'x86-irq-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-10-22 08:54:21 -07:00
Makefile Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-10-21 13:06:00 -07:00
Makefile_32.cpu