kernel_optimize_test/arch/x86
Sean Christopherson 6a3ea3e68b x86/entry/64: Do not use RDPID in paranoid entry to accomodate KVM
KVM has an optmization to avoid expensive MRS read/writes on
VMENTER/EXIT. It caches the MSR values and restores them either when
leaving the run loop, on preemption or when going out to user space.

The affected MSRs are not required for kernel context operations. This
changed with the recently introduced mechanism to handle FSGSBASE in the
paranoid entry code which has to retrieve the kernel GSBASE value by
accessing per CPU memory. The mechanism needs to retrieve the CPU number
and uses either LSL or RDPID if the processor supports it.

Unfortunately RDPID uses MSR_TSC_AUX which is in the list of cached and
lazily restored MSRs, which means between the point where the guest value
is written and the point of restore, MSR_TSC_AUX contains a random number.

If an NMI or any other exception which uses the paranoid entry path happens
in such a context, then RDPID returns the random guest MSR_TSC_AUX value.

As a consequence this reads from the wrong memory location to retrieve the
kernel GSBASE value. Kernel GS is used to for all regular this_cpu_*()
operations. If the GSBASE in the exception handler points to the per CPU
memory of a different CPU then this has the obvious consequences of data
corruption and crashes.

As the paranoid entry path is the only place which accesses MSR_TSX_AUX
(via RDPID) and the fallback via LSL is not significantly slower, remove
the RDPID alternative from the entry path and always use LSL.

The alternative would be to write MSR_TSC_AUX on every VMENTER and VMEXIT
which would be inflicting massive overhead on that code path.

[ tglx: Rewrote changelog ]

Fixes: eaad981291 ("x86/entry/64: Introduce the FIND_PERCPU_BASE macro")
Reported-by: Tom Lendacky <thomas.lendacky@amd.com>
Debugged-by: Tom Lendacky <thomas.lendacky@amd.com>
Suggested-by: Andy Lutomirski <luto@kernel.org>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20200821105229.18938-1-pbonzini@redhat.com
2020-08-21 16:15:27 +02:00
..
boot Kbuild updates for v5.9 2020-08-09 14:10:26 -07:00
configs x86/defconfigs: Refresh defconfig files 2020-07-25 12:02:14 +02:00
crypto crypto: x86/curve25519 - Remove unused carry variables 2020-07-31 18:25:29 +10:00
entry x86/entry/64: Do not use RDPID in paranoid entry to accomodate KVM 2020-08-21 16:15:27 +02:00
events perf/x86/rapl: Add support for Intel SPR platform 2020-08-14 12:35:12 +02:00
hyperv vmalloc: fix the owner argument for the new __vmalloc_node_range callers 2020-07-03 16:15:25 -07:00
ia32 mm: remove unneeded includes of <asm/pgalloc.h> 2020-08-07 11:33:26 -07:00
include A set oftimekeeping/VDSO updates: 2020-08-14 14:26:08 -07:00
kernel Misc fixes and small updates all around the place: 2020-08-15 10:38:03 -07:00
kvm PPC: 2020-08-12 12:25:06 -07:00
lib kbuild: remove cc-option test of -fno-stack-protector 2020-07-07 11:13:10 +09:00
math-emu Merge branch 'work.regset' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-08-07 09:29:25 -07:00
mm mm/x86: use general page fault accounting 2020-08-12 10:58:04 -07:00
net bpf, i386: Remove unneeded conversion to bool 2020-05-07 16:29:14 +02:00
oprofile
pci pci-v5.9-changes 2020-08-07 18:48:15 -07:00
platform mm: remove unneeded includes of <asm/pgalloc.h> 2020-08-07 11:33:26 -07:00
power Kbuild updates for v5.9 2020-08-09 14:10:26 -07:00
purgatory Misc fixes and small updates all around the place: 2020-08-15 10:38:03 -07:00
ras treewide: replace '---help---' in Kconfig files with 'help' 2020-06-14 01:57:21 +09:00
realmode Rebase locking/kcsan to locking/urgent 2020-06-11 20:02:46 +02:00
tools
um kbuild: remove cc-option test of -fno-stack-protector 2020-07-07 11:13:10 +09:00
video
xen xen: branch for v5.9-rc1b 2020-08-14 13:34:37 -07:00
.gitignore
Kbuild
Kconfig A set of posix CPU timer changes which allows to defer the heavy work of 2020-08-14 14:17:51 -07:00
Kconfig.assembler x86/delay: Introduce TPAUSE delay 2020-05-07 16:06:20 +02:00
Kconfig.cpu treewide: replace '---help---' in Kconfig files with 'help' 2020-06-14 01:57:21 +09:00
Kconfig.debug locking/lockdep: Fix TRACE_IRQFLAGS vs. NMIs 2020-07-27 15:13:29 +02:00
Makefile Kbuild updates for v5.9 2020-08-09 14:10:26 -07:00
Makefile_32.cpu
Makefile.um