kernel_optimize_test/kernel
Steven Rostedt (VMware) a33614d52e tracing: Restructure trace_clock_global() to never block
commit aafe104aa9096827a429bc1358f8260ee565b7cc upstream.

It was reported that a fix to the ring buffer recursion detection would
cause a hung machine when performing suspend / resume testing. The
following backtrace was extracted from debugging that case:

Call Trace:
 trace_clock_global+0x91/0xa0
 __rb_reserve_next+0x237/0x460
 ring_buffer_lock_reserve+0x12a/0x3f0
 trace_buffer_lock_reserve+0x10/0x50
 __trace_graph_return+0x1f/0x80
 trace_graph_return+0xb7/0xf0
 ? trace_clock_global+0x91/0xa0
 ftrace_return_to_handler+0x8b/0xf0
 ? pv_hash+0xa0/0xa0
 return_to_handler+0x15/0x30
 ? ftrace_graph_caller+0xa0/0xa0
 ? trace_clock_global+0x91/0xa0
 ? __rb_reserve_next+0x237/0x460
 ? ring_buffer_lock_reserve+0x12a/0x3f0
 ? trace_event_buffer_lock_reserve+0x3c/0x120
 ? trace_event_buffer_reserve+0x6b/0xc0
 ? trace_event_raw_event_device_pm_callback_start+0x125/0x2d0
 ? dpm_run_callback+0x3b/0xc0
 ? pm_ops_is_empty+0x50/0x50
 ? platform_get_irq_byname_optional+0x90/0x90
 ? trace_device_pm_callback_start+0x82/0xd0
 ? dpm_run_callback+0x49/0xc0

With the following RIP:

RIP: 0010:native_queued_spin_lock_slowpath+0x69/0x200

Since the fix to the recursion detection would allow a single recursion to
happen while tracing, this lead to the trace_clock_global() taking a spin
lock and then trying to take it again:

ring_buffer_lock_reserve() {
  trace_clock_global() {
    arch_spin_lock() {
      queued_spin_lock_slowpath() {
        /* lock taken */
        (something else gets traced by function graph tracer)
          ring_buffer_lock_reserve() {
            trace_clock_global() {
              arch_spin_lock() {
                queued_spin_lock_slowpath() {
                /* DEAD LOCK! */

Tracing should *never* block, as it can lead to strange lockups like the
above.

Restructure the trace_clock_global() code to instead of simply taking a
lock to update the recorded "prev_time" simply use it, as two events
happening on two different CPUs that calls this at the same time, really
doesn't matter which one goes first. Use a trylock to grab the lock for
updating the prev_time, and if it fails, simply try again the next time.
If it failed to be taken, that means something else is already updating
it.

Link: https://lkml.kernel.org/r/20210430121758.650b6e8a@gandalf.local.home

Cc: stable@vger.kernel.org
Tested-by: Konstantin Kharlamov <hi-angel@yandex.ru>
Tested-by: Todd Brandt <todd.e.brandt@linux.intel.com>
Fixes: b02414c8f0 ("ring-buffer: Fix recursion protection transitions between interrupt context") # started showing the problem
Fixes: 14131f2f98 ("tracing: implement trace_clock_*() APIs") # where the bug happened
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=212761
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-05-11 14:47:40 +02:00
..
bpf bpf: Fix leakage of uninitialized bpf stack under speculation 2021-05-07 11:04:31 +02:00
cgroup cgroup-v1: add disabled controller check in cgroup1_parse_param() 2021-02-17 11:02:25 +01:00
configs
debug kgdb: fix to kill breakpoints on initmem after boot 2021-03-04 11:38:46 +01:00
dma swiotlb: respect min_align_mask 2021-05-07 11:04:32 +02:00
entry x86/entry: Move nmi entry/exit into common code 2021-03-17 17:06:36 +01:00
events perf: Rework perf_event_exit_event() 2021-05-11 14:47:31 +02:00
gcov gcov: re-fix clang-11+ support 2021-04-14 08:41:58 +02:00
irq genirq/matrix: Prevent allocation counter corruption 2021-05-11 14:47:17 +02:00
kcsan kcsan, debugfs: Move debugfs file creation out of early init 2021-05-11 14:47:33 +02:00
livepatch
locking locking/qrwlock: Fix ordering in queued_write_lock_slowpath() 2021-04-28 13:40:00 +02:00
power PM: EM: postpone creating the debugfs dir till fs_initcall 2021-03-30 14:32:04 +02:00
printk printk: fix deadlock when kernel panic 2021-03-04 11:38:41 +01:00
rcu kvfree_rcu: Use same set of GFP flags as does single-argument 2021-05-11 14:47:23 +02:00
sched sched,psi: Handle potential task count underflow bugs more gracefully 2021-05-11 14:47:31 +02:00
time posix-timers: Preserve return value in clock_adjtime32() 2021-05-11 14:47:16 +02:00
trace tracing: Restructure trace_clock_global() to never block 2021-05-11 14:47:40 +02:00
.gitignore kbuild: update config_data.gz only when the content of .config is changed 2021-05-11 14:47:37 +02:00
acct.c
async.c
audit_fsnotify.c fsnotify: generalize handle_inode_event() 2020-12-30 11:54:18 +01:00
audit_tree.c fsnotify: generalize handle_inode_event() 2020-12-30 11:54:18 +01:00
audit_watch.c fsnotify: generalize handle_inode_event() 2020-12-30 11:54:18 +01:00
audit.c
audit.h
auditfilter.c
auditsc.c
backtracetest.c
bounds.c
capability.c
compat.c
configs.c
context_tracking.c
cpu_pm.c
cpu.c
crash_core.c
crash_dump.c
cred.c
delayacct.c
dma.c
exec_domain.c
exit.c kernel/io_uring: cancel io_uring before task works 2021-01-30 13:55:18 +01:00
extable.c
fail_function.c
fork.c mm/fork: clear PASID for new mm 2021-03-30 14:31:52 +02:00
freezer.c Revert "kernel: freezer should treat PF_IO_WORKER like PF_KTHREAD for freezing" 2021-04-07 15:00:14 +02:00
futex.c futex: Do not apply time namespace adjustment on FUTEX_LOCK_PI 2021-05-11 14:47:37 +02:00
gen_kheaders.sh
groups.c
hung_task.c
iomem.c
irq_work.c
jump_label.c static_call: Fix static_call_update() sanity check 2021-03-25 09:04:18 +01:00
kallsyms.c
kcmp.c exec: Transform exec_update_mutex into a rw_semaphore 2021-01-09 13:46:24 +01:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kexec_core.c kernel: kexec: remove the lock operation of system_transition_mutex 2021-02-03 23:28:37 +01:00
kexec_elf.c
kexec_file.c ima: Free IMA measurement buffer after kexec syscall 2021-03-04 11:37:50 +01:00
kexec_internal.h
kexec.c
kheaders.c
kmod.c
kprobes.c kprobes: Fix to delay the kprobes jump optimization 2021-03-04 11:38:35 +01:00
ksysfs.c
kthread.c kthread: Extract KTHREAD_IS_PER_CPU 2021-02-07 15:37:17 +01:00
latencytop.c
Makefile kbuild: update config_data.gz only when the content of .config is changed 2021-05-11 14:47:37 +02:00
module_signature.c module: harden ELF info handling 2021-03-25 09:04:11 +01:00
module_signing.c module: harden ELF info handling 2021-03-25 09:04:11 +01:00
module-internal.h
module.c module: harden ELF info handling 2021-03-25 09:04:11 +01:00
notifier.c
nsproxy.c
padata.c
panic.c
params.c
pid_namespace.c
pid.c exec: Transform exec_update_mutex into a rw_semaphore 2021-01-09 13:46:24 +01:00
profile.c
ptrace.c
range.c
reboot.c
regset.c
relay.c
resource.c
rseq.c
scftorture.c
scs.c
seccomp.c seccomp: Add missing return in non-void function 2021-03-04 11:38:32 +01:00
signal.c
smp.c smp: Process pending softirqs in flush_smp_call_function_from_idle() 2021-03-04 11:37:51 +01:00
smpboot.c kthread: Extract KTHREAD_IS_PER_CPU 2021-02-07 15:37:17 +01:00
smpboot.h
softirq.c
stackleak.c
stacktrace.c
static_call.c static_call: Align static_call_is_init() patching condition 2021-04-07 15:00:06 +02:00
stop_machine.c
sys_ni.c
sys.c
sysctl-test.c
sysctl.c sysctl.c: fix underflow value setting risk in vm_table 2021-03-17 17:06:25 +01:00
task_work.c
taskstats.c
test_kprobes.c
torture.c
tracepoint.c tracepoint: Do not fail unregistering a probe due to memory failure 2021-03-04 11:38:03 +01:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c
up.c
user_namespace.c capabilities: require CAP_SETFCAP to map uid 0 2021-05-07 11:04:31 +02:00
user-return-notifier.c
user.c
usermode_driver.c bpf: Fix umd memory leak in copy_process() 2021-03-30 14:32:03 +02:00
utsname_sysctl.c
utsname.c
watch_queue.c
watchdog_hld.c
watchdog.c
workqueue_internal.h
workqueue.c workqueue: Move the position of debug_work_activate() in __queue_work() 2021-04-14 08:42:10 +02:00