kernel_optimize_test/kernel/sched
Peter Zijlstra 95913d9791 sched/core: Fix TASK_DEAD race in finish_task_switch()
So the problem this patch is trying to address is as follows:

        CPU0                            CPU1

        context_switch(A, B)
                                        ttwu(A)
                                          LOCK A->pi_lock
                                          A->on_cpu == 0
        finish_task_switch(A)
          prev_state = A->state  <-.
          WMB                      |
          A->on_cpu = 0;           |
          UNLOCK rq0->lock         |
                                   |    context_switch(C, A)
                                   `--  A->state = TASK_DEAD
          prev_state == TASK_DEAD
            put_task_struct(A)
                                        context_switch(A, C)
                                        finish_task_switch(A)
                                          A->state == TASK_DEAD
                                            put_task_struct(A)

The argument being that the WMB will allow the load of A->state on CPU0
to cross over and observe CPU1's store of A->state, which will then
result in a double-drop and use-after-free.

Now the comment states (and this was true once upon a long time ago)
that we need to observe A->state while holding rq->lock because that
will order us against the wakeup; however the wakeup will not in fact
acquire (that) rq->lock; it takes A->pi_lock these days.

We can obviously fix this by upgrading the WMB to an MB, but that is
expensive, so we'd rather avoid that.

The alternative this patch takes is: smp_store_release(&A->on_cpu, 0),
which avoids the MB on some archs, but not important ones like ARM.

Reported-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: <stable@vger.kernel.org> # v3.1+
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Cc: manfred@colorfullife.com
Cc: will.deacon@arm.com
Fixes: e4a52bcb9a ("sched: Remove rq->lock from the first half of ttwu()")
Link: http://lkml.kernel.org/r/20150929124509.GG3816@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-10-06 17:05:17 +02:00
..
auto_group.c sched, timer: Convert usages of ACCESS_ONCE() in the scheduler to READ_ONCE()/WRITE_ONCE() 2015-05-08 12:11:32 +02:00
auto_group.h sched, timer: Convert usages of ACCESS_ONCE() in the scheduler to READ_ONCE()/WRITE_ONCE() 2015-05-08 12:11:32 +02:00
clock.c
completion.c
core.c sched/core: Fix TASK_DEAD race in finish_task_switch() 2015-10-06 17:05:17 +02:00
cpuacct.c
cpuacct.h
cpudeadline.c
cpudeadline.h
cpupri.c
cpupri.h
cputime.c sched/cputime: Guarantee stime + utime == rtime 2015-08-03 12:21:21 +02:00
deadline.c sched/deadline: Fix comment in enqueue_task_dl() 2015-08-12 12:06:10 +02:00
debug.c sched/fair: Provide runnable_load_avg back to cfs_rq 2015-08-03 12:24:31 +02:00
fair.c sched: Make sched_class::set_cpus_allowed() unconditional 2015-08-12 12:06:09 +02:00
features.h sched/numa: Prefer NUMA hotness over cache hotness 2015-07-07 08:46:10 +02:00
idle_task.c sched: Make sched_class::set_cpus_allowed() unconditional 2015-08-12 12:06:09 +02:00
idle.c sched/idle: Move latency tracing stop/start calls deeper inside the idle loop 2015-07-21 08:18:51 +02:00
loadavg.c sched: Move the loadavg code to a more obvious location 2015-05-08 12:04:12 +02:00
Makefile sched: Move the loadavg code to a more obvious location 2015-05-08 12:04:12 +02:00
rt.c sched: Change the sched_class::set_cpus_allowed() calling context 2015-08-12 12:06:10 +02:00
sched.h sched/core: Fix TASK_DEAD race in finish_task_switch() 2015-10-06 17:05:17 +02:00
stats.c
stats.h sched/stat: Simplify the sched_info accounting dependency 2015-07-04 10:04:30 +02:00
stop_task.c sched: Make sched_class::set_cpus_allowed() unconditional 2015-08-12 12:06:09 +02:00
wait.c userfaultfd: revert "userfaultfd: waitqueue: add nr wake parameter to __wake_up_locked_key" 2015-09-22 15:09:53 -07:00