kernel_optimize_test/ipc
Sebastian Andrzej Siewior ee51636ca5 ipc/msg: implement lockless pipelined wakeups
This patch moves the wakeup_process() invocation so it is not done under
the ipc global lock by making use of a lockless wake_q.  With this change,
the waiter is woken up once the message has been assigned and it does not
need to loop on SMP if the message points to NULL.  In the signal case we
still need to check the pointer under the lock to verify the state.

This change should also avoid the introduction of preempt_disable() in -RT
which avoids a busy-loop which pools for the NULL -> !NULL change if the
waiter has a higher priority compared to the waker.

By making use of wake_qs, the logic of sysv msg queues is greatly
simplified (and very well suited as we can batch lockless wakeups),
particularly around the lockless receive algorithm.

This has been tested with Manred's pmsg-shared tool on a "AMD A10-7800
Radeon R7, 12 Compute Cores 4C+8G":

test             |   before   |   after    | diff
-----------------|------------|------------|----------
pmsg-shared 8 60 | 19,347,422 | 30,442,191 | + ~57.34 %
pmsg-shared 4 60 | 21,367,197 | 35,743,458 | + ~67.28 %
pmsg-shared 2 60 | 22,884,224 | 24,278,200 | +  ~6.09 %

Link: http://lkml.kernel.org/r/1469748819-19484-2-git-send-email-dave@stgolabs.net
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Manfred Spraul <manfred@colorfullife.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 15:06:33 -07:00
..
compat_mq.c ipc, kernel: use Linux headers 2014-06-06 16:08:14 -07:00
compat.c ipc: resolve shadow warnings 2014-10-14 02:18:23 +02:00
ipc_sysctl.c ipc/msg: increase MSGMNI, remove scaling 2014-12-13 12:42:52 -08:00
Makefile ipc/msg: increase MSGMNI, remove scaling 2014-12-13 12:42:52 -08:00
mq_sysctl.c ipc: convert use of typedef ctl_table to struct ctl_table 2014-06-06 16:08:16 -07:00
mqueue.c fs: Replace CURRENT_TIME with current_time() for inode timestamps 2016-09-27 21:06:21 -04:00
msg.c ipc/msg: implement lockless pipelined wakeups 2016-10-11 15:06:33 -07:00
msgutil.c ipc: delete "nr_ipc_ns" 2016-08-02 19:35:44 -04:00
namespace.c Merge branch 'nsfs-ioctls' into HEAD 2016-09-22 20:00:36 -05:00
sem.c ipc/sem.c: fix complex_count vs. simple op race 2016-10-11 15:06:33 -07:00
shm.c shmem: make shmem_inode_info::lock irq-safe 2016-07-26 16:19:19 -07:00
syscall.c get rid of union semop in sys_semctl(2) arguments 2013-03-05 15:14:16 -05:00
util.c tree wide: use kvfree() than conditional kfree()/vfree() 2016-01-22 17:02:18 -08:00
util.h tree wide: use kvfree() than conditional kfree()/vfree() 2016-01-22 17:02:18 -08:00