kernel_optimize_test/Documentation
Andrey Konovalov eec028c938 kcov: remote coverage support
Patch series " kcov: collect coverage from usb and vhost", v3.

This patchset extends kcov to allow collecting coverage from backgound
kernel threads.  This extension requires custom annotations for each of
the places where coverage collection is desired.  This patchset
implements this for hub events in the USB subsystem and for vhost
workers.  See the first patch description for details about the kcov
extension.  The other two patches apply this kcov extension to USB and
vhost.

Examples of other subsystems that might potentially benefit from this
when custom annotations are added (the list is based on
process_one_work() callers for bugs recently reported by syzbot):

1. fs: writeback wb_workfn() worker,
2. net: addrconf_dad_work()/addrconf_verify_work() workers,
3. net: neigh_periodic_work() worker,
4. net/p9: p9_write_work()/p9_read_work() workers,
5. block: blk_mq_run_work_fn() worker.

These patches have been used to enable coverage-guided USB fuzzing with
syzkaller for the last few years, see the details here:

  https://github.com/google/syzkaller/blob/master/docs/linux/external_fuzzing_usb.md

This patchset has been pushed to the public Linux kernel Gerrit
instance:

  https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/1524

This patch (of 3):

Add background thread coverage collection ability to kcov.

With KCOV_ENABLE coverage is collected only for syscalls that are issued
from the current process.  With KCOV_REMOTE_ENABLE it's possible to
collect coverage for arbitrary parts of the kernel code, provided that
those parts are annotated with kcov_remote_start()/kcov_remote_stop().

This allows to collect coverage from two types of kernel background
threads: the global ones, that are spawned during kernel boot in a
limited number of instances (e.g.  one USB hub_event() worker thread is
spawned per USB HCD); and the local ones, that are spawned when a user
interacts with some kernel interface (e.g.  vhost workers).

To enable collecting coverage from a global background thread, a unique
global handle must be assigned and passed to the corresponding
kcov_remote_start() call.  Then a userspace process can pass a list of
such handles to the KCOV_REMOTE_ENABLE ioctl in the handles array field
of the kcov_remote_arg struct.  This will attach the used kcov device to
the code sections, that are referenced by those handles.

Since there might be many local background threads spawned from
different userspace processes, we can't use a single global handle per
annotation.  Instead, the userspace process passes a non-zero handle
through the common_handle field of the kcov_remote_arg struct.  This
common handle gets saved to the kcov_handle field in the current
task_struct and needs to be passed to the newly spawned threads via
custom annotations.  Those threads should in turn be annotated with
kcov_remote_start()/kcov_remote_stop().

Internally kcov stores handles as u64 integers.  The top byte of a
handle is used to denote the id of a subsystem that this handle belongs
to, and the lower 4 bytes are used to denote the id of a thread instance
within that subsystem.  A reserved value 0 is used as a subsystem id for
common handles as they don't belong to a particular subsystem.  The
bytes 4-7 are currently reserved and must be zero.  In the future the
number of bytes used for the subsystem or handle ids might be increased.

When a particular userspace process collects coverage by via a common
handle, kcov will collect coverage for each code section that is
annotated to use the common handle obtained as kcov_handle from the
current task_struct.  However non common handles allow to collect
coverage selectively from different subsystems.

Link: http://lkml.kernel.org/r/e90e315426a384207edbec1d6aa89e43008e4caf.1572366574.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: David Windsor <dwindsor@gmail.com>
Cc: Elena Reshetova <elena.reshetova@intel.com>
Cc: Anders Roxell <anders.roxell@linaro.org>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-12-04 19:44:14 -08:00
..
ABI chrome platform changes for v5.5 2019-12-03 14:37:12 -08:00
accounting
admin-guide TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
arm
arm64 arm64 updates for 5.5: 2019-11-25 15:39:19 -08:00
block block: add iostat counters for flush requests 2019-11-21 09:06:47 -07:00
bpf bpf: Add s390 testing documentation 2019-10-30 16:25:31 +01:00
cdrom
core-api lib/genalloc.c: rename addr_in_gen_pool to gen_pool_has_addr 2019-12-04 19:44:13 -08:00
cpu-freq
crypto crypto: ablkcipher - remove deprecated and unused ablkcipher support 2019-11-17 09:02:49 +08:00
dev-tools kcov: remote coverage support 2019-12-04 19:44:14 -08:00
devicetree TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
doc-guide kernel-doc: rename the kernel-doc directive 'functions' to 'identifiers' 2019-11-07 13:17:25 -07:00
driver-api Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
EDID
fault-injection
fb
features xtensa: update arch features 2019-10-20 23:48:28 -07:00
filesystems Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
firmware_class
firmware-guide ACPI: Documentation: Minor spelling fix in namespace.rst 2019-10-14 11:03:44 +02:00
fpga Documentation: fpga: dfl: add descriptions for thermal/power management interfaces 2019-10-16 19:18:26 -07:00
gpu drm/amdgpu: Improve RAS documentation (v2) 2019-11-06 16:27:48 -05:00
hid
hwmon Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
i2c Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
ia64
ide
iio
infiniband
input
isdn
kbuild kbuild: remove header compile test 2019-11-15 00:22:35 +09:00
kernel-hacking
leds
livepatch livepatch: Documentation of the new API for tracking system state changes 2019-11-01 13:08:24 +01:00
locking
m68k
maintainer docs: fix up the maintainer profile document 2019-11-25 08:42:12 -07:00
media media: Documentation: media: *_DEFAULT targets for subdevs 2019-11-10 17:49:27 +01:00
mic
mips docs: fix some broken references 2019-10-10 11:25:39 -06:00
misc-devices docs: misc: xilinx_sdfec: Actually add documentation 2019-10-11 10:01:27 -06:00
netlabel
networking Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
nios2
nvdimm libnvdimm, MAINTAINERS: Maintainer Entry Profile 2019-11-25 08:34:54 -07:00
openrisc
parisc
PCI
pcmcia
power pci-v5.5-changes 2019-12-03 13:58:22 -08:00
powerpc powerpc/fsl_booke/32: Document KASLR implementation 2019-11-13 19:27:58 +11:00
process TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
RCU Documentation: Rename rcu_node_context_switch() to rcu_note_context_switch() 2019-10-29 02:48:29 -07:00
riscv riscv-docs: correct the sequence of the magic number 2 since it's little endian 2019-10-03 10:51:42 -06:00
s390
scheduler Documentation/scheduler: fix links in sched-stats 2019-10-29 04:35:41 -06:00
scsi scsi: core: Clean up SG_NONE 2019-11-12 22:21:35 -05:00
security Documentation: security: core.rst: fix warnings 2019-11-22 10:01:47 -07:00
sh
sound ALSA: docs: Update about the new PCM sync_stop ops 2019-11-20 19:39:54 +01:00
sparc
sphinx docs, parallelism: Rearrange how jobserver reservations are made 2019-11-22 10:35:18 -07:00
sphinx-static doc-rst: Reduce CSS padding around Field 2019-10-02 10:03:06 -06:00
spi
target
timers
trace Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
translations TTY/Serial patches for 5.5-rc1 2019-12-03 14:09:14 -08:00
usb USB: rio500: Remove Rio 500 kernel driver 2019-10-04 10:53:36 +02:00
userspace-api Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
virt KVM/arm updates for Linux 5.5: 2019-11-21 09:58:35 +01:00
virtual
vm mm/hmm: remove hmm_mirror and related 2019-11-23 19:56:45 -04:00
w1 docs: w1: Fix SPDX-License-Identifier syntax 2019-10-18 09:45:56 -06:00
watchdog linux-watchdog 5.4-rc1 tag 2019-09-27 11:17:38 -07:00
x86 Merge branch 'x86-boot-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-11-26 08:40:20 -08:00
xtensa
.gitignore
asm-annotations.rst x86/ftrace: Get rid of function_hook 2019-10-25 10:52:22 +02:00
atomic_bitops.txt
atomic_t.txt
bus-virt-phys-mapping.txt
Changes
CodingStyle
conf.py doc-rst: Programmatically render MAINTAINERS into ReST 2019-10-02 10:03:17 -06:00
COPYING-logo
crc32.txt
debugging-modules.txt
debugging-via-ohci1394.txt
digsig.txt
DMA-API-HOWTO.txt
DMA-API.txt
DMA-attributes.txt dma-mapping: remove the DMA_ATTR_WRITE_BARRIER flag 2019-11-14 12:01:54 -04:00
DMA-ISA-LPC.txt
docutils.conf
dontdiff modpost: dump missing namespaces into a single modules.nsdeps file 2019-11-11 20:10:01 +09:00
futex-requeue-pi.txt
hwspinlock.txt
index.rst Here's the main documentation changes for 5.5: 2019-12-02 11:51:02 -08:00
io_ordering.txt
io-mapping.txt
IPMI.txt
IRQ-affinity.txt
IRQ-domain.txt
IRQ.txt
irqflags-tracing.txt
Kconfig
kobject.txt
kprobes.txt
kref.txt
logo.gif
lzo.txt
mailbox.txt
Makefile Kbuild updates for v5.5 2019-12-02 17:35:04 -08:00
memory-barriers.txt docs/memory-barriers.txt: Remove remaining references to mmiowb() 2019-11-22 10:00:46 -07:00
nommu-mmap.txt
padata.txt
percpu-rw-semaphore.txt
pi-futex.txt
preempt-locking.txt
rbtree.txt
remoteproc.txt
robust-futex-ABI.txt
robust-futexes.txt
rpmsg.txt
speculation.txt
static-keys.txt
SubmittingPatches
tee.txt
this_cpu_ops.txt
unaligned-memory-access.txt
xz.txt