kernel_optimize_test/block
Jens Axboe c616cbee97 blk-mq: punt failed direct issue to dispatch list
After the direct dispatch corruption fix, we permanently disallow direct
dispatch of non read/write requests. This works fine off the normal IO
path, as they will be retried like any other failed direct dispatch
request. But for the blk_insert_cloned_request() that only DM uses to
bypass the bottom level scheduler, we always first attempt direct
dispatch. For some types of requests, that's now a permanent failure,
and no amount of retrying will make that succeed. This results in a
livelock.

Instead of making special cases for what we can direct issue, and now
having to deal with DM solving the livelock while still retaining a BUSY
condition feedback loop, always just add a request that has been through
->queue_rq() to the hardware queue dispatch list. These are safe to use
as no merging can take place there. Additionally, if requests do have
prepped data from drivers, we aren't dependent on them not sharing space
in the request structure to safely add them to the IO scheduler lists.

This basically reverts ffe81d4532 and is based on a patch from Ming,
but with the list insert case covered as well.

Fixes: ffe81d4532 ("blk-mq: fix corruption with direct issue")
Cc: stable@vger.kernel.org
Suggested-by: Ming Lei <ming.lei@redhat.com>
Reported-by: Bart Van Assche <bvanassche@acm.org>
Tested-by: Ming Lei <ming.lei@redhat.com>
Acked-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2018-12-07 08:16:11 -07:00
..
partitions partitions/aix: append null character to print data from disk 2018-07-27 09:17:41 -06:00
badblocks.c badblocks: fix wrong return value in badblocks_set if badblocks are disabled 2017-11-03 11:29:50 -07:00
bfq-cgroup.c blkcg: revert blkcg cleanups series 2018-11-01 19:59:53 -06:00
bfq-iosched.c block, bfq: fix decrement of num_active_groups 2018-12-07 07:40:07 -07:00
bfq-iosched.h block, bfq: fix decrement of num_active_groups 2018-12-07 07:40:07 -07:00
bfq-wf2q.c block, bfq: fix decrement of num_active_groups 2018-12-07 07:40:07 -07:00
bio-integrity.c block: remove bio_rewind_iter() 2018-09-06 15:12:24 -06:00
bio.c block: copy ioprio in __bio_clone_fast() and bounce 2018-11-12 10:35:25 -07:00
blk-cgroup.c blkcg: revert blkcg cleanups series 2018-11-01 19:59:53 -06:00
blk-core.c SCSI: fix queue cleanup race before queue initialization is done 2018-11-14 08:19:10 -07:00
blk-exec.c blk-mq-sched: remove unused 'can_block' arg from blk_mq_sched_insert_request 2018-01-17 09:49:21 -07:00
blk-flush.c blk-mq: change gfp flags to GFP_NOIO in blk_mq_realloc_hw_ctxs 2018-10-13 15:42:01 -06:00
blk-integrity.c block: merge BIOVEC_SEG_BOUNDARY into biovec_phys_mergeable 2018-09-24 12:33:57 -06:00
blk-ioc.c block, mm: remove unnecessary __GFP_HIGH flag 2018-07-09 09:07:54 -06:00
blk-iolatency.c for-linus-20181102 2018-11-02 11:25:48 -07:00
blk-lib.c block: fix 32 bit overflow in __blkdev_issue_discard() 2018-11-14 08:17:18 -07:00
blk-map.c Merge branch 'for-4.16/block' of git://git.kernel.dk/linux-block 2018-01-29 11:51:49 -08:00
blk-merge.c block: fix single range discard merge 2018-11-30 10:07:57 -07:00
blk-mq-cpumap.c blk-mq: don't keep offline CPUs mapped to hctx 0 2018-04-10 08:38:46 -06:00
blk-mq-debugfs-zoned.c block: Make struct request_queue smaller for CONFIG_BLK_DEV_ZONED=n 2018-07-09 09:07:52 -06:00
blk-mq-debugfs.c block: add a report_zones method 2018-10-25 11:17:40 -06:00
blk-mq-debugfs.h block: Make struct request_queue smaller for CONFIG_BLK_DEV_ZONED=n 2018-07-09 09:07:52 -06:00
blk-mq-pci.c blk-mq: code clean-up by adding an API to clear set->mq_map 2018-07-09 09:07:53 -06:00
blk-mq-rdma.c block: Add rdma affinity based queue mapping helper 2017-08-08 14:58:03 -04:00
blk-mq-sched.c blk-mq: init hctx sched after update ctx and hctx mapping 2018-08-21 09:02:55 -06:00
blk-mq-sched.h block: move call of scheduler's ->completed_request() hook 2018-09-27 17:34:52 -06:00
blk-mq-sysfs.c block drivers/block: Use octal not symbolic permissions 2018-05-24 13:38:59 -06:00
blk-mq-tag.c This is the 4.19-rc6 release 2018-10-01 08:58:57 -06:00
blk-mq-tag.h Merge branch 'for-4.15/block' of git://git.kernel.dk/linux-block 2017-11-14 15:32:19 -08:00
blk-mq-virtio.c blk-mq: provide a default queue mapping for virtio device 2017-02-27 20:54:05 +02:00
blk-mq.c blk-mq: punt failed direct issue to dispatch list 2018-12-07 08:16:11 -07:00
blk-mq.h blk-mq: issue directly if hw queue isn't busy in case of 'none' 2018-07-17 16:04:00 -06:00
blk-pm.c blk-mq: Enable support for runtime power management 2018-09-26 15:11:29 -06:00
blk-pm.h block: Split blk_pm_add_request() and blk_pm_put_request() 2018-09-26 15:11:28 -06:00
blk-rq-qos.c blk-rq-qos: make depth comparisons unsigned 2018-07-22 11:30:53 -06:00
blk-rq-qos.h blk-rq-qos: make depth comparisons unsigned 2018-07-22 11:30:53 -06:00
blk-settings.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
blk-softirq.c blk-mq: complete req in softirq context in case of single queue 2018-10-08 10:50:43 -06:00
blk-stat.c block: export blk_stat_enable_accounting() 2018-09-27 17:34:54 -06:00
blk-stat.h blk-stat: export helpers for modifying blk_rq_stat 2018-07-09 09:07:54 -06:00
blk-sysfs.c block: call rq_qos_exit() after queue is frozen 2018-10-31 08:40:36 -06:00
blk-tag.c for-linus-20180616 2018-06-17 05:37:55 +09:00
blk-throttle.c blkcg: revert blkcg cleanups series 2018-11-01 19:59:53 -06:00
blk-timeout.c blk-mq: Fix timeout handling in case the timeout handler returns BLK_EH_DONE 2018-06-23 10:25:45 -06:00
blk-wbt.c blk-wbt: wake up all when we scale up, not down 2018-10-11 13:31:28 -06:00
blk-wbt.h block: remove external dependency on wbt_flags 2018-07-09 09:07:54 -06:00
blk-zoned.c block: Introduce blk_revalidate_disk_zones() 2018-10-25 11:17:40 -06:00
blk.h block: make sure discard bio is aligned with logical block size 2018-11-09 06:23:14 -07:00
bounce.c block: copy ioprio in __bio_clone_fast() and bounce 2018-11-12 10:35:25 -07:00
bsg-lib.c block/bsg-lib: use PTR_ERR_OR_ZERO to simplify the flow path 2018-08-01 09:13:03 -06:00
bsg.c block: bsg: move atomic_t ref_count variable to refcount API 2018-08-27 19:17:02 -06:00
cfq-iosched.c blkcg: revert blkcg cleanups series 2018-11-01 19:59:53 -06:00
cmdline-parser.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
compat_ioctl.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
deadline-iosched.c block drivers/block: Use octal not symbolic permissions 2018-05-24 13:38:59 -06:00
elevator.c This is the 4.19-rc6 release 2018-10-01 08:58:57 -06:00
genhd.c This is the 4.19-rc6 release 2018-10-01 08:58:57 -06:00
ioctl.c block: Introduce BLKGETNRZONES ioctl 2018-10-25 11:17:40 -06:00
ioprio.c block: add ioprio_check_cap function 2018-05-31 10:50:54 -04:00
Kconfig block: remove redundant 'default n' from Kconfig-s 2018-10-10 14:11:07 -06:00
Kconfig.iosched block: remove redundant 'default n' from Kconfig-s 2018-10-10 14:11:07 -06:00
kyber-iosched.c kyber: fix integer overflow of latency targets on 32-bit 2018-09-28 10:49:39 -06:00
Makefile block: Move power management code into a new source file 2018-09-26 15:11:28 -06:00
mq-deadline.c block drivers/block: Use octal not symbolic permissions 2018-05-24 13:38:59 -06:00
noop-iosched.c block: move existing elevator ops to union 2017-01-17 10:03:33 -07:00
opal_proto.h block: sed-opal: Set MBRDone on S3 resume path if TPER is MBREnabled 2017-09-11 09:45:52 -06:00
partition-generic.c block: use nanosecond resolution for iostat 2018-09-21 20:26:59 -06:00
scsi_ioctl.c block: consistently use GFP_NOIO instead of __GFP_NORECLAIM 2018-05-14 08:55:18 -06:00
sed-opal.c block: sed-opal: Fix a couple off by one bugs 2018-06-20 12:04:06 -06:00
t10-pi.c block: move dif_prepare/dif_complete functions to block layer 2018-07-30 08:27:02 -06:00