kernel_optimize_test/block
Tejun Heo 780db2071a blk-mq: decouble blk-mq freezing from generic bypassing
blk_mq freezing is entangled with generic bypassing which bypasses
blkcg and io scheduler and lets IO requests fall through the block
layer to the drivers in FIFO order.  This allows forward progress on
IOs with the advanced features disabled so that those features can be
configured or altered without worrying about stalling IO which may
lead to deadlock through memory allocation.

However, generic bypassing doesn't quite fit blk-mq.  blk-mq currently
doesn't make use of blkcg or ioscheds and it maps bypssing to
freezing, which blocks request processing and drains all the in-flight
ones.  This causes problems as bypassing assumes that request
processing is online.  blk-mq works around this by conditionally
allowing request processing for the problem case - during queue
initialization.

Another weirdity is that except for during queue cleanup, bypassing
started on the generic side prevents blk-mq from processing new
requests but doesn't drain the in-flight ones.  This shouldn't break
anything but again highlights that something isn't quite right here.

The root cause is conflating blk-mq freezing and generic bypassing
which are two different mechanisms.  The only intersecting purpose
that they serve is during queue cleanup.  Let's properly separate
blk-mq freezing from generic bypassing and simply use it where
necessary.

* request_queue->mq_freeze_depth is added and
  blk_mq_[un]freeze_queue() now operate on this counter instead of
  ->bypass_depth.  The replacement for QUEUE_FLAG_BYPASS isn't added
  but the counter is tested directly.  This will be further updated by
  later changes.

* blk_mq_drain_queue() is dropped and "__" prefix is dropped from
  blk_mq_freeze_queue().  Queue cleanup path now calls
  blk_mq_freeze_queue() directly.

* blk_queue_enter()'s fast path condition is simplified to simply
  check @q->mq_freeze_depth.  Previously, the condition was

	!blk_queue_dying(q) &&
	    (!blk_queue_bypass(q) || !blk_queue_init_done(q))

  mq_freeze_depth is incremented right after dying is set and
  blk_queue_init_done() exception isn't necessary as blk-mq doesn't
  start frozen, which only leaves the blk_queue_bypass() test which
  can be replaced by @q->mq_freeze_depth test.

This change simplifies the code and reduces confusion in the area.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Nicholas A. Bellinger <nab@linux-iscsi.org>
Signed-off-by: Jens Axboe <axboe@fb.com>
2014-07-01 10:31:13 -06:00
..
partitions block: Use macros from compiler.h instead of __attribute__((...)) 2014-02-18 12:20:01 -08:00
bio-integrity.c block: move bio.c and bio-integrity.c from fs/ to block/ 2014-05-19 08:34:46 -06:00
bio.c block: add support for limiting gaps in SG lists 2014-06-24 16:22:24 -06:00
blk-cgroup.c Revert "block: add __init to blkcg_policy_register" 2014-06-22 16:34:11 -06:00
blk-cgroup.h Revert "block: add __init to blkcg_policy_register" 2014-06-22 16:34:11 -06:00
blk-core.c blk-mq: decouble blk-mq freezing from generic bypassing 2014-07-01 10:31:13 -06:00
blk-exec.c block: blk-exec.c: Cleaning up local variable address returnd 2014-06-08 19:51:31 -06:00
blk-flush.c block: remove elv_abort_queue and blk_abort_flushes 2014-06-11 15:31:21 -06:00
blk-integrity.c bio-integrity: Convert to bvec_iter 2013-11-23 22:33:50 -08:00
blk-ioc.c block: Substitute rcu_access_pointer() for rcu_dereference_raw() 2014-02-18 12:21:26 -08:00
blk-iopoll.c Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next 2014-06-03 12:57:53 -07:00
blk-lib.c block/blk-lib.c: make __blkdev_issue_zeroout static 2014-05-26 17:39:09 -06:00
blk-map.c block: remove struct request buffer member 2014-04-15 14:03:02 -06:00
blk-merge.c block: add support for limiting gaps in SG lists 2014-06-24 16:22:24 -06:00
blk-mq-cpu.c blk-mq: add file comments and update copyright notices 2014-05-28 10:15:41 -06:00
blk-mq-cpumap.c blk-mq: add file comments and update copyright notices 2014-05-28 10:15:41 -06:00
blk-mq-sysfs.c blk-mq: blk_mq_unregister_hctx() can be static 2014-05-30 10:31:13 -06:00
blk-mq-tag.c blk-mq: bitmap tag: fix races in bt_get() function 2014-06-17 22:13:08 -07:00
blk-mq-tag.h blk-mq: bitmap tag: fix races on shared ::wake_index fields 2014-06-17 22:12:35 -07:00
blk-mq.c blk-mq: decouble blk-mq freezing from generic bypassing 2014-07-01 10:31:13 -06:00
blk-mq.h blk-mq: decouble blk-mq freezing from generic bypassing 2014-07-01 10:31:13 -06:00
blk-settings.c block: ensure that bio_add_page() always accepts a page for an empty bio 2014-06-10 12:53:56 -06:00
blk-softirq.c block: fix regression with block enabled tagging 2014-04-09 21:54:06 -06:00
blk-sysfs.c block, blk-mq: draining can't be skipped even if bypass_depth was non-zero 2014-07-01 10:29:17 -06:00
blk-tag.c
blk-throttle.c Merge branch 'for-3.16' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup 2014-06-09 15:03:33 -07:00
blk-timeout.c block: ensure that the timer is always added 2014-05-30 15:41:39 -06:00
blk.h block: remove elv_abort_queue and blk_abort_flushes 2014-06-11 15:31:21 -06:00
bounce.c mm: convert some level-less printks to pr_* 2014-06-06 16:08:18 -07:00
bsg-lib.c
bsg.c block: add blk_rq_set_block_pc() 2014-06-06 07:57:37 -06:00
cfq-iosched.c Merge branch 'for-3.16' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup 2014-06-09 15:03:33 -07:00
cmdline-parser.c block: remove unrelated header files and export symbol 2014-01-21 20:18:26 -08:00
compat_ioctl.c kernel-wide: fix missing validations on __get/__put/__copy_to/__copy_from_user() 2013-09-11 15:58:18 -07:00
deadline-iosched.c block: Stop abusing csd.list for fifo_time 2014-02-24 14:46:32 -08:00
elevator.c Revert "block: add __init to elv_register" 2014-06-22 16:34:11 -06:00
genhd.c block: Convert kmalloc_node(...GFP_ZERO...) to kzalloc_node(...) 2013-09-11 13:22:03 -06:00
ioctl.c block: replace IS_ERR and PTR_ERR with PTR_ERR_OR_ZERO 2013-11-08 09:05:31 -07:00
ioprio.c block: move ioprio.c from fs/ to block/ 2014-05-19 11:02:18 -06:00
Kconfig block: change config option name for cmdline partition parsing 2013-09-30 14:31:02 -07:00
Kconfig.iosched
Makefile block: move mm/bounce.c to block/ 2014-05-19 20:01:52 -06:00
noop-iosched.c elevator: Fix a race in elevator switching 2013-07-03 13:25:24 +02:00
partition-generic.c
scsi_ioctl.c block: add blk_rq_set_block_pc() 2014-06-06 07:57:37 -06:00