kernel_optimize_test/include/net
Jesper Dangaard Brouer 5772e9a346 qdisc: bulk dequeue support for qdiscs with TCQ_F_ONETXQUEUE
Based on DaveM's recent API work on dev_hard_start_xmit(), that allows
sending/processing an entire skb list.

This patch implements qdisc bulk dequeue, by allowing multiple packets
to be dequeued in dequeue_skb().

The optimization principle for this is two fold, (1) to amortize
locking cost and (2) avoid expensive tailptr update for notifying HW.
 (1) Several packets are dequeued while holding the qdisc root_lock,
amortizing locking cost over several packet.  The dequeued SKB list is
processed under the TXQ lock in dev_hard_start_xmit(), thus also
amortizing the cost of the TXQ lock.
 (2) Further more, dev_hard_start_xmit() will utilize the skb->xmit_more
API to delay HW tailptr update, which also reduces the cost per
packet.

One restriction of the new API is that every SKB must belong to the
same TXQ.  This patch takes the easy way out, by restricting bulk
dequeue to qdisc's with the TCQ_F_ONETXQUEUE flag, that specifies the
qdisc only have attached a single TXQ.

Some detail about the flow; dev_hard_start_xmit() will process the skb
list, and transmit packets individually towards the driver (see
xmit_one()).  In case the driver stops midway in the list, the
remaining skb list is returned by dev_hard_start_xmit().  In
sch_direct_xmit() this returned list is requeued by dev_requeue_skb().

To avoid overshooting the HW limits, which results in requeuing, the
patch limits the amount of bytes dequeued, based on the drivers BQL
limits.  In-effect bulking will only happen for BQL enabled drivers.

Small amounts for extra HoL blocking (2x MTU/0.24ms) were
measured at 100Mbit/s, with bulking 8 packets, but the
oscillating nature of the measurement indicate something, like
sched latency might be causing this effect. More comparisons
show, that this oscillation goes away occationally. Thus, we
disregard this artifact completely and remove any "magic" bulking
limit.

For now, as a conservative approach, stop bulking when seeing TSO and
segmented GSO packets.  They already benefit from bulking on their own.
A followup patch add this, to allow easier bisect-ability for finding
regressions.

Jointed work with Hannes, Daniel and Florian.

Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: David S. Miller <davem@davemloft.net>
2014-10-03 12:37:06 -07:00
..
9p 9pnet: p9_client->conn field is unused. Remove it. 2014-03-25 16:38:16 -05:00
bluetooth Bluetooth: Fix mgmt pairing failure when authentication fails 2014-09-09 03:12:15 +02:00
caif caif_hsi.h: Remove extern from function prototypes 2013-09-23 16:29:41 -04:00
irda include/net/: Fix FSF address in file headers 2013-12-06 12:37:56 -05:00
iucv
netfilter netfilter: nf_tables: store and dump set policy 2014-09-29 11:28:03 +02:00
netns Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec-next 2014-09-28 17:19:15 -04:00
nfc NFC: digital: Add 'tg_listen_md' and 'tg_get_rf_tech' driver hooks 2014-07-23 01:17:31 +02:00
phonet
sctp net: sctp: fix ABI mismatch through sctp_assoc_to_state helper 2014-08-29 20:31:08 -07:00
tc_act net_sched: act: hide struct tcf_common from API 2014-02-12 19:23:32 -05:00
6lowpan.h 6lowpan: remove unused function 2014-07-30 19:28:41 +02:00
act_api.h net_sched: act: refuse to remove bound action outside 2014-02-12 19:23:32 -05:00
addrconf.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2014-09-23 12:09:27 -04:00
af_ieee802154.h ieee802154: add dgram sockopts for security control 2014-05-16 17:23:41 -04:00
af_rxrpc.h
af_unix.h af_unix: improve STREAM behavior with fragmented memory 2013-08-10 01:16:44 -07:00
af_vsock.h vsock: Make transport the proto owner 2014-05-05 13:13:50 -04:00
ah.h ipsec: Remove obsolete MAX_AH_AUTH_LEN 2014-09-18 10:54:36 +02:00
arp.h arp: make arp_invalidate static 2013-12-28 17:02:46 -05:00
atmclip.h
ax25.h
ax88796.h
busy_poll.h sched, net: Fixup busy_loop_us_clock() 2014-01-13 17:39:11 +01:00
cfg80211-wext.h
cfg80211.h cfg80211: allow requesting SMPS mode on ap start 2014-09-11 13:37:02 +02:00
checksum.h net: optimise csum_replace4() 2014-09-26 16:14:16 -04:00
cipso_ipv4.h cipso: cleanup cipso_v4_translate() when !CONFIG_NETLABEL 2013-12-10 17:56:54 -05:00
cls_cgroup.h cgroup: clean up cgroup_subsys names and initialization 2014-02-08 10:36:58 -05:00
codel.h net: use ktime_get_ns() and ktime_get_real_ns() helpers 2014-08-22 19:57:23 -07:00
compat.h compat.h: Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
datalink.h net: Move prototype declaration to header file include/net/datalink.h from net/ipx/af_ipx.c 2014-02-09 17:32:50 -08:00
dcbevent.h include/net/: Fix FSF address in file headers 2013-12-06 12:37:56 -05:00
dcbnl.h Update setapp/getapp prototypes in dcbnl_rtnl_ops to return int instead of u8 2014-07-17 16:02:29 -07:00
dn_dev.h dn_dev: add support for IFA_FLAGS nl attribute 2013-12-10 21:50:00 -05:00
dn_fib.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_neigh.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_nsp.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_route.h net: Move prototype declaration to appropriate header file from decnet/af_decnet.c 2014-02-09 17:32:49 -08:00
dn.h net: Move prototype declaration to header file include/net/dn.h from net/decnet/af_decnet.c 2014-02-09 17:32:49 -08:00
dsa.h net: dsa: allow switches driver to implement get/set EEE 2014-09-28 17:14:09 -04:00
dsfield.h
dst_ops.h
dst.h xfrm: Generate queueing routes only from route lookup functions 2014-09-16 10:08:49 +02:00
esp.h net: move pskb_put() to core code 2013-11-07 19:28:58 -05:00
ethoc.h net: ethoc: set up MII management bus clock 2014-02-04 20:19:51 -08:00
fib_rules.h fib_rules.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
firewire.h
flow_keys.h net: Add function for parsing the header length out of linear ethernet frames 2014-09-05 17:47:02 -07:00
flow.h ipv4, fib: pass LOOPBACK_IFINDEX instead of 0 to flowi4_iif 2014-04-16 15:05:11 -04:00
flowcache.h flowcache: Make flow cache name space aware 2014-02-12 07:02:11 +01:00
garp.h garp.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
gen_stats.h net: sched: enable per cpu qstats 2014-09-30 01:02:26 -04:00
genetlink.h genetlink: add function genl_has_listeners() 2014-09-19 17:28:26 -04:00
gre.h gre: Call gso_make_checksum 2014-06-04 22:46:38 -07:00
gro_cells.h
icmp.h icmp.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
ieee80211_radiotap.h mac80211: propagate STBC / LDPC flags to radiotap 2014-02-06 09:34:58 +01:00
ieee802154_netdev.h ieee802154, mac802154: implement devkey record option 2014-05-16 17:23:42 -04:00
ieee802154.h ieee802154: add definitions for link-layer security and header functions 2014-05-15 15:51:42 -04:00
if_inet6.h ipv6: addrconf: implement address generation modes 2014-07-11 15:05:45 -07:00
inet6_connection_sock.h ipv4: add a sock pointer to ip_queue_xmit() 2014-04-15 12:58:34 -04:00
inet6_hashtables.h ipv6: split inet6_ehashfn to hash functions per compilation unit 2013-10-19 19:45:34 -04:00
inet_common.h inet*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
inet_connection_sock.h tcp: avoid possible arithmetic overflows 2014-09-22 16:27:10 -04:00
inet_ecn.h tunnel: fix RFC number in comment for INET_ECN_decapsulate() 2014-05-07 15:30:52 -04:00
inet_frag.h inet: frags: use kmem_cache for inet_frag_queue 2014-08-02 15:31:31 -07:00
inet_hashtables.h net: Use a more standard macro for INET_ADDR_COOKIE 2014-05-14 16:07:23 -04:00
inet_sock.h net: remove inet6_reqsk_alloc 2014-06-27 15:53:35 -07:00
inet_timewait_sock.h inet: move ipv6only in sock_common 2014-07-01 23:46:21 -07:00
inetpeer.h inet: remove dead inetpeer sequence code 2014-09-08 16:42:42 -07:00
ip6_checksum.h net: add gro_compute_pseudo functions 2014-08-24 18:09:23 -07:00
ip6_fib.h ipv6: remove rt6i_genid 2014-09-30 14:00:48 -04:00
ip6_route.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2014-05-24 00:32:30 -04:00
ip6_tunnel.h net: unify the pcpu_tstats and br_cpu_netstats as one 2014-01-04 20:10:24 -05:00
ip_fib.h ipv4: harden fnhe_hashfun() 2014-09-05 17:40:33 -07:00
ip_tunnels.h net: Changes to ip_tunnel to support foo-over-udp encapsulation 2014-09-19 17:15:32 -04:00
ip_vs.h ipvs: prevent mixing heterogeneous pools and synchronization 2014-09-16 09:03:35 +09:00
ip.h ipv4: rename ip_options_echo to __ip_options_echo() 2014-09-28 16:35:42 -04:00
ipcomp.h
ipconfig.h
ipv6.h ipv6: add a struct inet6_skb_parm param to ipv6_opt_accepted() 2014-09-28 16:35:43 -04:00
ipx.h net: Move prototype declaration to header file include/net/ipx.h from net/ipx/af_ipx.c 2014-02-09 17:32:50 -08:00
iw_handler.h iw_handler.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
lapb.h lapb.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
lib80211.h
llc_c_ac.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_c_ev.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_c_st.h
llc_conn.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_if.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_pdu.h net: llc: fix order of evaluation in llc_conn_ac_inc_vr_by_1 2014-01-01 22:22:43 -05:00
llc_s_ac.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_s_ev.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_s_st.h
llc_sap.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc.h llc: make lock static 2014-01-03 20:56:48 -05:00
mac80211.h mac80211: replace SMPS hw flags with wiphy feature bits 2014-09-11 13:37:02 +02:00
mac802154.h mac802154: at86rf230: add hw flags and merge ops 2014-07-07 21:29:24 -07:00
mip6.h include/net/: Fix FSF address in file headers 2013-12-06 12:37:56 -05:00
mld.h ipv6: mld: answer mldv2 queries with mldv1 reports in mldv1 fallback 2014-09-22 16:23:15 -04:00
mrp.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
ndisc.h ndisc.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
neighbour.h netfilter: bridge: move br_netfilter out of the core 2014-09-26 18:42:31 +02:00
net_namespace.h ipv6: remove rt6i_genid 2014-09-30 14:00:48 -04:00
net_ratelimit.h
netdma.h
netevent.h netevent/netlink.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
netlabel.h netlabel: fix the netlbl_catmap_setlong() dummy function 2014-08-07 20:55:21 -04:00
netlink.h netlink: Fix shadow warning on jiffies 2014-07-28 17:20:43 -07:00
netprio_cgroup.h cgroup: clean up cgroup_subsys names and initialization 2014-02-08 10:36:58 -05:00
netrom.h netrom.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
nexthop.h
nl802154.h ieee802154: use ieee802154_addr instead of *_sa variants 2014-03-14 22:15:26 -04:00
p8022.h p8022.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
ping.h ipv6: make IPV6_RECVPKTINFO work for ipv4 datagrams 2014-01-19 19:53:18 -08:00
pkt_cls.h net_sched: avoid calling tcf_unbind_filter() in call_rcu callback 2014-10-01 22:00:42 -04:00
pkt_sched.h net: use ktime_get_ns() and ktime_get_real_ns() helpers 2014-08-22 19:57:23 -07:00
protocol.h net: Eliminate no_check from protosw 2014-05-23 16:28:53 -04:00
psnap.h psnap.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
raw.h raw/rawv6.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
rawv6.h raw/rawv6.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
red.h reciprocal_divide: update/correction of the algorithm 2014-01-21 23:17:20 -08:00
regulatory.h regulatory: add NUL to alpha2 2014-08-15 13:51:40 +02:00
request_sock.h inet: includes a sock_common in request_sock 2013-10-10 00:08:07 -04:00
rose.h rose.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
route.h ipv4: remove ip_rt_dump from route.c 2014-03-24 12:45:01 -04:00
rtnetlink.h net: rtnetlink - make create_link take name_assign_type 2014-07-15 16:13:07 -07:00
sch_generic.h qdisc: bulk dequeue support for qdiscs with TCQ_F_ONETXQUEUE 2014-10-03 12:37:06 -07:00
scm.h scm.h: Remove extern from function prototypes 2013-09-23 01:51:09 -04:00
secure_seq.h inetpeer: get rid of ip_id_count 2014-06-02 11:00:41 -07:00
slhc_vj.h
snmp.h net: fix sparse warnings in SNMP_UPD_PO_STATS(_BH) 2014-09-19 17:22:31 -04:00
sock.h net-timestamp: optimize sock_tx_timestamp default path 2014-09-09 17:34:41 -07:00
Space.h drivers: net: Include new header file in sbni.c 2013-12-19 18:51:20 -05:00
stp.h stp.h: Remove extern from function prototypes 2013-09-23 01:51:09 -04:00
tcp_memcontrol.h tcp_memcontrol: Kill struct tcp_memcontrol 2013-10-21 18:43:02 -04:00
tcp_states.h
tcp.h tcp: Change tcp_slow_start function to return void 2014-09-30 17:09:16 -04:00
timewait_sock.h
transp_v6.h ipv6: make IPV6_RECVPKTINFO work for ipv4 datagrams 2014-01-19 19:53:18 -08:00
tso.h net: Add a software TSO helper API 2014-05-22 14:57:15 -04:00
udp_tunnel.h udp-tunnel: Add a few more UDP tunnel APIs 2014-09-19 15:57:15 -04:00
udp.h udp: Generalize skb_udp_segment 2014-10-01 21:35:51 -04:00
udplite.h udplite.h: Remove extern from function prototypes 2013-09-23 16:29:40 -04:00
vsock_addr.h
vxlan.h vxlan: Call udp_flow_src_port 2014-07-07 21:14:21 -07:00
wext.h wext.h: Remove extern from function prototypes 2013-09-23 16:29:40 -04:00
wimax.h net: treewide: Fix typo found in DocBook/networking.xml 2014-09-05 17:35:28 -07:00
wpan-phy.h ieee802154: add netlink APIs for smartMAC configuration 2014-02-17 16:42:39 -05:00
x25.h x25.h: Remove extern from function prototypes 2013-09-23 16:29:41 -04:00
x25device.h
xfrm.h xfrm: configure policy hash table thresholds by netlink 2014-09-02 13:37:56 +02:00