AcoSail/kernel_optimize_test
8
0
Fork 0
forked from luck/tmp_suning_uos_patched
Code Pull Requests Activity
c8c8b12709
kernel_optimize_test/net/ipv4
History
Eric Dumazet c8c8b12709 udp: under rx pressure, try to condense skbs 
Under UDP flood, many softirq producers try to add packets to
UDP receive queue, and one user thread is burning one cpu trying
to dequeue packets as fast as possible.

Two parts of the per packet cost are :
- copying payload from kernel space to user space,
- freeing memory pieces associated with skb.

If socket is under pressure, softirq handler(s) can try to pull in
skb->head the payload of the packet if it fits.

Meaning the softirq handler(s) can free/reuse the page fragment
immediately, instead of letting udp_recvmsg() do this hundreds of usec
later, possibly from another node.

Additional gains :
- We reduce skb->truesize and thus can store more packets per SO_RCVBUF
- We avoid cache line misses at copyout() time and consume_skb() time,
and avoid one put_page() with potential alien freeing on NUMA hosts.

This comes at the cost of a copy, bounded to available tail room, which
is usually small. (We might have to fix GRO_MAX_HEAD which looks bigger
than necessary)

This patch gave me about 5 % increase in throughput in my tests.

skb_condense() helper could probably used in other contexts.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-12-08 13:25:07 -05:00
..
netfilter netfilter: rpfilter: bypass ipv4 lbcast packets with zeronet source 2016-12-07 13:22:50 +01:00
af_inet.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-03 12:29:53 -05:00
ah4.c
arp.c
cipso_ipv4.c
datagram.c
devinet.c
esp4.c esp4: Fix integrity verification when ESN are used 2016-11-30 11:09:39 +01:00
fib_frontend.c ipv4: fib: Allow for consistent FIB dumping 2016-12-03 19:29:35 -05:00
fib_lookup.h
fib_rules.c switchdev: remove FIB offload infrastructure 2016-09-28 04:48:00 -04:00
fib_semantics.c ipv4: fib: Export free_fib_info() 2016-12-03 19:29:35 -05:00
fib_trie.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-06 21:33:19 -05:00
fou.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-10-30 12:42:58 -04:00
gre_demux.c
gre_offload.c net: add recursion limit to GRO 2016-10-20 14:32:22 -04:00
icmp.c icmp: correct return value of icmp_rcv() 2016-12-08 11:24:23 -05:00
igmp.c igmp: do not remove igmp souce list info when set link down 2016-11-15 19:51:16 -05:00
inet_connection_sock.c net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
inet_diag.c tcp: no longer hold ehash lock while calling tcp_get_info() 2016-11-09 13:02:27 -05:00
inet_fragment.c
inet_hashtables.c net: Require exact match for TCP socket lookups if dif is l3mdev 2016-10-17 10:17:05 -04:00
inet_timewait_sock.c
inetpeer.c
ip_forward.c ipv4: allow local fragmentation in ip_finish_output_gso() 2016-11-03 16:10:26 -04:00
ip_fragment.c
ip_gre.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
ip_input.c
ip_options.c
ip_output.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-03 12:29:53 -05:00
ip_sockglue.c net/sock: add an explicit sk argument for ip_cmsg_recv_offset() 2016-11-07 13:24:41 -05:00
ip_tunnel_core.c ipv4: allow local fragmentation in ip_finish_output_gso() 2016-11-03 16:10:26 -04:00
ip_tunnel.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
ip_vti.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
ipcomp.c
ipconfig.c
ipip.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
ipmr.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-11-15 10:54:36 -05:00
Kconfig Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-03 12:29:53 -05:00
Makefile net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
netfilter.c netfilter: Update ip_route_me_harder to consider L3 domain 2016-11-24 12:44:36 +01:00
ping.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-06 21:33:19 -05:00
proc.c net: Suppress the "Comparison to NULL could be written" warnings 2016-09-30 01:50:45 -04:00
protocol.c
raw_diag.c net: ip, raw_diag -- Use jump for exiting from nested loop 2016-11-03 15:25:26 -04:00
raw.c net: Update raw socket bind to consider l3 domain 2016-11-07 13:14:34 -05:00
route.c route: Set lwtstate for local traffic and cached input dsts 2016-12-02 10:51:49 -05:00
syncookies.c tcp: randomize tcp timestamp offsets for each connection 2016-12-02 12:49:59 -05:00
sysctl_net_ipv4.c ipv4: use the right lock for ping_group_range 2016-10-22 16:23:12 -04:00
tcp_bbr.c tcp_bbr: add a state transition diagram and accompanying comment 2016-10-29 17:12:43 -04:00
tcp_bic.c
tcp_cdg.c tcp: cdg: rename struct minmax in tcp_cdg.c to avoid a naming conflict 2016-09-21 00:22:59 -04:00
tcp_cong.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-11-22 13:27:16 -05:00
tcp_cubic.c
tcp_dctcp.c Revert "dctcp: update cwnd on congestion event" 2016-12-06 11:34:24 -05:00
tcp_diag.c
tcp_fastopen.c
tcp_highspeed.c tcp: add cwnd_undo functions to various tcp cc algorithms 2016-11-21 13:20:17 -05:00
tcp_htcp.c
tcp_hybla.c tcp: make undo_cwnd mandatory for congestion modules 2016-11-21 13:20:17 -05:00
tcp_illinois.c tcp: add cwnd_undo functions to various tcp cc algorithms 2016-11-21 13:20:17 -05:00
tcp_input.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-06 21:33:19 -05:00
tcp_ipv4.c tcp: tsq: move tsq_flags close to sk_wmem_alloc 2016-12-05 13:32:24 -05:00
tcp_lp.c tcp: make undo_cwnd mandatory for congestion modules 2016-11-21 13:20:17 -05:00
tcp_metrics.c genetlink: mark families as __ro_after_init 2016-10-27 16:16:09 -04:00
tcp_minisocks.c tcp: randomize tcp timestamp offsets for each connection 2016-12-02 12:49:59 -05:00
tcp_nv.c
tcp_offload.c gso: Support partial splitting at the frag_list pointer 2016-09-19 20:59:34 -04:00
tcp_output.c tcp: tsq: move tsq_flags close to sk_wmem_alloc 2016-12-05 13:32:24 -05:00
tcp_probe.c
tcp_rate.c tcp: export data delivery rate 2016-09-21 00:23:00 -04:00
tcp_recovery.c
tcp_scalable.c tcp: add cwnd_undo functions to various tcp cc algorithms 2016-11-21 13:20:17 -05:00
tcp_timer.c tcp: tsq: move tsq_flags close to sk_wmem_alloc 2016-12-05 13:32:24 -05:00
tcp_vegas.c tcp: make undo_cwnd mandatory for congestion modules 2016-11-21 13:20:17 -05:00
tcp_vegas.h
tcp_veno.c tcp: add cwnd_undo functions to various tcp cc algorithms 2016-11-21 13:20:17 -05:00
tcp_westwood.c tcp: make undo_cwnd mandatory for congestion modules 2016-11-21 13:20:17 -05:00
tcp_yeah.c tcp: add cwnd_undo functions to various tcp cc algorithms 2016-11-21 13:20:17 -05:00
tcp.c tcp: tsq: move tsq_flags close to sk_wmem_alloc 2016-12-05 13:32:24 -05:00
tunnel4.c
udp_diag.c
udp_impl.h udplite: call proper backlog handlers 2016-11-24 15:32:14 -05:00
udp_offload.c net: add recursion limit to GRO 2016-10-20 14:32:22 -04:00
udp_tunnel.c
udp.c udp: under rx pressure, try to condense skbs 2016-12-08 13:25:07 -05:00
udplite.c udplite: call proper backlog handlers 2016-11-24 15:32:14 -05:00
xfrm4_input.c
xfrm4_mode_beet.c
xfrm4_mode_transport.c
xfrm4_mode_tunnel.c
xfrm4_output.c
xfrm4_policy.c
xfrm4_protocol.c
xfrm4_state.c
xfrm4_tunnel.c