From 94f49a163aa570ea272bf3bbd7734187098d88b7 Mon Sep 17 00:00:00 2001 From: Etienne Dechamps Date: Sun, 15 Mar 2015 17:50:53 +0000 Subject: [PATCH] Set the default for UDPRcvBuf and UDPSndBuf to 1M. It may not be obvious, but due to the way tinc operates (single-threaded control loop with no intermediate packet buffer), UDP send and receive buffers can have a massive impact on performance. It is therefore of paramount importance that the buffers be large enough to prevent packet drops that could occur while tinc is processing a packet. Leaving that value to the OS default could be reasonable if we weren't relying on it so much. Instead, this makes performance somewhat unpredictable. In practice, the worst case scenario occurs on Windows, where Microsoft had the brillant idea of making the buffers 8K in size by default, no matter what the link speed is. Considering that 8K flies past in a matter of microseconds on >1G links, this is extremely inappropriate. On these systems, changing the buffer size to 1M results in *obscene* raw throughput improvements; I have observed a 10X jump from 40 Mbit/s to 400 Mbit/s on my system. In this commit, we stop trusting the OS to get this right and we use a fixed 1M value instead, which should be enough for <=1G links. --- doc/tinc.conf.5.in | 10 ++++++---- doc/tinc.texi | 10 ++++++---- src/net_setup.c | 4 ++-- src/net_socket.c | 4 ++-- 4 files changed, 16 insertions(+), 12 deletions(-) diff --git a/doc/tinc.conf.5.in b/doc/tinc.conf.5.in index bc481ead..2ba10124 100644 --- a/doc/tinc.conf.5.in +++ b/doc/tinc.conf.5.in @@ -500,12 +500,14 @@ The minimum amount of time between sending UDP ping datagrams to try to establis .It Va UDPDiscoveryTimeout Li = Ar seconds Pq 30 If tinc doesn't receive any UDP ping replies over the specified interval, it will assume UDP communication is broken and will fall back to TCP. -.It Va UDPRcvBuf Li = Ar bytes Pq OS default +.It Va UDPRcvBuf Li = Ar bytes Pq 1048576 Sets the socket receive buffer size for the UDP socket, in bytes. -If unset, the default buffer size will be used by the operating system. -.It Va UDPSndBuf Li = Ar bytes Pq OS default +If set to zero, the default buffer size will be used by the operating system. +Note: this setting can have a significant impact on performance, especially raw throughput. +.It Va UDPSndBuf Li = Ar bytes Pq 1048576 Sets the socket send buffer size for the UDP socket, in bytes. -If unset, the default buffer size will be used by the operating system. +If set to zero, the default buffer size will be used by the operating system. +Note: this setting can have a significant impact on performance, especially raw throughput. .El .Sh HOST CONFIGURATION FILES The host configuration files contain all information needed diff --git a/doc/tinc.texi b/doc/tinc.texi index 581da6ea..3694d6da 100644 --- a/doc/tinc.texi +++ b/doc/tinc.texi @@ -1254,14 +1254,16 @@ If tinc doesn't receive any UDP ping replies over the specified interval, it will assume UDP communication is broken and will fall back to TCP. @cindex UDPRcvBuf -@item UDPRcvBuf = (OS default) +@item UDPRcvBuf = (1048576) Sets the socket receive buffer size for the UDP socket, in bytes. -If unset, the default buffer size will be used by the operating system. +If set to zero, the default buffer size will be used by the operating system. +Note: this setting can have a significant impact on performance, especially raw throughput. @cindex UDPSndBuf -@item UDPSndBuf = Pq OS default +@item UDPSndBuf = (1048576) Sets the socket send buffer size for the UDP socket, in bytes. -If unset, the default buffer size will be used by the operating system. +If set to zero, the default buffer size will be used by the operating system. +Note: this setting can have a significant impact on performance, especially raw throughput. @end table diff --git a/src/net_setup.c b/src/net_setup.c index d1d5c04b..fee4159f 100644 --- a/src/net_setup.c +++ b/src/net_setup.c @@ -854,14 +854,14 @@ static bool setup_myself(void) { } if(get_config_int(lookup_config(config_tree, "UDPRcvBuf"), &udp_rcvbuf)) { - if(udp_rcvbuf <= 0) { + if(udp_rcvbuf < 0) { logger(DEBUG_ALWAYS, LOG_ERR, "UDPRcvBuf cannot be negative!"); return false; } } if(get_config_int(lookup_config(config_tree, "UDPSndBuf"), &udp_sndbuf)) { - if(udp_sndbuf <= 0) { + if(udp_sndbuf < 0) { logger(DEBUG_ALWAYS, LOG_ERR, "UDPSndBuf cannot be negative!"); return false; } diff --git a/src/net_socket.c b/src/net_socket.c index 85bc4df5..3bf7d757 100644 --- a/src/net_socket.c +++ b/src/net_socket.c @@ -43,8 +43,8 @@ int addressfamily = AF_UNSPEC; int maxtimeout = 900; int seconds_till_retry = 5; -int udp_rcvbuf = 0; -int udp_sndbuf = 0; +int udp_rcvbuf = 1024 * 1024; +int udp_sndbuf = 1024 * 1024; int max_connection_burst = 100; listen_socket_t listen_socket[MAXSOCKETS]; -- 2.20.1