[arch-general] Performance with Linux Bridge
Hello to all I'm using Arch Linux in a SuperMicro A1SAi-2750F motherboard and I'm having performance issues while bridging 3 Ethernet Gigabit network interfaces. The 4th network interface of that motherboard, is used to connect to the ISP. This setup is my Router/server and I use the 3 bridged interfaces to connect my home network. But now, I'm noticing severe performance issues. I conducted a small iperf3 test. The scenario was: PC A (100Mbps) <---> Server Ether 1(1Gbps) <---> Linux Bridge (br0) <---> Server Ether 2 (1Gbps) <---> PC B (1Gbps) The result: Connecting to host 192.168.123.136, port 5201 [ 4] local 192.168.123.4 port 55462 connected to 192.168.123.136 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 1.47 MBytes 12.3 Mbits/sec 56 9.90 KBytes [ 4] 1.00-2.00 sec 1.16 MBytes 9.70 Mbits/sec 50 8.48 KBytes [ 4] 2.00-3.00 sec 1.10 MBytes 9.25 Mbits/sec 30 12.7 KBytes [ 4] 3.00-4.00 sec 1.14 MBytes 9.52 Mbits/sec 40 5.66 KBytes [ 4] 4.00-5.00 sec 1.04 MBytes 8.76 Mbits/sec 37 8.48 KBytes [ 4] 5.00-6.00 sec 1.19 MBytes 9.96 Mbits/sec 40 12.7 KBytes [ 4] 6.00-7.00 sec 1.14 MBytes 9.55 Mbits/sec 46 7.07 KBytes [ 4] 7.00-8.00 sec 1.02 MBytes 8.55 Mbits/sec 39 14.1 KBytes [ 4] 8.00-9.00 sec 1.14 MBytes 9.52 Mbits/sec 47 12.7 KBytes [ 4] 9.00-10.00 sec 1.04 MBytes 8.72 Mbits/sec 35 11.3 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 11.4 MBytes 9.58 Mbits/sec 420 sender [ 4] 0.00-10.00 sec 11.4 MBytes 9.57 Mbits/sec receiver Then i conducted another iperf3 test with a different scenario: PC A (100Mbps) <---> Server Ether 1 (1Gbps) <---> Linux Bridge (br0) The result: Connecting to host 192.168.123.1, port 5201 [ 4] local 192.168.123.4 port 58982 connected to 192.168.123.1 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 11.3 MBytes 94.5 Mbits/sec 0 24.0 KBytes [ 4] 1.00-2.00 sec 11.2 MBytes 94.1 Mbits/sec 0 24.0 KBytes [ 4] 2.00-3.00 sec 11.2 MBytes 94.1 Mbits/sec 0 24.0 KBytes [ 4] 3.00-4.00 sec 11.2 MBytes 94.1 Mbits/sec 0 24.0 KBytes [ 4] 4.00-5.00 sec 11.2 MBytes 94.2 Mbits/sec 0 24.0 KBytes [ 4] 5.00-6.00 sec 11.2 MBytes 94.3 Mbits/sec 0 24.0 KBytes [ 4] 6.00-7.00 sec 11.2 MBytes 94.1 Mbits/sec 0 24.0 KBytes [ 4] 7.00-8.00 sec 11.2 MBytes 94.1 Mbits/sec 0 24.0 KBytes [ 4] 8.00-9.00 sec 11.2 MBytes 94.1 Mbits/sec 0 24.0 KBytes [ 4] 9.00-10.00 sec 11.2 MBytes 94.2 Mbits/sec 0 24.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 112 MBytes 94.2 Mbits/sec 0 sender [ 4] 0.00-10.00 sec 112 MBytes 94.1 Mbits/sec receiver iperf Done. Does anyone know if this low performance behaviour by a Linux bridge is "normal" or, if it is a configuration problem? I disabled iptables. Despite gaining a little improvement, I still have severe performance issues. Can anyone provide some light in this case? Thank you! Carlos Miguel Ferreira
On 27.12.2015 19:23, Carlos Ferreira wrote:
I'm using Arch Linux in a SuperMicro A1SAi-2750F motherboard and I'm having performance issues while bridging 3 Ethernet Gigabit network interfaces. The 4th network interface of that motherboard, is used to connect to the ISP. [..] I conducted a small iperf3 test. The scenario was: PC A (100Mbps) <---> Server Ether 1(1Gbps) <---> Linux Bridge (br0) <---> Server Ether 2 (1Gbps) <---> PC B (1Gbps)
Are these values from a tool like ethtool or are they just what you think they should be? I've sometimes had bad cables/connectors cause speeds to be limited to 100Mbit instead of gbit. As for software problems you'd need to provide/look at some more details like cpu load, interrupts, top, htop, dstat -lpma, mpstat -I ALL, dmesg. That cpu should be able to easily handle gbit speed though and a bridge in linux should be able to far exceed your network speeds. Hope that helps, Florian
Well, this is embarrassing... It seems that in my haste to blame the Linux Bridge, I discovered that the real problem was an Ethernet Cable, more specifically, the one connecting the Linux Box to the PC B. Recentl, I changed the location of the Router/server and during that change, I must have accidentally damaged on cable. Before that, I repeated the iperf3 test several times and from time to time, the performance fluctuated between 5 and 40 Mbps. Because of that, I decided to change the Ethernet Cables just to check it. After installing the new cable, the performance is now what was expected... I'm sorry to have wasted your time... I hope this helps future generations not to be so hasty... On 27 December 2015 at 20:05, Florian Pritz <bluewind@xinu.at> wrote:
On 27.12.2015 19:23, Carlos Ferreira wrote:
I'm using Arch Linux in a SuperMicro A1SAi-2750F motherboard and I'm having performance issues while bridging 3 Ethernet Gigabit network interfaces. The 4th network interface of that motherboard, is used to connect to the ISP. [..] I conducted a small iperf3 test. The scenario was: PC A (100Mbps) <---> Server Ether 1(1Gbps) <---> Linux Bridge (br0) <---> Server Ether 2 (1Gbps) <---> PC B (1Gbps)
Are these values from a tool like ethtool or are they just what you think they should be? I've sometimes had bad cables/connectors cause speeds to be limited to 100Mbit instead of gbit.
As for software problems you'd need to provide/look at some more details like cpu load, interrupts, top, htop, dstat -lpma, mpstat -I ALL, dmesg. That cpu should be able to easily handle gbit speed though and a bridge in linux should be able to far exceed your network speeds.
Hope that helps, Florian
-- Carlos Miguel Ferreira Researcher at Telecommunications Institute Aveiro - Portugal Work E-mail - cmf@av.it.pt Skype & GTalk -> carlosmf.pt@gmail.com LinkedIn -> http://www.linkedin.com/in/carlosmferreira
participants (2)
-
Carlos Ferreira
-
Florian Pritz