We noticed a peculiarity on our Comcast EDI 50M link which uses a Ciena LE-311v for hand off. This device provided by Comcast is something like a decade old, so maybe that's the cause:
»
web.archive.org/web/2005 ··· ions.pdfStart high bandwidth and high RTT download (doesnt really matter where from as long as high RTT and has good bandwidth to Comcast):
Aim large UDP datagram 1Mbits iperf at target IP address from any high performance link (nothing on target side has to be listening, in fact 5001 UDP can even be blocked on target side):
iperf -c [TargetIP] -u -t 60 -i 1 -b 1M -l 26k
Here is what happens:
1. HTTP download from link is able to consistently saturate 50M connection
2. While HTTP download is still running, run the iperf command above from a high performance connection
3. Immediately once iperf starts, HTTP download mentioned above falls to 5-10M and stays there
4. Once iperf is turned off, HTTP download recovers to saturate 50M connection
What I think is happening is that the buffers on some device are filling up as they are too small. Datagrams smaller than 26k do not cause this issue. As the UDP streams fills up the buffers it induces packet loss on the circuit. Its not our router, I have replicated this individually through our pfSense (FreeBSD) firewall, a Cisco ISR, and a Lenovo T530 Laptop with Windows 8.1 hooked up to the Ciena.
High RTT TCP downloads are severely degraded, low RTT downloads are degraded a bit but its not really noticeable.
Does anyone have any ideas as to the root cause of this? Can anyone test this on their EDI circuit?