XRv and CSRv Packet drops

[obattal]

Hello,

I have a test topology that includes mostly vm xrv9000 (powered on esxi as virtual machines). All of the routers can form IS-IS neighbor and able to ping every Ip on the lab network. Traffic generators (Trex/iperf3/scapy) are connected to the routers to simulate traffic. Traffic generators interfaces which are connected to the lab can ping each other and Ip addresses of the routers. No packet loss on ping, but the traffic created by the generators have nearly %90 loss.

To troubleshoot the routing issues, I have connected both traffic generator interfaces to only one router (directly connected interfaces/no need for extra routing protocol). Traffic drop rates are the same.

The tests that I have done can be summarized as below:

XRv9000 as vm esxi (7.3.1)

XRv9000 on EveNG (7.1.1)
Csr1000v as vm esxi (Virtual XE Software 17.03.02)

C8K on EveNG (Virtual XE Software 17.06.01a)

I have checked the packets dest/src, Mac/IP addresses and everything seems to be in order.

No crc, input or output drops on interface counters. No log corresponding the excessive drops.

Thank you in advance.

[Georg Pauwen]

Hello,

difficult to troubleshoot, as the issue might be with iperf and the way it interacts with the CPU and the NIC(s). The bug report linked below lists at least one bug.

https://iperf.fr/

[obattal]

Hello Georg,

Thank you for the fast reply. Same packet loss happens when I use Trex as traffic generator.

[Georg Pauwen]

Hello,

can you hook up Wireshark to capture some of the traffic ?

[obattal]

Hello Georg,

Sorry for the late response. Had a storage issue which impacted the lab.
I have managed to solve the packet loss with manually adjusting the window size. Now I can send and receive the same amount of data. Also checked with wireshark to be sure. The maximum bandwidth seem to be ~150Kbits/sec.

[obattal]

Update:
Devices running on servers with Westmere EX processor (Intel(R) Xeon(R) CPU E7- 4870) are losing packets without tcp windows size restrictions and max bandwidth seems to be around 150Kbits/sec.

Devices running on servers with Skylake processor (Intel(R) Xeon(R) Silver 4110) can reach bandwidth of 750Kbits/sec(tcp) and 1.17Mbits/sec(udp).

Is there any known processor limitations in virtual routing platforms?