Hello!
I have a pretty standard XVLAN setup. In that setup server "A" with IP 10.0.0.100/24 can successfully reach server "B" with IP 10.0.0.200/24. Iperf3 performance if very good - line rate.
My goal is to have additional IPs on those servers, server "A" assigned IP 203.0.113.1/32 and server "B" assigned IP 203.0.113.2/32
[ srvA 203.0.113.1/32-lo + 10.0.0.100/24-eth0 ] <-> |vxlan-anycast gw 10.0.0.254/24| <-> [srvB 10.0.0.200/24-eth0 + 203.0.113.2/32-lo]
Those /32 IP are BGP propagated
Server's have only `default` route via anycast gateway 10.0.0.254
Output from the Nexus C9336C-FX2 switch. (Both 10.0.0.100 and 10.0.0.200 are on the same switch)
# show bgp vrf myvrf999 ipv4 unicast neighbor 10.0.0.100 routes
*>e203.0.113.1/32 10.0.0.100 0 65000 i
# show bgp vrf myvrf999 ipv4 unicast neighbor 10.0.0.200 routes
*>e203.0.113.2/32 10.0.0.200 0 65000 i
There is a reach-ability between those IPs:
root@serverA:~# tracepath -n 203.0.113.2
1?: [LOCALHOST] pmtu 1500
1: 10.0.0.254 0.682ms
1: 10.0.0.254 0.558ms
2: 203.0.113.2 0.409ms reached
Resume: pmtu 1500 hops 2 back 1
However iperf3 performance is very poor for those IPs
serverA:~# iperf3 -c 203.0.113.2 -p 5001 --bind 203.0.113.1
Connecting to host 203.0.113.2, port 5001
[ 5] local 203.0.113.1 port 38739 connected to 203.0.113.2 port 5001
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 294 KBytes 2.41 Mbits/sec 5 4.24 KBytes
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 13 11.3 KBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 1 15.6 KBytes
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 20 9.90 KBytes
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 4 7.07 KBytes
^C[ 5] 5.00-5.39 sec 0.00 Bytes 0.00 bits/sec 0 7.07 KBytes
Same test with the 10.0.0.x/24 IPs shows great throughput
root@serverA:~# iperf3 -c 10.0.0.200 -p 5001 --bind 10.0.0.100
Connecting to host 10.0.0.200, port 5001
[ 5] local 10.0.0.100 port 42939 connected to 10.0.0.200 port 5001
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.03 GBytes 26.0 Gbits/sec 0 1.84 MBytes
[ 5] 1.00-2.00 sec 2.58 GBytes 22.2 Gbits/sec 0 1.84 MBytes
[ 5] 2.00-3.00 sec 2.33 GBytes 20.0 Gbits/sec 0 3.02 MBytes
^C[ 5] 3.00-3.69 sec 1.57 GBytes 19.5 Gbits/sec 0 3.02 MBytes
What could be the issue there?
