Nexus doesn't forward traffic from server via vxlan

[ievhen.ivanysko1]

Hello!

I have problem that server serv1 can't reach serv2 , but serv2 can send data to serv1 (verified by starting tcpdump on both servers).

Both servers have routes for remote networks.

serv1# ip r | grep 240
172.18.240.0/24 via 172.18.224.254 dev vlan224

serv2#ip r | grep 224
172.18.224.0/24 via 172.18.240.254 dev vlan240

Some debug from nexus said that everything ok

traceroute nve ip 172.18.240.1 vrf openstack source 172.18.224.254 verbose

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'D' - Destination Unreachable, 'X' - unknown return code,
'm' - malformed request(parameter problem),
'c' - Corrupted Data/Test, '#' - Duplicate response,
'v' - Other - Use verbose to see the result

Traceroute Request to peer ip 172.19.1.206 source ip 172.19.1.201
Sender handle: 52
  1 !Reply from 172.19.0.14,time = 2 ms
  2 !Reply from 172.19.1.206,time = 1 ms
  3 !Reply from 172.18.240.1,time = 2 ms

If i specify as source ip address of server i've got following result when switch sends icmp packets to serv2 and serv1 successfully recieve reply. It's ok and confirm that connection is present.

traceroute nve ip 172.18.240.1 vrf openstack source 172.18.224.14 verbose

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'D' - Destination Unreachable, 'X' - unknown return code,
'm' - malformed request(parameter problem),
'c' - Corrupted Data/Test, '#' - Duplicate response,
'v' - Other - Use verbose to see the result

Traceroute Request to peer ip 172.19.1.206 source ip 172.19.1.201
Sender handle: 58
  1 !Reply from 172.19.0.14,time = 7 ms
  2 !Reply from 172.19.1.206,time = 1 ms
  3 .
  4 .
  5 .


tcpdump on serv2
03:53:16.822195 IP 172.18.224.14 > 172.18.240.1: ICMP echo request, id 3, seq 40963, length 16
03:53:16.822220 IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40963, length 16
03:53:18.824427 IP 172.18.224.14 > 172.18.240.1: ICMP echo request, id 3, seq 40964, length 16
03:53:18.824449 IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40964, length 16
03:53:20.826835 IP 172.18.224.14 > 172.18.240.1: ICMP echo request, id 3, seq 40965, length 16
03:53:20.826859 IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40965, length 16

tcpdump on serv1
03:53:16.823011 ethertype IPv4, IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40963, length 16
03:53:16.823011 ethertype IPv4, IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40963, length 16
03:53:16.823011 IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40963, length 16
03:53:18.824992 ethertype IPv4, IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40964, length 16
03:53:18.824992 ethertype IPv4, IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40964, length 16
03:53:18.824992 IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40964, length 16
03:53:20.827602 ethertype IPv4, IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40965, length 16
03:53:20.827602 ethertype IPv4, IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40965, length 16
03:53:20.827602 IP 172.18.240.1 > 172.18.224.14: ICMP echo reply, id 3, seq 40965, length 16

if i start ping/traceroute/mtr from serv1 - nothing happens. for example mtr stops at first hop. Serv2 didn't recieve any packet.

 Host                                                                                    Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 172.18.224.252                                                                        0.0%    11    0.4   0.4   0.3   0.6   0.1
 2. (waiting for reply)

So it looks like nexus doesn't forward any packets from serv1 via vxlan.

Little bit configuration on vtep1 (vtep2 has similar config)

vlan 299
  vn-segment 12299

interface Vlan224
  no shutdown
  mtu 9216
  vrf member openstack
  no ip redirects
  ip address 172.18.224.252/24
  no ipv6 redirects
  ip ospf passive-interface
  ip router ospf underlay area 0.0.0.0
  vrrp 224
    priority 200
    address 172.18.224.254
    no shutdown

vrf context openstack
  vni 12299
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface Vlan299
  no shutdown
  vrf member openstack
  no ip redirects
  ip forward
  no ipv6 redirects

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  global suppress-arp
  global ingress-replication protocol bgp
  member vni 12230
  member vni 12231
  member vni 12282
  member vni 12299 associate-vrf

interface Ethernet1/42
  description serv1
  lacp rate fast
  switchport
  switchport mode trunk
  switchport trunk native vlan 350
  switchport trunk allowed vlan 224-225
  spanning-tree port type edge trunk
  mtu 9216
  channel-group 42 mode active
  no shutdown

About devices:

1. VTEP1-VTEP2 - Cisco Nexus 9k (NXOS: version 10.1(2)), VTEP3 - Third-party vendor. 

I've following topology 

I don't thinks that there is problem with vxlan, because serv2 can successfully ping gateway of serv1 via vxlan. 

 Host                                                                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 172.18.240.253                                                                                 0.0%    16    0.3   0.3   0.3   0.4   0.0
 2. 172.18.224.254                                                                                 0.0%    16    1.3   1.3   1.1   1.7   0.2

Could someone help me identify problem? I've no idea what i can do(

[marce1000]

1. Verify that the VXLAN interface is up and operational by running the "show interface" command.

2. Check that the VXLAN is properly configured and that the VNI is correct.

3. Make sure that the VXLAN is associated with the correct VLAN and that the VLAN is active.

4. Check that the VXLAN is configured with the correct IP address and netmask.

5. Verify that the VXLAN is configured with the correct MTU size.

6. Make sure that there are no ACLs or other policies that are blocking traffic to or from the VXLAN.

7. Check the device logs for any error messages related to the VXLAN.

8. If you are unable to resolve the issue, you may want to consider gathering more information about the problem by capturing packets or enabling debugging on the device.

 M.

[Georg Pauwen]

Hello,

try and get the full configuration of VTEP3 as well, without seeing that, it is going to be difficult to find out what is going on. Also, post the full running configs of VTEP1 and VTEP2 (sh run).