Solved: Inter-vlan routing EVPN

velo84 · ‎09-29-2020

I'm trying to setup EVPN with Anycast Gateway in my lab and having some issues routing between the L3 networks. I have attached a diagram, some configs and some outputs from each switch. My issue is I can't communicate between devices on different VLANs on different switches.

Initially I was trying to get a device on one switch to ping a device on another switch in a different VLAN:

PC6 on switch1 (VLAN 301 172.16.100.100) to ping PC7 on switch 2 (VLAN 302 172.16.200.100)

Switch1 does have a route for it:

nexus-sw1# show ip route 172.16.200.100 vrf Tenant-1
IP Route Table for VRF "Tenant-1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.200.100/32, ubest/mbest: 1/0
    *via 2.2.2.2%default, [200/0], 02:10:10, bgp-65535, internal, tag 65535 (evp
n) segid: 900001 tunnelid: 0x2020202 encap: VXLAN

The opposing switch1 has a route back as well:

nexus-sw2# show ip route 172.16.100.100 vrf Tenant-1
IP Route Table for VRF "Tenant-1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.100.100/32, ubest/mbest: 1/0
    *via 1.1.1.1%default, [200/0], 01:50:24, bgp-65535, internal, tag 65535 (evp
n) segid: 900001 tunnelid: 0x1010101 encap: VXLAN

If I create the Layer2 VNI and Anycast gateway for VLAN vlan 302 on Switch1 I am able to ping between the VLANs. (So: PC6 on switch1 (VLAN 301 172.16.100.100) to ping PC8 on switch1 (VLAN 302 172.16.200.250))

Diagram:

Configuration for Switch1:

nexus-sw1# sh run

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay


fabric forwarding anycast-gateway-mac 0000.2222.3333
vlan 1,101-102,301-302,900,1000-1001
vlan 101
  vn-segment 900001

vlan 301
  vn-segment 5501
vlan 302
  vn-segment 5502


vrf context Tenant-1
  vni 900001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

hardware access-list tcam region vpc-convergence 0
hardware access-list tcam region arp-ether 256




interface Vlan101
  desc L3-VNI
  no shutdown
  vrf member Tenant-1
  ip forward



interface Vlan301
  description app
  no shutdown
  vrf member Tenant-1
  ip address 172.16.100.1/24
  fabric forwarding mode anycast-gateway

interface Vlan302
  description db
  no shutdown
  vrf member Tenant-1
  ip address 172.16.200.1/24
  fabric forwarding mode anycast-gateway




interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 5501
    suppress-arp
    ingress-replication protocol bgp
  member vni 5502
    suppress-arp
    ingress-replication protocol bgp
  member vni 900001 associate-vrf


interface Ethernet1/1
  no switchport
  ip address 10.10.10.1/30
  ip router ospf 10 area 0.0.0.0
  no shutdown


interface Ethernet1/6
  switchport access vlan 302

interface Ethernet1/7
  switchport access vlan 301

interface loopback0
  ip address 1.1.1.1/32
  ip router ospf 10 area 0.0.0.0



router ospf 10
router bgp 65535
  router-id 1.1.1.1
  neighbor 2.2.2.2
    remote-as 65535
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
evpn
  vni 5501 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 5502 l2
    rd auto
    route-target import auto
    route-target export auto




Troubleshooting Commands:
-------------------------

nexus-sw2# show ip route 172.16.100.100 vrf Tenant-1
IP Route Table for VRF "Tenant-1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.100.100/32, ubest/mbest: 1/0
    *via 1.1.1.1%default, [200/0], 01:50:24, bgp-65535, internal, tag 65535 (evp
n) segid: 900001 tunnelid: 0x1010101 encap: VXLAN





nexus-sw1# show nve vni
Codes: CP - Control Plane        DP - Data Plane
       UC - Unconfigured         SA - Suppress ARP
       SU - Suppress Unknown Unicast

Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      5501     UnicastBGP        Up    CP   L2 [301]           SA
nve1      5502     UnicastBGP        Up    CP   L2 [302]           SA
nve1      900001   n/a               Up    CP   L3 [Tenant-1]

Configuration for Switch2:

nexus-sw2# shrun

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay


fabric forwarding anycast-gateway-mac 0000.2222.3333
vlan 1,101-102,302,900,1000-1001
vlan 101
  vn-segment 900001

vlan 302
  vn-segment 5502


vrf context Tenant-1
  vni 900001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

hardware access-list tcam region vpc-convergence 0
hardware access-list tcam region arp-ether 256




interface Vlan101
  no shutdown
  vrf member Tenant-1


interface Vlan302
  description db
  no shutdown
  vrf member Tenant-1
  ip address 172.16.200.1/24
  fabric forwarding mode anycast-gateway

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 5502
    suppress-arp
    ingress-replication protocol bgp
  member vni 900001 associate-vrf


interface Ethernet1/1
  no switchport
  ip address 10.10.10.2/30
  ip router ospf 10 area 0.0.0.0
  no shutdown



interface Ethernet1/7
  switchport access vlan 302
  no shutdown



interface loopback0
  ip address 2.2.2.2/32
  ip router ospf 10 area 0.0.0.0


router ospf 10
router bgp 65535
  router-id 2.2.2.2
  neighbor 1.1.1.1
    remote-as 65535
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
evpn
  vni 5502 l2
    rd auto
    route-target import auto
    route-target export auto




Troubleshooting Commands:
-------------------------

nexus-sw2# show ip route 172.16.100.100 vrf Tenant-1
IP Route Table for VRF "Tenant-1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.100.100/32, ubest/mbest: 1/0
    *via 1.1.1.1%default, [200/0], 01:50:24, bgp-65535, internal, tag 65535 (evp
n) segid: 900001 tunnelid: 0x1010101 encap: VXLAN



nexus-sw2# show nve vni
Codes: CP - Control Plane        DP - Data Plane
       UC - Unconfigured         SA - Suppress ARP
       SU - Suppress Unknown Unicast

Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      5502     UnicastBGP        Up    CP   L2 [302]           SA
nve1      900001   n/a               Up    CP   L3 [Tenant-1]

OSPF established:

nexus-sw1# show ip ospf neighbors
 OSPF Process ID 10 VRF default
 Total number of neighbors: 1
 Neighbor ID     Pri State            Up Time  Address         Interface
 2.2.2.2           1 FULL/BDR         04:25:56 10.10.10.2      Eth1/1

VLAN 301 VNI is not configured on switch2 but my understanding is you only need to configure the VNIs on the switches where you have hosts connected in that VNI/VLAN.

Any help appreciated.

Thanks

horia.gunica · ‎10-13-2020

Hello!

Configure this on your Switch 2 :

interface Vlan101
  desc L3-VNI
  no shutdown
  vrf member Tenant-1
  ip forward ----> this command is currently missing and is the cause of the problem (you have it on switch 2).

Without that command - leafs cannot do symmetric IRB .

View solution in original post

horia.gunica · ‎10-13-2020

Hello!

Configure this on your Switch 2 :

interface Vlan101
  desc L3-VNI
  no shutdown
  vrf member Tenant-1
  ip forward ----> this command is currently missing and is the cause of the problem (you have it on switch 2).

Without that command - leafs cannot do symmetric IRB .

velo84 · ‎10-15-2020

Hi Horia

Thanks very much, good spot (that was the issue) I did eventually figure it out. I forgot to come back here and post.

Thank you