cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
259
Views
5
Helpful
3
Replies
Highlighted
Beginner

EVPN+VxLAN very strange silent host issue

I have very basic EVPN+VxLAN design where i used OSPF/Multicast/BGP/vPC technologies to build small size datacenter. L2VNI working great without any issue. I have ISP connectivity via Border-Leaf and issue is when i spin up VM using public IP then its not pinging from outside world until unless i logged into vm and ping 8.8.8.8 (I believe it generate BGP l3vni route on border-leaf and then it learn about that vm and everything start pinging from inside and outside)

 

Question: why it's only issue with Public IPs which using border-leaf. I haven't seen same issue with my internal L2VNI network. I think when someone from outside trying to ping my public IP vm then border-leaf should generate ARP broadcast to find mac address of VM (am i missing something here?)

 

In my following config you can see i didn't create anycast-gateway on border-leaf also didn't create VNI 10100, is that correct or i should create anycast-gateway and L2VNI on border-leaf?

 

Screen Shot 2021-01-05 at 9.04.35 AM.png

 

Question: If border-leaf doesn't have any VTEP then how does it discover silent host located on other leaf using BUM traffic? 

 

My border-leaf config:

 

nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature bfd
feature nv overlay

ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 1,555
vlan 555
  name L3VNI-For-IRB
  vn-segment 10555

vrf context RED
  description ** VRF-RED **
  vni 10555
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface Vlan555
  description ** L3VNI-For-IRB **
  no shutdown
  vrf member RED
  ip forward
  ipv6 address use-link-local-only

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1
  member vni 10555 associate-vrf

interface loopback1
  description ** VTEP/Overlay **
  ip address 10.255.255.1/32
  ip ospf authentication-key 3 fa3ab8e90610229c
  ip router ospf UNDERLAY-NET area 0.0.0.0
  ip pim sparse-mode

router ospf UNDERLAY-NET
  log-adjacency-changes
  area 0.0.0.0 authentication
router bgp 65001
  router-id 10.255.1.1
  log-neighbor-changes
  template peer VXLAN_SPINE
    remote-as 65001
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.255.0.1
    inherit peer VXLAN_SPINE
    description ** iBGP Peer to Spine-1 **
    no shutdown
  neighbor 10.255.0.2
    inherit peer VXLAN_SPINE
    description ** iBGP Peer to Spine-2 **
    no shutdown
  vrf RED
    log-neighbor-changes
    address-family ipv4 unicast
      aggregate-address 60.25.124.0/23 summary-only
    address-family ipv6 unicast
    neighbor 2001:c05:3002:3::1
      bfd
      remote-as 21855
      local-as 293218
      description ** IPv6 ISP eBGP peer to Border5 **
      address-family ipv6 unicast
        send-community
    neighbor 60.25.120.201
      bfd
      remote-as 21855
      local-as 293218
      description ** ISP eBGP peer to Border5 **
      address-family ipv4 unicast
        send-community
        route-map LOCAL_PREF_150_IN in

 

My Leaf-1 config (I have vPC pair so i am just posting leaf-1 config)

 

nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature nv overlay

fabric forwarding anycast-gateway-mac 0000.dead.beef
ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 1,60-62,64,100,444,555
vlan 60 name ostack_1_br-storage vn-segment 10060 vlan 61 name ostack_1_ceph_replication vn-segment 10061 vlan 62 name ostack_1_br-lbaas vn-segment 10062 vlan 64 name inside vn-segment 10064 vlan 100 name ostack_1_public_1 vn-segment 10100 vlan 444 name BACKUP_VLAN_ROUTING_VPC vlan 555 name L3VNI-For-IRB vn-segment 10555
route-map DIRECT-PERMIT-ALL permit 10
description ** Route-Map for BGP to redist route **
vrf context RED vni 10555 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn vpc domain 1 peer-switch role priority 10 peer-keepalive destination 172.30.0.32 source 172.30.0.31 delay restore 90 peer-gateway delay restore interface-vlan 30 ipv6 nd synchronize ip arp synchronize interface Vlan100 description ** Anycast Gateway For Public ** no shutdown mtu 9216 vrf member RED ip address 60.25.124.1/23 ipv6 address 2001:c05:3011::1/64 ipv6 nd prefix default no-advertise ipv6 nd ra route suppress no ipv6 redirects fabric forwarding mode anycast-gateway interface Vlan444 description ** Underlay Backup over vPC Peer-Link ** no shutdown ip address 192.168.1.1/30 ip ospf authentication-key 3 fa3ab8e90610229c ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 ip pim sparse-mode interface Vlan555 description ** L3VNI-For-IRB ** no shutdown mtu 9216 vrf member RED ip forward ipv6 address use-link-local-only interface nve1 no shutdown description ** VTEP/NVE Interface ** host-reachability protocol bgp source-interface loopback1 member vni 10060 mcast-group 239.1.1.1 member vni 10061 mcast-group 239.1.1.1 member vni 10062 mcast-group 239.1.1.1 member vni 10064 mcast-group 239.1.1.1 member vni 10100 mcast-group 239.1.1.1 member vni 10555 associate-vrf router ospf UNDERLAY-NET router-id 10.255.1.11 log-adjacency-changes area 0.0.0.0 authentication router bgp 65001 router-id 10.255.1.11 log-neighbor-changes template peer VXLAN_SPINE remote-as 65001 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community send-community extended neighbor 10.255.0.1 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-1 ** neighbor 10.255.0.2 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-2 ** vrf RED log-neighbor-changes address-family ipv4 unicast redistribute direct route-map DIRECT-PERMIT-ALL address-family ipv6 unicast redistribute direct route-map DIRECT-PERMIT-ALL evpn vni 10060 l2 rd auto route-target import auto route-target export auto vni 10061 l2 rd auto route-target import auto route-target export auto vni 10062 l2 rd auto route-target import auto route-target export auto vni 10064 l2 rd auto route-target import auto route-target export auto vni 10100 l2 rd auto route-target import auto route-target export auto

 

3 REPLIES 3
Highlighted
Beginner

First of all, of course it has a VTEP. It just doesn't have the L2 VNI which is fine. The way it works is that the L3VNI only devices will have your /23 subnet which you redistribute into the VRF. Since you didn't include your route-maps I can't verify that.

As long as the border leaf has the /23 or whatever prefix redistributed from the VTEPs which have the L2 VNI, it will send the traffic to one of those leafs and that leaf will get it and do a glean (send the ARP request for silent host).

So check to see if the border leaf has the aggregate (/23 in your case) subnet in the routing table correctly.

And you need the 'advertise l2vpn evpn' if you are running < 9.2.1 as another poster replied below.  The command is removed in 9.2.1 and later.  I'd suggest using 9.3.x now

 

 

Highlighted

Thank you for reply,

 

I am running nxos.9.3.4 (pretty much latest) 

 

This is what i have in route-map of Leaf-1  (in short it's just empty), I am not advertising /23 subnet because i am running anycast-gateway and each host should advertise individual to find their location on each leaf in datacenter. 

route-map DIRECT-PERMIT-ALL permit 10
  description ** Route-Map for BGP to redist route **

Because BGP EVPN advertise individual host route in /32  (here is my leaf-1 routing advertisement)

 

Leaf-1# show bgp vpnv4 unicast

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 10.255.1.11:3    (VRF RED)
*>i0.0.0.0/0          10.255.255.1                      150          0 293218
21855 i
* i60.25.124.0/23     10.255.255.2                      100          0 i
* i                   10.255.255.1                      100          0 i
*>r                   0.0.0.0                  0        100      32768 ?
* i                   10.255.255.20            0        100          0 ?
*>i60.25.125.200/32   10.255.255.20                     100          0 i

As you can see in my border-leaf configuration i do have L3VNI but that is Inter-VLAN-IRB, I don't have Vlan100 L3VNI on border-leaf. (Do you think i need Vlan100 L3VNI on border-leaf to handle BUM/ARP traffic?) 

 

 

Highlighted
Rising star

....

Content for Community-Ad