01-04-2021 12:27 PM - edited 01-05-2021 10:12 PM
I have very basic EVPN+VxLAN design where i used OSPF/Multicast/BGP/vPC technologies to build small size datacenter. L2VNI working great without any issue. I have ISP connectivity via Border-Leaf and issue is when i spin up VM using public IP then its not pinging from outside world until unless i logged into vm and ping 8.8.8.8 (I believe it generate BGP l3vni route on border-leaf and then it learn about that vm and everything start pinging from inside and outside)
Question: why it's only issue with Public IPs which using border-leaf. I haven't seen same issue with my internal L2VNI network. I think when someone from outside trying to ping my public IP vm then border-leaf should generate ARP broadcast to find mac address of VM (am i missing something here?)
In my following config you can see i didn't create anycast-gateway on border-leaf also didn't create VNI 10100, is that correct or i should create anycast-gateway and L2VNI on border-leaf?
Question: If border-leaf doesn't have any VTEP then how does it discover silent host located on other leaf using BUM traffic?
My border-leaf config:
nv overlay evpn feature ospf feature bgp feature pim feature interface-vlan feature vn-segment-vlan-based feature lacp feature bfd feature nv overlay ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8 ip pim ssm range 232.0.0.0/8 vlan 1,555 vlan 555 name L3VNI-For-IRB vn-segment 10555 vrf context RED description ** VRF-RED ** vni 10555 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn interface Vlan555 description ** L3VNI-For-IRB ** no shutdown vrf member RED ip forward ipv6 address use-link-local-only interface nve1 no shutdown host-reachability protocol bgp source-interface loopback1 member vni 10555 associate-vrf interface loopback1 description ** VTEP/Overlay ** ip address 10.255.255.1/32 ip ospf authentication-key 3 fa3ab8e90610229c ip router ospf UNDERLAY-NET area 0.0.0.0 ip pim sparse-mode router ospf UNDERLAY-NET log-adjacency-changes area 0.0.0.0 authentication router bgp 65001 router-id 10.255.1.1 log-neighbor-changes template peer VXLAN_SPINE remote-as 65001 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community send-community extended neighbor 10.255.0.1 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-1 ** no shutdown neighbor 10.255.0.2 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-2 ** no shutdown vrf RED log-neighbor-changes address-family ipv4 unicast aggregate-address 60.25.124.0/23 summary-only address-family ipv6 unicast neighbor 2001:c05:3002:3::1 bfd remote-as 21855 local-as 293218 description ** IPv6 ISP eBGP peer to Border5 ** address-family ipv6 unicast send-community neighbor 60.25.120.201 bfd remote-as 21855 local-as 293218 description ** ISP eBGP peer to Border5 ** address-family ipv4 unicast send-community route-map LOCAL_PREF_150_IN in
My Leaf-1 config (I have vPC pair so i am just posting leaf-1 config)
nv overlay evpn feature ospf feature bgp feature pim feature fabric forwarding feature interface-vlan feature vn-segment-vlan-based feature lacp feature vpc feature nv overlay
fabric forwarding anycast-gateway-mac 0000.dead.beef
ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 1,60-62,64,100,444,555
vlan 60 name ostack_1_br-storage vn-segment 10060 vlan 61 name ostack_1_ceph_replication vn-segment 10061 vlan 62 name ostack_1_br-lbaas vn-segment 10062 vlan 64 name inside vn-segment 10064 vlan 100 name ostack_1_public_1 vn-segment 10100 vlan 444 name BACKUP_VLAN_ROUTING_VPC vlan 555 name L3VNI-For-IRB vn-segment 10555
route-map DIRECT-PERMIT-ALL permit 10
description ** Route-Map for BGP to redist route **
vrf context RED vni 10555 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn vpc domain 1 peer-switch role priority 10 peer-keepalive destination 172.30.0.32 source 172.30.0.31 delay restore 90 peer-gateway delay restore interface-vlan 30 ipv6 nd synchronize ip arp synchronize interface Vlan100 description ** Anycast Gateway For Public ** no shutdown mtu 9216 vrf member RED ip address 60.25.124.1/23 ipv6 address 2001:c05:3011::1/64 ipv6 nd prefix default no-advertise ipv6 nd ra route suppress no ipv6 redirects fabric forwarding mode anycast-gateway interface Vlan444 description ** Underlay Backup over vPC Peer-Link ** no shutdown ip address 192.168.1.1/30 ip ospf authentication-key 3 fa3ab8e90610229c ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 ip pim sparse-mode interface Vlan555 description ** L3VNI-For-IRB ** no shutdown mtu 9216 vrf member RED ip forward ipv6 address use-link-local-only interface nve1 no shutdown description ** VTEP/NVE Interface ** host-reachability protocol bgp source-interface loopback1 member vni 10060 mcast-group 239.1.1.1 member vni 10061 mcast-group 239.1.1.1 member vni 10062 mcast-group 239.1.1.1 member vni 10064 mcast-group 239.1.1.1 member vni 10100 mcast-group 239.1.1.1 member vni 10555 associate-vrf router ospf UNDERLAY-NET router-id 10.255.1.11 log-adjacency-changes area 0.0.0.0 authentication router bgp 65001 router-id 10.255.1.11 log-neighbor-changes template peer VXLAN_SPINE remote-as 65001 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community send-community extended neighbor 10.255.0.1 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-1 ** neighbor 10.255.0.2 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-2 ** vrf RED log-neighbor-changes address-family ipv4 unicast redistribute direct route-map DIRECT-PERMIT-ALL address-family ipv6 unicast redistribute direct route-map DIRECT-PERMIT-ALL evpn vni 10060 l2 rd auto route-target import auto route-target export auto vni 10061 l2 rd auto route-target import auto route-target export auto vni 10062 l2 rd auto route-target import auto route-target export auto vni 10064 l2 rd auto route-target import auto route-target export auto vni 10100 l2 rd auto route-target import auto route-target export auto
01-05-2021 08:34 PM - edited 01-05-2021 09:10 PM
First of all, of course it has a VTEP. It just doesn't have the L2 VNI which is fine. The way it works is that the L3VNI only devices will have your /23 subnet which you redistribute into the VRF. Since you didn't include your route-maps I can't verify that.
As long as the border leaf has the /23 or whatever prefix redistributed from the VTEPs which have the L2 VNI, it will send the traffic to one of those leafs and that leaf will get it and do a glean (send the ARP request for silent host).
So check to see if the border leaf has the aggregate (/23 in your case) subnet in the routing table correctly.
And you need the 'advertise l2vpn evpn' if you are running < 9.2.1 as another poster replied below. The command is removed in 9.2.1 and later. I'd suggest using 9.3.x now
01-05-2021 10:22 PM
Thank you for reply,
I am running nxos.9.3.4 (pretty much latest)
This is what i have in route-map of Leaf-1 (in short it's just empty), I am not advertising /23 subnet because i am running anycast-gateway and each host should advertise individual to find their location on each leaf in datacenter.
route-map DIRECT-PERMIT-ALL permit 10 description ** Route-Map for BGP to redist route **
Because BGP EVPN advertise individual host route in /32 (here is my leaf-1 routing advertisement)
Leaf-1# show bgp vpnv4 unicast Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 10.255.1.11:3 (VRF RED) *>i0.0.0.0/0 10.255.255.1 150 0 293218 21855 i * i60.25.124.0/23 10.255.255.2 100 0 i * i 10.255.255.1 100 0 i *>r 0.0.0.0 0 100 32768 ? * i 10.255.255.20 0 100 0 ? *>i60.25.125.200/32 10.255.255.20 100 0 i
As you can see in my border-leaf configuration i do have L3VNI but that is Inter-VLAN-IRB, I don't have Vlan100 L3VNI on border-leaf. (Do you think i need Vlan100 L3VNI on border-leaf to handle BUM/ARP traffic?)
01-05-2021 08:55 PM - edited 01-05-2021 09:18 PM
....
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide