I have very basic EVPN+VxLAN design where i used OSPF/Multicast/BGP/vPC technologies to build small size datacenter. L2VNI working great without any issue. I have ISP connectivity via Border-Leaf and issue is when i spin up VM using public IP then its not pinging from outside world until unless i logged into vm and ping 8.8.8.8 (I believe it generate BGP l3vni route on border-leaf and then it learn about that vm and everything start pinging from inside and outside)
Question: why it's only issue with Public IPs which using border-leaf. I haven't seen same issue with my internal L2VNI network. I think when someone from outside trying to ping my public IP vm then border-leaf should generate ARP broadcast to find mac address of VM (am i missing something here?)
In my following config you can see i didn't create anycast-gateway on border-leaf also didn't create VNI 10100, is that correct or i should create anycast-gateway and L2VNI on border-leaf?
Question: If border-leaf doesn't have any VTEP then how does it discover silent host located on other leaf using BUM traffic?
My border-leaf config:
nv overlay evpn
feature ospf
feature bgp
feature pim
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature bfd
feature nv overlay
ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 1,555
vlan 555
name L3VNI-For-IRB
vn-segment 10555
vrf context RED
description ** VRF-RED **
vni 10555
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
interface Vlan555
description ** L3VNI-For-IRB **
no shutdown
vrf member RED
ip forward
ipv6 address use-link-local-only
interface nve1
no shutdown
host-reachability protocol bgp
source-interface loopback1
member vni 10555 associate-vrf
interface loopback1
description ** VTEP/Overlay **
ip address 10.255.255.1/32
ip ospf authentication-key 3 fa3ab8e90610229c
ip router ospf UNDERLAY-NET area 0.0.0.0
ip pim sparse-mode
router ospf UNDERLAY-NET
log-adjacency-changes
area 0.0.0.0 authentication
router bgp 65001
router-id 10.255.1.1
log-neighbor-changes
template peer VXLAN_SPINE
remote-as 65001
update-source loopback0
address-family ipv4 unicast
address-family l2vpn evpn
send-community
send-community extended
neighbor 10.255.0.1
inherit peer VXLAN_SPINE
description ** iBGP Peer to Spine-1 **
no shutdown
neighbor 10.255.0.2
inherit peer VXLAN_SPINE
description ** iBGP Peer to Spine-2 **
no shutdown
vrf RED
log-neighbor-changes
address-family ipv4 unicast
aggregate-address 60.25.124.0/23 summary-only
address-family ipv6 unicast
neighbor 2001:c05:3002:3::1
bfd
remote-as 21855
local-as 293218
description ** IPv6 ISP eBGP peer to Border5 **
address-family ipv6 unicast
send-community
neighbor 60.25.120.201
bfd
remote-as 21855
local-as 293218
description ** ISP eBGP peer to Border5 **
address-family ipv4 unicast
send-community
route-map LOCAL_PREF_150_IN in
My Leaf-1 config (I have vPC pair so i am just posting leaf-1 config)
nv overlay evpn feature ospf feature bgp feature pim feature fabric forwarding feature interface-vlan feature vn-segment-vlan-based feature lacp feature vpc feature nv overlay
fabric forwarding anycast-gateway-mac 0000.dead.beef
ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 1,60-62,64,100,444,555
vlan 60 name ostack_1_br-storage vn-segment 10060 vlan 61 name ostack_1_ceph_replication vn-segment 10061 vlan 62 name ostack_1_br-lbaas vn-segment 10062 vlan 64 name inside vn-segment 10064 vlan 100 name ostack_1_public_1 vn-segment 10100 vlan 444 name BACKUP_VLAN_ROUTING_VPC vlan 555 name L3VNI-For-IRB vn-segment 10555
route-map DIRECT-PERMIT-ALL permit 10
description ** Route-Map for BGP to redist route **
vrf context RED vni 10555 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn vpc domain 1 peer-switch role priority 10 peer-keepalive destination 172.30.0.32 source 172.30.0.31 delay restore 90 peer-gateway delay restore interface-vlan 30 ipv6 nd synchronize ip arp synchronize interface Vlan100 description ** Anycast Gateway For Public ** no shutdown mtu 9216 vrf member RED ip address 60.25.124.1/23 ipv6 address 2001:c05:3011::1/64 ipv6 nd prefix default no-advertise ipv6 nd ra route suppress no ipv6 redirects fabric forwarding mode anycast-gateway interface Vlan444 description ** Underlay Backup over vPC Peer-Link ** no shutdown ip address 192.168.1.1/30 ip ospf authentication-key 3 fa3ab8e90610229c ip ospf network point-to-point ip router ospf UNDERLAY-NET area 0.0.0.0 ip pim sparse-mode interface Vlan555 description ** L3VNI-For-IRB ** no shutdown mtu 9216 vrf member RED ip forward ipv6 address use-link-local-only interface nve1 no shutdown description ** VTEP/NVE Interface ** host-reachability protocol bgp source-interface loopback1 member vni 10060 mcast-group 239.1.1.1 member vni 10061 mcast-group 239.1.1.1 member vni 10062 mcast-group 239.1.1.1 member vni 10064 mcast-group 239.1.1.1 member vni 10100 mcast-group 239.1.1.1 member vni 10555 associate-vrf router ospf UNDERLAY-NET router-id 10.255.1.11 log-adjacency-changes area 0.0.0.0 authentication router bgp 65001 router-id 10.255.1.11 log-neighbor-changes template peer VXLAN_SPINE remote-as 65001 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community send-community extended neighbor 10.255.0.1 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-1 ** neighbor 10.255.0.2 inherit peer VXLAN_SPINE description ** iBGP Peer to Spine-2 ** vrf RED log-neighbor-changes address-family ipv4 unicast redistribute direct route-map DIRECT-PERMIT-ALL address-family ipv6 unicast redistribute direct route-map DIRECT-PERMIT-ALL evpn vni 10060 l2 rd auto route-target import auto route-target export auto vni 10061 l2 rd auto route-target import auto route-target export auto vni 10062 l2 rd auto route-target import auto route-target export auto vni 10064 l2 rd auto route-target import auto route-target export auto vni 10100 l2 rd auto route-target import auto route-target export auto
