Nexus 9K - Disabling HSRP with VXLAN

[cHrome08]

Hi,

I have successfully set up VXLAN configuration in GNS3 and its working fine.
Even with HSRP, its working fine.
However, this is not what I would like to do.
I have 2 DC, namely PROD and DR.
On each DC, there are 2 Nexus 9k switches with VPC enabled.
I would like the HSRP traffic to communicate between the nexus in each DC but not inter-DC.
Meaning the Prod-Nexus-Pri can communicate via hsrp to Prod-Nexus-Sec.
However, Prod-Nexus-Pri should not be able to communicate via hsrp to DR-Nexus-Pri,

This is because I have a cluster of ASA firewall (1 each in Prod and DR) that is currently working as an active/standby.

I have tried applying ACL outbound on interface A, B, C and D (based on the picture attached) but its not working properly as DR-NEXUS-SEC can still see that the active HSRP is PROD-NEXUS-PRI.

My configs are mostly based on the URL https://www.hospitableit.com/howto/dci-using-vxlan-with-mp-bgp-evpn-and-ingress-replication-on-a-nexus-9k/ with the exception of multicast config. I have replaced that configuration with the ingress-replication protocol bgp command.

ip access-list HSRP-DENY
statistics per-entry
40 deny udp any eq 1985 224.0.0.102/32 eq 1985
50 permit ip any any
!

interface A
no switchport
mtu 9216
ip access-group HSRP-DENY out
ip address 10.19.1.1/30
ip ospf dead-interval 2
ip ospf hello-interval 1
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf NEXUS_CORE area 0.0.0.0
no shutdown
!

vlan 10
name FW-INSIDE
vn-segment 10010
!
interface nve1
no shutdown
host-reachability protocol bgp
source-interface loopback1
source-interface hold-down-time 30
member vni 10010
ingress-replication protocol bgp
!

router bgp 100
router-id 10.19.1.21
address-family ipv4 unicast
address-family l2vpn evpn
neighbor 10.19.1.24
remote-as 100
update-source loopback0
address-family ipv4 unicast
address-family l2vpn evpn
send-community
send-community extended

!

evpn
vni 10010 l2
rd auto
route-target import auto
route-target export auto
!

[cHrome08]

To add on, I also did a packet capture in GNS3 and it seems like the HSRP communication is using the VTEP IPs, that could be the reason why the ACL is not working.
I have attached a screenshot of the pcap in this post.
Thank you

[Sergiu.Daniluk]

Hi @cHrome08 

Just to confirm, you have HSRP configured on the Nexus, right?

In this case, I would recommend you using Distributed Anycast Gateway functionality:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/vxlan/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_VXLAN_Configuration_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_VXLAN_Configuration_Guide_7x_chapter_0100.html#task_CDF4... 

It is more scalable, easier to use and you do not need to worry about filtering.

Stay safe,

Sergiu

[cHrome08]

Thank you Sergiu

I will certainly check this out.

[cHrome08]

Hi @Sergiu.Daniluk 

Yes looking at it, Distributed Anycast gateway is what I was looking for.
Along the way, I have to learn L3VNI too, which is interesting and I am now able to get inter-vni routing working fine.
But just a question though when I have 2 machines, 1 in vlan 10, another in vlan 20.
I am able to ping from vlan 20 to vlan 10 and vice versa.
Initially when I perform the ping, there will be a few packets of request time out before the ping works. (Seen by the attached picture, its the first try)
Each time I perform the ping, the PC send about 5 packets.

For the next few consecutive ping, I have got success for 5 packets.

When I leave the GNS3 on its own, for maybe 3 to 4 minutes, and try the ping again, i got the same results as the first set (seen by the attached picture, its the last set).

Not too sure if I make any sense, but was wondering is this behavior expected?
Thank you.

[Sergiu.Daniluk]

Hi @cHrome08 

If the fabric/switches/end-hosts does not know about the DMAC it will need to resolve it.

So it is expected that 1-2 icmp requests to be lost while ARP resolution.

Stay safe,

Sergiu

[cHrome08]

Hi @Sergiu.Daniluk,

Is it a norm that to configure L3VNI, the L3VNI needs to be in a VRF?
Can L3VNI be configured in a way where there is no VRF required?
All the examples on the internet shows thats VRF is required.
Thanks.

[Sergiu.Daniluk]

Hello @cHrome08,

Nexus switches only support symmetric IRB, which means the presence of Layer 2 VNIs and Layer 3 VNIs. L2VNIs are used in case of, as the name says, layer 2 traffic, while the L3VNI is required for routing the traffic between L2VNIs.

It is not mandatory to have L3VNIs. You can have a pure L2 VNI fabric, and have the gateway external to the fabric, similar to this:

In this case, the routed traffic will traverse this path:

Source of the images: https://yves-louis.com/DCI/?p=1480 

However, if you want to have the gateway distributed on the vxlan leaf switches, then you must configure the L3VNI which is associated to a VRF (only one L3VNI per VRF).

Stay safe,

Sergiu