System NVE Infra-Vlans on Nexus 9300 Running on 10.3.x

[john.gregory]

Hi,

I have a vPC Border Leaf Switches that is connected to a pair of Firewall.

Border Leaf-1 Eth1/1 <Po1> Firewall-1 Port1

Border Leaf-2 Eth1/1 <Po1> Firewall-1 Port2

Border Leaf-1 Eth1/2 <Po2> Firewall-2 Port1

Border Leaf-2 Eth1/2 <Po2> Firewall-2 Port2

I have transit vlan 609 (10.0.69.0/29) between the Border Leafs and Firewalls, this serves as an L3 External Connectivity of a Tenant-VRF. In Border Switches, SVI 609 is using a anycast gateway address 10.0.69.1/29, while the Firewall SVI 609 is using 10.0.69.2/29. From Border Leaf-1, I cannot ping the Firewall 10.0.69.2, but from Border Leaf-2 I can able to ping the Firewall without any issue. I shutdown the vPC Po2 towards Firewall-2 and when it happened I can able to ping 10.0.69.2 from Border Leaf-1. It looks like from Border Leaf-1 the traffic is being blackholed when all vPC links are up.

Question 1: Do I need to setup/configure System NVE Infra-Vlans between the Border Leafs? Not sure if my problem is related to this feature.
Question 2: Is this feature still applicable on nx-os 10.3.x?

[AshSe]

@john.gregory  could you please draw topology diagram and share for better understanding. PFB, my first hand understanding of your topology:

It looks yours is a double sided vPC configuration. Hope your BLs are properly configured for the same. Please confirm and improvise the diagram.

[john.gregory]

Hi AshSe,

Yes the diagram is correct. Port-Channel 1 is in vpc 1 and Port-Channel-2 is in vpc 2. Fortigate Firewall is HA, so whatever you configure from Firewall 1, will just replicate it over to the Standby Firewall 2.

[AshSe]

Hello @john.gregory  here comes the improvised diagram as per your explanation:

As per the diagram:

• There are two vPCs viz. vPC1 and vPC2
• There is a single Firewall cluster

IMO, you need:

• Single vPC towards firewall cluster
• Check if you can create a vPC in the firewall and create a single vPC Po towards the BLs.

Apparently your topology should look like:

[Pavel Tarakanov · Accepted solution]

>In Border Switches, SVI 609 is using a anycast gateway address 10.0.69.1/29

>From Border Leaf-1, I cannot ping the Firewall 10.0.69.2, but from Border Leaf-2 I can able to ping the Firewall without any issue.

With what source do you ping 10.0.69.2? If from 10.0.69.1 then it's expected, as ICMP reply can be balanced to another border leaf and will not be forwarded to the first one.

Question 1: Do I need to setup/configure System NVE Infra-Vlans between the Border Leafs? Not sure if my problem is related to this feature.

https://www.cisco.com/c/en/us/support/docs/switches/nexus-9000-series-switches/214624-configure-system-nve-infra-vlans-in-vxla.html

Infra vlan needed for transmitting VXLAN encapsulated traffic via peer-link

Question 2: Is this feature still applicable on nx-os 10.3.x?

Before configuring it as an SVI, the backup VLAN needs to be configured on Cisco Nexus 9200, 9300-EX, and 9300-FX/FX2/FX3 and 9300-GX platform switches as an infra-VLAN with the system nve infra-vlans command.

https://www.cisco.com/c/en/us/td/docs/dcn/nx-os/nexus9000/103x/configuration/vxlan/cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-release-103x/m_configuring_vxlan_93x.html