vPC & OSPF

[abimadaro4462]

Hello,

I have the following scenario, two nexus 9K are configured as vPC peers with HSRP, the upstream is connected to active/standby firewall with OSPF configured.

below is the configuration;

SW1 & SW2

----

router ospf 1
bfd
router-id 10.10.10.4
area 0.0.0.4 nssa no-summary

interface Vlan20
ip ospf dead-interval 3
ip ospf hello-interval 1
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.4
ip ospf bfd

interface Vlan20
hsrp 20
ip 20.20.20.1

vpc domain 10
peer-switch
role priority 100
peer-keepalive destination 10.10.10.2 source 10.10.10.1 vrf vpcvrf
peer-gateway
layer3 peer-router
auto-recovery
ip arp synchronize

interface port-channel1
vpc peer-link

Switch one neighbor is flapping between EXSTART and EXCHANG

Neighbor ID Pri State Up Time Address Interface
10.10.10.5 1 FULL/ - 00:25:17 20.20.20.3 Vlan20
10.20.10.10 1 EXCHANGE/ - 0.065025 20.20.20.6 Vlan20

Please advice if the above configuration is correct.

[Reza Sharifi]

Hi,

What is the physical connectivity look like?

Are both Nexus switches connected to both firewalls?

Do you have OSPF running between the switches?

You would need a vlan with a /28 subnet to span between the 2 switches and both firewalls.

HTH

[abimadaro4462]

Hello,

Thanks for your reply, I have the following scenario;

The links from the FWs are aggregated links with a vPC, currently for testing only one VLAN in this trunk, used on both NEXUS and FWs for OSPF. HSRP is configured from the NEXUS side.

[Reza Sharifi]

Hi,

The physical diagram looks correct. What is the vlan that spans between the firewalls and the switches? Is that a /28 subnet with one IP on each switch and one IP on the firewall? Also, is OSPF running between the 2 switches? What type of Firewall are these?

HTH

[abimadaro4462]

Hello,
It's /29 subnet as the following;
SW1 20.20.20.2
SW2 20.20.20.3
HSRP 20.20.20.1
FW 20.20.20.4
Yes the OSPF is running between the two switches, as it's configured under vlan 20 on both switches. the firewalls are FortiGate.

[Reza Sharifi]

Hi,

You don't need HSRP for this subnet. So, try deleting the HSRP config and test. Make sure you have ospf running between the 2 switches.

HTH

[abimadaro4462]

I will test it, but in case of I delete the HSRP and I want to configure the OSPF area as Totally Stubby area, what the nexthop of the default route which will be generated in this case from the firewall to the switch, will be?

[abimadaro4462]

Hello,
I did test it, still the same issue, the OSPF is flapping;
10.10.10.2 1 FULL/BDR 00:17:29 20.16.8.4 Vlan20
10.20.10.10 1 INIT/DROTHER 00:00:02 20.16.8.1 Vlan20

The first neighbor is between the peers which is stable, the second one is with the FW

[abimadaro4462]

Hello,
Even on the FW, for 40 seconds (Dead time interval) is forming neighbor ship with the first peer switch, then forming with the second one. So it's not forming the neighbor ship with both switches.

[paul driver]

Hello
It doesn't look correct - However  you can check the vpc consistency for type1/2 errors
sh vpc
sh vpc peer-keepalive
sh vpc consistency-check global
sh vpc consistency-check vpc xx

As for the physical connectivity, you'll need at a minimum of 4 ports for a valid vpc domain creation.

Each vpc switch.
1x routed port p2p connection in a vrf for peer-keep-alive link ( not to traverse the peer-link)

1x routed port p2p connection for routing backup path between vpc switches ( this could run over the peer-link via svi however a better way would to have its own physical routered port)

2x aggregated trunk l2 interfaces for peer-link 

[abimadaro4462]

Hello,

Thanks for your response, what's wrong with the posted configuration please?

I've checked the status of the vPC everything just looks fine and identical between both vPC peers.

[paul driver]

Hello
What i meant was i didn’t see any separate routed interface for your ospf , So your routed backup path between vpc switches seems to be running over the peer link which isnt ideal (recommended to be separate routed port)

Also you have a routed vpc towards each fw which if am not mistaken isnt supported in vPC due to vpc loop avoidance ( vpc FW1<-> vpcswitch1<->vpcswitch2<-x->FW1) these links should not be in any vPC

[paul driver]

Hello
I managed to dig out ciscos best practice design relates to the 7ks which actually states what i am trying to explain.

Layer 3 and vPC: Guidelines and Restrictions

Attaching a L3 device (router or firewall configured in routed mode for instance) to vPC domain using a vPC is not a supported design because of vPC loop avoidance rule.

To connect a L3 device to vPC domain, simply use L3 links from L3 device to each vPC peer device.

[abimadaro4462]

Hello,
Thanks this is very helpful so far, but I have some questions please;
- Do not use a Layer 2 vPC to attach L3 device to a vPC domain unless L3 device can statically route to the HSRP address configured on vPC peer device. This statement means that definitely I won't be able to use OSPF between the FW and vPC peers in case of the links are configured as a vPCm right?
- In case of using p2p L3 between the FWs and vPC peer based on the topology that I've shared before, how the OSPF will looks like in this case? FW1 will form neighborship with SW1 and SW2? how the routing table on the FW will looks like in this case? or should I use HSRP but even in this case as I do have another link to the second peer switch the routing table still ambiguous for me.
- one last point, as for the back routed path, I believe I don't need it in my case I don't have orphan ports connected on any port of vPC peers, am I correct?

[abimadaro4462]

Hello,
I've found this, please check this section in the following URL;
Layer 3 over vPC Supported Topologies
https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/interfaces/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_Interfaces_Configuration_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_Interfaces_Configuration_Guide_7x_chapter_0100...

This means that it's now supported to have L3 over a vPC link in case of the FW or the router supports LACP, right?