Solved: OSPF Peering on Nexus vPC Environment

Geunyoung Park · ‎01-15-2023

I have studied that without

layter3 peer-router

command it is impossible to take ospf neighbor relationship between Nexus vPC Devices and L3 device on vPC environment.

But in my PNETLab, they peer very well without any problem(I placed 3 N9K devices, of which 2 devices are vPC Peer Device and the other is a host(just L3 device)). Host is connected with two vPC Peer Devices on vPC, and the host uses layer3 port channel on which an IP address is set. And vPC Peer Devices uses SVI on ospf peering.

Main configurations on vPC domain are these

 peer-switch, peer-gateway, ip arp synchronize, auto-recovery.

Is this an exceptional case caused by simulation environment? Or do I miss something about it?

Christopher Hart · ‎01-17-2023

Hello!

It is possible to form a unicast routing protocol adjacency between a vPC-connected router and two Cisco Nexus switches in a vPC domain without the

 layer3 peer-router

vPC domain enhancement configured. This happens when the unicast routing protocol packets (such as OSPF DBD, LSU, LSR, LSAck, etc. packets) hash to the "correct" vPC peer instead of hashing to the "incorrect" vPC peer. However, this configuration is not supported without the

layer3 peer-router

vPC domain enhancement configured because in certain failure scenarios (such as a single link of the vPC going down, or a change in the hashing algorithm of the vPC-connected router after a software upgrade), OSPF unicast packets will start hashing to the "incorrect" vPC peer and cause the OSPF adjacencies to go down (typically stuck in an EXSTART state, in the case of OSPF).

This failure scenario is described in more detail in the "Unicast Routing Protocol Adjacencies over a vPC with vPC Peer Gateway" example failure scenario section of the Understand Virtual Port Channel (vPC) Enhancements document.

In other words, just because you are able to form a unicast routing protocol adjacency over a vPC without the

layer3 peer-router

vPC domain enhancement does not mean that doing so is supported. This applies to both a virtual lab (such as the one you have here) as well as a production environment.

I hope this helps - thank you!

-Christopher

View solution in original post

Geunyoung Park · ‎01-17-2023

Thanks a lot everyone.

Actually, yesterday I found the reason why the 3 devices peered well in my PNETLab. It was simply because of the limitation of virtual lab environment.

I captured a OSPF unicast message(from NXOS-Host to NXOS-01) on vPC Peer Link with Wireshark, and the message's TTL was 1. It means, the unicast message's TTL was not decremented WITHOUT

LAYER3 PEER-ROUTER

COMMAND even though it flowed

NXOS-Host" --> "NXOS-02" --> "NXOS-01.

Additionally I read a post on Reddit (https://www.reddit.com/r/networking/comments/lcgxow/nexus_9kv_ospf_works_without_the_layer3_peerrouter/). He or she experienced the same phenomenon like me, and applied the same configuration on both EVE-NG lab environment and real Nexus devices. The result was different. Devices peered well on EVE-NG without

layer3 peer-router

command, but in the real physical environment they couldn't peer as expected.

So my the conclusion is simple. There is a problem in virtual lab environment.

Thanks again.

View solution in original post

MHM Cisco World · ‎01-18-2023

and same as you mention before it lab problem,
I doing and check the TTL decrement, and it not
even when I run peer-gateway in both NSK vPC peer the TTL not decrement.
I also change the load balance in Port-channel to include only scr IP and same result.

View solution in original post

Georg Pauwen · ‎01-16-2023

Hello,

post a schematic drawing of your topology showing how your devices are connected, and indicate what part of your topology is not working according to what you have studied.

MHM Cisco World · ‎01-16-2023

connect one router to each vPC pair and check if the two router make OSPF between or not ?
that the goal of peer-router, it dont have effect to vPC peers it have effect on L3 device connect to vPC peers.

balaji.bandi · ‎01-16-2023

depends on use case and test scenarios - how the traffic flow passing.

you need to more test and post us you environment and config to validate is that correct. (by forming OSPF does not mean its working) when the traffic flows need to be in the correct manner

below example give you idea and failure scenarious help you :

https://www.cisco.com/c/en/us/support/docs/ip/ip-routing/118997-technote-nexus-00.html

https://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/

OLD docs but still valid for best practice :

https://www.cisco.com/c/en/us/support/switches/nexus-5000-series-switches/products-implementation-design-guides-list.html

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Geunyoung Park · ‎01-16-2023

Hi, I'm uploading the configuration and topology.

=======================================================

<< NXOS-01 >>

feature ospf
feature interface-vlan
feature lacp
feature vpc

vlan 1,40

vrf context KEEP
vpc domain 10
peer-switch
role priority 100
peer-keepalive destination 192.168.1.2 source 192.168.1.1 vrf KEEP
peer-gateway
auto-recovery
ip arp synchronize

interface Vlan40
no shutdown
no ip redirects
ip address 40.40.40.1/24
no ipv6 redirects
ip router ospf 1 area 0.0.0.0

interface port-channel1
switchport mode trunk
spanning-tree port type network
vpc peer-link

interface port-channel40
switchport access vlan 40
vpc 40

interface Ethernet1/1
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/2
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/3
no switchport
vrf member KEEP
ip address 192.168.1.1/30
no shutdown

interface Ethernet1/4
switchport access vlan 40
channel-group 40 mode active




=======================================================

<< NXOS-02 >>

feature ospf
feature interface-vlan
feature lacp
feature vpc

vlan 1,40

vrf context KEEP
vpc domain 10
peer-switch
role priority 200
peer-keepalive destination 192.168.1.1 source 192.168.1.2 vrf KEEP
peer-gateway
auto-recovery
ip arp synchronize

interface Vlan40
no shutdown
no ip redirects
ip address 40.40.40.2/24
no ipv6 redirects
ip router ospf 1 area 0.0.0.0

interface port-channel1
switchport mode trunk
spanning-tree port type network
vpc peer-link

interface port-channel40
switchport access vlan 40
vpc 40

interface Ethernet1/1
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/2
switchport mode trunk
channel-group 1 mode active

interface Ethernet1/3
no switchport
vrf member KEEP
ip address 192.168.1.2/30
no shutdown

interface Ethernet1/4
switchport access vlan 40
channel-group 40 mode active




=======================================================

<< NXOS-Host >>

feature ospf
feature interface-vlan
feature lacp

vlan 1,40

interface port-channel40
no switchport
ip address 40.40.40.40/24
ip router ospf 1 area 0.0.0.0

interface Ethernet1/1
no switchport
channel-group 40 mode active
no shutdown

interface Ethernet1/2
no switchport
channel-group 40 mode active
no shutdown

=======================================================

I mainly refered this document.

https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/nx-os-software/217274-understand-virtual-port-channel-vpc-en.html#anc33

As you can see, there is no

layer3 peer-router

command in these configurations. According to documents it is impossible to peer well between vPC Deivces and host router, however, they do very well.

Thanks

MHM Cisco World · ‎01-16-2023

check NSK-host are there two OSPF neighbor or ONE ??

MHM Cisco World · ‎01-16-2023

interface port-channel40
no switchport
ip address 40.40.40.40/24
ip router ospf 1 area 0.0.0.0

interface Ethernet1/1
no switchport
channel-group 40 mode active
no shutdown

interface Ethernet1/2
no switchport
channel-group 40 mode active
no shutdown

I get it, you use PO L3 for OSPF you not use VLAN SVI in NSK-host, that why you not see the effect,
use VLAN SVI for OSPF and check

show ip ospf neighbor again

Geunyoung Park · ‎01-16-2023

Hi

I changed OSPF interface from po40 to SVI(interface vlan 40), but nothing changed. They establish OSPF neighbor relationship well.

Actually I'm also curious why they really can't peer each other without

layer3 peer-router

config. Document says it's because of the unicast messages of the routing protocols, but when I captured the OSPF messages with Wireshark, the Hello messages were multicast, not unicast.

Geunyoung Park · ‎01-16-2023

NSOS-Host peers with both of two vPC Peer Devices, and the state is FULL.

MHM Cisco World · ‎01-16-2023

can I see


show ip ospf neighbor ?

MHM Cisco World · ‎01-16-2023

can I see the last config and

show ip ospf neighbor ?

Geunyoung Park · ‎01-16-2023

Hi

This is the result of

show ip ospf neighbor

NXOS-Host# sh ip ospf nei
OSPF Process ID 1 VRF default
Total number of neighbors: 2
Neighbor ID   Pri    State            Up Time     Address        Interface
40.40.40.1       1   FULL/BDR     00:48:07   40.40.40.1    Po40 
40.40.40.2       1   FULL/DR       00:48:06   40.40.40.2    Po40

Thanks

paul driver · ‎01-16-2023

Hello

Cisco strongly recommends that you follow these guidelines when connecting a Layer 3 device to vPC domain.
Strong Recommendations:
● Use separate Layer 3 links to connect L3 device (like router or firewall in routed mode for instance) to a vPC domain
● Do not use a Layer 2 vPC to attach L3 device to a vPC domain unless L3 device can statically route to the HSRP address configured on vPC peer devices.
● Use individual Layer 3 links for routed traffic and a separate Layer 2 port-channel for bridged traffic if both routed and bridged traffic are required.
● Enable Layer 3 connectivity between vPC peer device by configuring a VLAN network interface for the same VLAN from both devices or by using a dedicated L3 link between the 2 peer devices (for L3 backup routing path purposes).

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Geunyoung Park · ‎01-16-2023

Hello Paul

Yes, you're right. But the

layer3 peer-router

command is the config that enables L3 connectivity without any dedicated L3 link.

In order to test the effect of this command, first I removed the command and attatched a router(NXOS-Host) to the vPC Peer Devices with vPC. The result I expected is the OSPF peering problem issue, but actually there is no special peering issue in my PNETLab environment.

Thanks.

paul driver · ‎01-16-2023

Hello

@Geunyoung Park wrote:

without
layter3 peer-router
command it is impossible to take ospf neighbor relationship between Nexus vPC Devices and L3 device on vPC environment.

But in my PNETLab, they peer very well without any problem

Apologes now I understand what you are asking, that you ARE using the L3 peer-router , I rea you OP as your NOT using that feature.

If you run show VPC , you should see at least peer-gateway correct, if so then yes this would be applicable solution to run a routing protocol over

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul