Which topology is better and recommend for vPC Peer Keepalive Link?

[mhiyoshi]

Dear all,

I am little curious about the vPC management design for Peer Keepalive Link.

According to the following URL which is traditional Nexus7000 Series vPC Best Practices.

Best Practices for Virtual Port Channels (vPC) on Cisco Nexus 7000 Series Switches  

Strong Recommendations:
When building a vPC peer-keepalive link, use the following in descending order of preference:

1. Dedicated link(s) (1-Gigabit Ethernet port is enough) configured as L3. Port-channel with 2 X 1G port is even better.
2. Mgmt0 interface (along with management traffic)
3. As a last resort, route the peer-keepalive link over the Layer 3 infrastructure

Page 27 of 129

This is strong recommendation however if there is the attached topology which is actually recommendation?

In my understanding if the traffic is devided between management and data. Option 2 is better and can be recommendation

but if the topology is shared network, Option 1 can be possible.

Best Reagards,

Masanobu Hiyoshi

[Sergiu.Daniluk · Accepted solution]

Hi @mhiyoshi 

The "Strong Recommendations" are strictly for the Nexus 7000/7700 simply because in this case you have most of the times two SUPs.

In this case, connecting the mgmt0 interface directly between N7K is not recommended, simply because a SUP switchover or a failure on one chassis can lead to PKA failure.

In my opinion, based on few years of troubleshooting VPC almost everyday, I would always recommend using option2 for Nexus 9000: PKA over mgmt0, and mgmt0 connected to oob mgmt network. There are two advantages using this option:

1. You still have access to mgmt0 interface which is an out-of-band management port (not subject to COPP) - so it can save you in case of a network meltdown.

2.  You don't have to use dedicated ports just for some keepalive messages.

Hope it helps,

Sergiu

[mhiyoshi]

Hi Sergiu,

Thank you for your precious comment!  I have reflected below. I appreciate if you can check it.

I am normaly using mgmt0 interface for OOB and PKA (without OOB I use Option1) however CCO URL below normally recomendation is redundant management ports. I think basically Nexus9K or 3K BOX switch has only 1 mgmt port so redundancy can not be possible right? Ofcourse if I use other data ports (Downlink/Uplink) redundancy can be possible howerver I have to separate for OOB and DATA network and at least 1 to 2 ports are consumed for OOB network.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 9.3(x) 

Figure 3. Separate Switch Required to Connect Management Ports for vPC Peer-Keepalive Link

However, if you use the management interfaces for the peer-keepalive link, you must put a management switch connected to both the active and standby management ports on each vPC peer device (see figure).

Best Regards,

Masanobu Hiyoshi

[Sergiu.Daniluk]

Hi @mhiyoshi 

"you must put a management switch connected to both the active and standby management ports on each vPC peer device "

This refers to Nexus 9500 where you can have 2x SUP modules.

However, my recommendation stays - use mgmt0 interface for PKA, by connecting both SUPs of each N9500 to the OOB switch. This way regardless if you have a SUP failure or a switchover, the PKA will continue to function and you will still have oob connectivity to your switch.

About your statement:

"Ofcourse if I use other data ports (Downlink/Uplink) redundancy can be possible howerver I have to separate for OOB and DATA network and at least 1 to 2 ports are consumed for OOB network."

Note that front ports (downlink/uplink) are not technically OOB. They are actually inband. The big difference is that the communication from all frontports to CPU (mgmt data) is subject to COPP (control plane protection policy). If there is one port which sens excessive traffic to CPU and COPP kicks in and start dropping packets, then all front ports will be affected by this.

In other words, regardless of the platform you have - use mgmt0 for PKA and connect it to an OOB switch/network.

Stay safe,

Sergiu

[mhiyoshi]

Hi Sergiu,

Thank you very much! I am gradually understanding the difference between OOB and inbound frontports.

I mean that the regardless of the Nexus platform, inband frontports management communication is subject to CoPP, so you can recommend for using mgmt0 for PKA connected to OOB switch/network.

Best Regards,

Masanobu Hiyoshi