Cisco Nexus N93108TC-EX VPC Recommendations

[aghufran]

Hello,

I have to create a vpc domain comprising of 2 x Cisco N93108TC-EX.

For the peer-link I intend to use a port-channel comprising of 2x40-G QSFP links on both the devices.

For the keep-alive I wanted to request advise on the following please:

1) Is it better to connect the mgmt-0 interface on both devices as a cross-connect (point-to-point) directly?

2) Is it better to use an out-of-band switch and then use the mgmt-0 interface?

3) Keep the mgmt-0 interface for pure out-of-band management and use a 1-Gig port under keep-alive dedicated vrf and connect the two swithces together in a cross-connect (point-to-point)?

4) Also, if the mgmt-0 interface can and/or should be used both for VPC keep-alive and out-of-band switch management as well?

I am thinking about using the mgmt-0 interface (under mgmt vrf) for pure out-of-band management and use 1-Gig port (under keep-alive dedicated vrf) for vpc keep-alive.

If it is essentail to know about the NX-OS please assume 7.x.

Please provide reasoning for your answers.

Kind advise.

Regards,

Abid Ghufran.

[Reza Sharifi]

Hi,

Is it better to use an out-of-band switch and then use the mgmt-0 interface?

Yes, use mgmt-0 for that by connecting both interfaces to an OOB switch.  This works really well.

Also, if the mgmt-0 interface can and/or should be used both for VPC keep-alive and out-of-band switch management as well?

Yes, that is correct. It can be used for both.

HTH

[aghufran]

Hi Reza,

Thanks a lot.

Actually i am more interested in finding out about the recommended industry practice. From research so far the recommendations i have come across are to keep the management in-line and use a separate physical Gig interface (not the mgmt0) back to back for keep-alive. My concern is if a back to back connection would make it difficult to differentiate in between a link failure or a node failure. If the keep-alives go through a switch then it would be possible to differentiate. However that would create an additional dependency on an 3rd device for the vpc to operate for example in case of a power outage related reboots.

Regards,

Abid Ghufran.

[Andrea Testino · Accepted solution]

Hi Abid,

Regarding your vPC keepalive question - The Cisco Best Practice for the keepalive is the following, in descending order of preference:

1. Dedicated link(s) (1-Gigabit Ethernet port is enough) configured as L3. Port-channel with 2 X 1G port is even better.

2. Mgmt0 interface (along with management traffic)

3. As a last resort, route the peer-keepalive link over the Layer 3 infrastructure

In my experience working in the Datacenter TAC, most customers have the L3 dedicated port-channel between the vPC pair. You can also view this recommendation in the Nexus 9000 vPC Configuration Guide here

Hope that helps.

Thank you!

[aghufran]

Thank you very much for the information you have provided. It is certainly what I had been looking - clear and precise.

Kind regards,

Abid Ghufan.