The question regards to vPC without peer-keepalive link

[wangchunhao20001]

Hey there, 

I got a question about vPC when I was watching a demo of ACI on Youtube. The speaker said vPC can be set up between 9396 leafs and Nexus 5548 even without peer-keepalive link between both 9396s or 5548s like the diagram shown. But he didn't tell why and how, so my question is that is that possible? If the answer is YES, how that works?  And how to configure? In such scenario that doesn't have synchronization of peer-keepalive through group channel.

Thanks,

Bob

[wangchunhao20001]

anyone here know about this issue? Please feel free to give any comments and suggestion to me. I really appreciate that. 

Thanks,

Bob

[mfarrenkopf]

Hi Bob,

It is my understanding that the peer link and the peer keepalive are integral parts of vPC connectivity.  If the peer keepalive goes away, the switch in the secondary role will suspend its vPC links.

That said . . . the peer keepalive does not need to be a physical link.  It just needs to be connectivity at layer 3.  For example, we have a routed access layer using Nexus 5548UP switches.  They establish EIGRP relationships with the upstream distribution layer.  I use the loopback IPs as my peer keepalive endpoints.  Works just fine.

So if there's layer 3 connectivity between the Nexus 5548s and the Nexus 9396 switches, yes, it will work fine, even without direct physical connectivity.

Matt

[mfarrenkopf]

And I just realized the folly of my reply . . .

Of course, the peer keepalive has to be between the pair of switches.  So the peer keepalive between the 5548s.  And a peer keepalive between the 9396s.  Unless there's been an update to vPC (I've not investigated the 9396s), the keepalives are only between the same model switches -- the direct vPC peers.  There should not be any keepalive communication between the 5548s and the 9396s.

But my statement still applies -- layer 3 connectivity between the 5548s will suffice for establishing the peer keepalive.  Layer 3 connectivity between the 9396s will suffice for establishing the peer keepalive.  It doesn't need to be a physical link between the pairs of switches.

Matt

[wangchunhao20001]

Hi Matt,

Thanks for your response and answers, I really appreciate that. As you mentioned, vPC peer devices can be established through L3 connection as long as both are same devices, such 5548s or 9396s, because peer-keeplive and peer-link can work through L3 routing protocol, so that the physical connection between vPC peer devices is not necessary.   

As Cisco recommended, the better way to do so is using the dedicated port and port-channel as trucks for redundancy, and also do not use the peer link itself to send and receive vPC peer-keepalive messages. So that makes me a little bit confuse which one should be a good design. If we use non physical connection we should establish four physical connections at least two for peer-keeplive, two for peer-link and business traffic. Also, we will have to take a risk that is one more potential failure point for the vPC establishing in case N5548 goes down as above figure shown, right? Any suggestions for designing if I want to use such feature on the network?  Thanks a lot!

In addition, I alway see Cisco shows the classic Leaf-spine architecture that leafs do not connect each other and spines do not connect each other as well. So I assume either they don't use vPC in this scenario or use L3 to establish vPC peer devices. What do you think? By the way, do you have any references or configuration examples that regard to establishing vPC peer through L3 can recommend to me? I appreciate it. 

Thanks,

Bob

[mfarrenkopf]

Hi Bob,

Last few weeks have been irregular for my schedule.  My apologies for the delay.

No, I don't run routed traffic through any vPC VLANs.  Yes, I have separate physical connections for the layer 2 and layer 3 traffic.  On my 5548s I have the 16-port expansion module.  As a side note, we also have the layer 3 module.  If you don't have that, I believe your option for the peer keepalive will indeed be limited to the management link.  With this in mind:

Eth1/1 and Eth2/16 aggregated as Po1 and is the vPC peer link.

Eth1/2 and Eth2/15 are layer 3 links back to the distribution using EIGRP.

Loopback0 is the management IP address and is used for the vPC peer keepalive.

Here's my vPC domain configuration:

vpc domain 1
role priority 8192
system-priority 4096
peer-keepalive destination 10.4.255.24 source 10.4.255.23 vrf default interval 400 timeout 3
delay restore 120
peer-gateway
auto-recovery
ip arp synchronize

I've been running this configuration for . . . 6 years?  2009-ish . . . and have never had a partial failure that took down one of the two peer link interfaces or one of the two routed links.

I'll be honest and say I don't recall seeing a Cisco best practices document on this.  We run a routed access network.  When I was presented with building our vPC environment, I just took the routed access principles and applied it to this setup -- adding in two layer 3 and configuring EIGRP to the distribution.  Documentation at the time talked of establishing the vPC peer keepalive through mgmt0; that seemed like a single point of failure to me.  At the time, there were strong admonitions about trying to run a routing protocol on vPC-environment VLANs.  So the simplest solution was to just run two layer 3 links and be done with it.

If you'd like more specific diagrams of what our design looks like, I'm happy to draw something up for you.  It's really not complex.

Matt

[wangchunhao20001]

Hi Matt,

No worries. Thanks for your response with your busy schedule. I really appreciate if you are comfortable to draw a specific diagram that it's using in your environment. If the peer-keepalive traffic could also be marked that will be very helpful for my understanding and designing in the future. Becuase I'm still not really sure how vPC peer-keepalive is maintained through the physical connection in your particular network. I'm assuming you are using layer 3 links to carry peer-keepalive traffic as well as vrf DEFAULT is used as isolated routing table for peer-keepalive traffic based on you mentioned that you have two separate physical connection, layer 2 links are using as vPC peer link, so layer 3 links are using for...... 

By the way, have you ever done the verify test? The STP blocking ports can be eliminated when vPC is being configured? 

Have a good day ahead of you!

Thanks,

Bob