VPC for L3 links

[sanjay.sangwan]

Hi,

I have 2 cat 6509 working as core switches mostly on L3 interfaces running OSPF and further connected to the Campus distribution ( 2x6509) and datacentre distribution ( 2x6509).I have to replace both the core switches with 2 Nexus 7K with the same configuration.Is there any possibility that I can use VPC on L3 links , Is it recommended using VPC on L3 links or What is the way that both Nexus can act as a single cluster.

Sanjay

[Lucien Avramov]

Yes you sure can have L3 links on your router side / cat 6k.

On the N5k, you will have L2 VPC going to the L3 port-channel and that should be fine, just use the mode as access on the 5k and specify a vlan going towards this L3 link.

[sanjay.sangwan]

Hi,

Actually my question was that if I have point-to-point layer 3 port channels between 2 Nexus 7K and cat 6500 and OSPF is running on to that.

One /30 IP is on Nexus and another on CAT6500.Does VPC applicable here also as the Cisco documentation says that VPC only

works with Layer 2 port channels.

Please suggest..

regards

Sanjay

[Lucien Avramov]

No in this scenario it will not work.

Here is what can work:

catk6 (l3 link)-----vPC-----Nexus 7000-----vPC-----(l3 link) other router

Nexus 7000 would be a L2 vPC. Note this can also be a Nexus 5000.

[Chad Peterson]

To expand on Lucian's comment, because I'm sure the next though will be...can I run OSFP over a vlan and just carry THAT over my VPC.  You don't want to do this either.

We don't support running routing protocols over VPC enabled VLANs.

What happens is that your 6500 will form routing adj with each Nexus 7000....lets say Nexus 7000-1 and 7000-2.  Note my picture below.

Lets say that R1 is trying to send to a network that is behind R2.  R1 is adj to 7000-1 and 7000-2...we have equal cost paths.  CEF chooses that 7000-1 to route the packet, however Etherchannel load balancing chooses the physical link to 7000-2.  7000-2 will need to switch the packet over the VPC peer-link to 7000-1.  7000-1 receives the packet and tries to send it out VPC member port to R2....however egress port drops the packet.  This happens because we don't allow packets received from VPC member link send over VPC peer-link to be sent out another VPC member link.

I'd suggest to run an L3 link from your 6500 to each Nexus 7000 if you do want to do L3 on it.

[NguyenT11]

Lets say I have the 2x7K's in my DC connecting to a VSS system in my campus.  What is the best way to configure the connections between the DC and my Campus?  I want each N7K to connect to each 6500 (4 links total).

I was thinking the following:

-  From the VSS, create MEC to each of the 7K's.

-  From each N7K, configure EC to the VSS system, no VPC involved.

-  Create a point to point network with OSPF on each MEC.

Is it necessary to create a routed VLAN between the N7K's for OSPF?  If so can I have this VLAN traverse the VPC Peer-Link or do I need to have this VLAN on it's own dedicated link (non peer-link)?

Thanks

[Chad Peterson]

The plan you proposed will work well.

As far as the OSPF peering between the two Nexus, overall I'd say your safer NOT running it over the peer-link.  However if you do just ensure that it is the ONLY peering done over VPC enabled vlans, and those are the ONLY peers you see for that vlan.  The main thing is to just ensure traffic does not come in on VPC member link, cross peer link, and leave a member link.

If you peer on a separate vlan or L3 link between your Nexus that is not carried over VPC, then you will always ensure this cannot happen.

[alwayschillz]

Hii,

I am bit confused with this Nexus Vpc Layer -3 issue!!..Really need a help on it.Let me explain my scenario.I have 2  nexus 7k connecting to a pair of N5k with a double sided VPC.Now i have a router connected to access port of N5k and need a EIGRP neighbor ship with both N7k.It is single broadcast segment.The N7k 1 & 2 will have SVI s with  same subnet as that of  router.Above the N7k it will be only layer 3 links connected with no VPC.

So i need to know whether the router connected to N5k can form a routing protocol neighbor ship with the N7k through a VPC and reach the other destination above N7k on Layer 3.

Attaching the topology for reference.

Thanks in Advance....

[Chad Peterson]

Hi Roney,

This design wouldn't be supported as the router is forming a neighbor with the Nexus 7000s over a VPC link.  This can cause packets to be dropped due to being passed over the VPC Peer-Link, then back out another VPC.  However if the only path it will go through is single L3 links, then it will work...however just not advised.

You can get this to work, but you just need to make sure the traffic from that router will never need to cross another VPC.

Chad

[m-radojevic]

Chad,

I am curious why the behaviour as you described it, is not allowed. If i am reading your diagram correctly, you are stating that traffic coming in from R1 to N7K2 on a related VPC and relayed over VPC peer link to finaly leave N7K1 towards R2 on a different VPC will be dropped?

I beleive that this has something to do with loop prevention mechanism, but i do not see how is it possible to create a loop if this was to be an allowed behaviour, since traffic is leaving out different VPC? Etherchannel should drop any eventual looped traffic on R2 side anyways?

Maybe i am looking at things from a wrong perspective...

I will appreciate any answer.

Thanks.

Misha

[Chad Peterson]

So take routing out of the equation....assume its all L2.  If I get a packet on my peer-link and the dest is a vpc member port.  I check to see if my peer has his member port up...assume he does.   How do I know if you forwarded that packet or not?  I don't...I must assume that since I got the packet from you and you also have a vpc link to the destination up (remember this is a shared virtual interface), I assume you must have also sent the packet...so I don't.

It makes more sense if you take routing out of the equation.  The thing is when routing is in the picture we don't care...the logic is the same.  Traffic ingress on peer link, traffic egress on vpc member...peer's vpc is up too...drop

[m-radojevic]

Chad,

Makes perfect sense. I was looking for this answer for a while and I really appreciate your prompt and clear response.

I am wondering if you ever tried to setup routing protocol over VPC enabled VLAN in a lab.

What symptoms would one experience, 50% traffic loss due to the etherchannel load balancing?

Would this also cause neighbors to flap?

I apologize for asking hypotetical question.

Thanks again.

Misha