Dynamic Routing over vPC possible in 7.2 NX-OS ?

[ahmad82pkn]

Hi, As per Nexus Release 7.2 , now dynamic routing is supported over Nexus that may over come Design 3 Problem.

but i am not getting any example configuration for this, Also there is confusing pre req for this, can someone help explaining ? or may be some good example config?

http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/

7.2 Release for Nexus states.

Dynamic Routing over vPC

Dynamic Routing over vPC feature enables L3 routing protocols such as OPSF to form adjacency with the two vPC peer chassis. The equal routing cost matrices must be configured on applicable interface on each of the vPC peers, failure to do so can result in blocking the traffic. Asymmetric routing feature has to be implemented to address this issue and to configure Dynamic Routing over vPC. Additionally, when Dynamic Routing over vPC is enabled a warning log message is printed.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/7_x/nx-os/release/notes/72_nx-os_release_note.html

[Steve Fuller · Accepted solution]

Hi,

The Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide has a section Layer 3 over vPC for F2, F2E and F3 Modules that shows the supported designs as of today. The design 3 that is shown over at Brad's blog is I believe what is shown in Figure 16 as one of the supported designs.

In terms of configuration, see the Configuring Layer 3 over vPC for F2, F2E and F3 Modules section for details. It's essentially a single command,  layer3 peer-router, within the vpc domain context. As noted in this section you also need the peer-gateway command enabled.

Regards

[ahmad82pkn]

Hi Steve, you really shared some useful information, and i was happy until i read

at the end of paragraph and sample configuration of your second hyper link (Configuring Layer 3 over vPC for F2, F2E and F3 Modules)

Next Paragraph is about (Configuring a vPC Peer-Gateway) where it says again.

(If routing protocol adjacencies are needed between vPC peer devices and a generic Layer 3 device, you must use physical routed interfaces for the interconnection)

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/nx-os/interfaces/configuration/guide/b-Cisco-Nexus-7000-Series-NX-OS-Interfaces-Configuration-Guide-Book/configuring-vpcs.html#task_31495353B68A43729F336446F6CFCE2E

(is it outdated information ? i mean in one paragraph it says its doable and then in next it says not doable, or may be not doable specific to 4.2? if so in that case cisco shouldn't confuse readers :( )

Will really appreciate if you can clarify this doubt.

[Steve Fuller · Accepted solution]

Hi,

I agree that it's confusing and think the wording you highlighted is associated with how vPC originally worked and is related to how things would work with only the peer-gateway configuration.

As I understood it from the link to Brad Hedlund's blog, the topology you want to be able to run is with two Nexus 7000 and two firewalls, all connected to a common subnet and each establishing some routing protocol peering relationship e.g., OSPF, to all other devices on the subnet.

As shown in Figure 16 of the Configuring Layer 3 over vPC for F2, F2E and F3 Modules section of the configuration guide, this is now supported.

The reason for this is that when Layer 3 with vPC is configured, the Nexus 7000 series router no longer decrements the TTL of packets crossing the peer link. This means that for example an OSPF hello packet sent with TTL=1 from the router on lower left of figure 16 can be sent to the left Nexus 7000, across the vPC peer link to the right Nexus 7K, and then forwarded by the right Nexus 7K and received by the lower right hand router.

I think the confusion simply comes from the fact that the vPC Peer-Gateway section of the document was not updated when the Layer 3 over vPC section was added.

Regards

[ahmad82pkn]

Thank you Steve, just waiting for Nexus to come :) then it will be play time :) Bit tensed as it will be first time working on nexus and need to migrate from 65xx to Nexus at Core. Hope all goes well. Thanx once again.

[avvotyakov]

Hi all!

Today turn on this feache on my N7K pair, but they start logg messege

%VPC-2-L3_VPC_UNEQUAL_WEIGHT: Unequal weight routing is not supported in L3 over vPC. Please make sure both vPC peers have equal link cost configuration.

I chek configure on N7K and they same on both. How i can debug this problem, and  how to find out what exactly Unequal?

[Steve Fuller]

Hi Alexandr,

What is required is, as the error message states, that the routing metric be the same across the path via Switch 1 and Switch 2.

If you look at Figure 20 of the Configuring Layer 3 over vPC for F2, F2E and F3 Modules section of the configuration guide, you'll see a metric of 10 configured on the SVI of Switch 1 and a metric of 20 configured on the SVI of switch 2.

In this configuration the routed path would be preferred by Switch 1 i.e., using Switch 1 as the next hop. As this topology uses a vPC, any traffic from B and R1 at the bottom of the diagram could be hashed on the port-channel to Switch 2. In that case the traffic would have to sent across the vPC peer link to Switch 1, for forwarding on to R2 and A at the top of the diagram.

In this topology the normal vPC loop avoidance mechanism i.e., traffic that has crossed the peer link cannot be sent on a vPC member port, would result in Switch 1 dropping the traffic.

What is the topology you're trying to use in which you see this message?

Regards

[avvotyakov]

Hi.

My topology is full mesh dual vpc(4 N7k)

all metric on interface in routung protokol is default and same on both configure nexus.

You say about loop avoidance rule for vpc, but new feache  Layer 3 over vPC, came for resolve this problem ?

[Steve Fuller]

Hi,

I'm not clear what you mean by full mesh dual vPC. Are you able to include a diagram or more information detailing the physical and logical topology?

In terms of the L3 over vPC feature, it made some subtle changes such as not decrementing TTL for traffic over the peer link but it does not, and cannot, change the fundamental loop avoidance mechanism for vPC.

Regards

[avvotyakov]

My topologi like this.

[avvotyakov]

But if this feache dos't change loop avoidence rule, why this feache need? packet will drop if they came from vpc port, transmit to peer link and tray to out to vpc port. and what change with L3 over vPC feature? i know about dos't decrement ttl, why it's done?

[Steve Fuller]

Hi,

The feature can't change the loop avoidance rules to any great extent. If I assume OSPF as your routing protocol, then changing the loop avoidance would allow an OSPF Hello packet (DMAC 0100:5e00:0005) from N5K-1 to flow to N7K-1, across the peer link to N7K-2 and then on the vPC member port back to N5K-1. As it's a multicast frame, there would then be nothing to stop N5K-1 sending the frame straight back up to N7K-1 and so circulated again. This basic loop avoidance rule of vPC cannot be changed.

What it now allows is that a MAC frame sent from N5K-1 to N7K-1 can be sent across the peer link without the TTL being decremented such that N7K-2 can receive and process the frame, allowing it to form a routing relationship with N5K-1.

In terms of the above diagram, I presuming you want all devices to peer with each other? If so this looks to be Figure 17 from the configuration guide so should be supported, but please confirm which devices are trying to peer with each other, and with which routing protocol.

If you're trying to peer the N5K can you confirm the NX-OS version operating on those switches?

Also can you post the relevant configuration from each of the four devices i.e., the interface they're using to establish the peering, the routing protocol configuration etc.?

Regards

[ahmad82pkn]

Today we got Nexus and were able to test. OSPF neighborship is working fine  over common vlan segment 10.0.0.0/24 between Router 1 and Nexus 2.

where physical topology is Router1---Nexus1---VPC---Nexus 2

and loopbacks were pingable, so we are good to go :)

[IT Department]

We also turned on this feature to help with a ASA transparent firewall we are sticking between our 2 N7K VDCs.  

%VPC-2-L3_VPC_UNEQUAL_WEIGHT: Unequal weight routing is not supported in L3 over vPC. Please make sure both vPC peers have equal link cost configuration.

We too get this error message but we have not changed any of the EIGRP metrics or introduced delays on any interfaces.  What could be causing the Nexus to think there is unequal weight routing occuring in our core?

[avvotyakov]

Cisco tac say, this is a cosmetic message and you can ignore it. If you have problem with cpu, may say 'no layer3 peer-router syslog'