WAN Routing between N5K (L3-vPC) & VSS (MEC) with Link Aggregation

[AdnanShahid]

Hi All,

We want a solution for routing between N5K and VSS with aggregated WAN links. Here is the scenario.

DC1: It has 2 cisco 6509 with VSS. There are 4 server farm cisco4948 switches connected with VSS with redundant uplink via MEC. Server gateway is the VSS. VSS is running Eigrp routing.

DC2: This is a new datacenter we are going to establish soon. We are planning 2 N5K at core layer with L3 daughter card and 4 N2K as server farm switch. 2 N5K will have vPC peer between them. Each 4 N2K will connect with redundant uplink via vPC with this N5K. N5K will run Eigrp routing and will be the gateway of this new DC server.

WAN between DC1 & DC2: DC1 VSS will connect with DC 2XN5K with 2X10G links. we want to do MEC at VSS side and L3 vPC at DC2 side.

If we have VSS at both end it might not be a problem. Both the link will work together as 20G aggregated link. But as we are using N5K at one end, so it creates a confusion whether it will work properly is this scenerio or not. This is my 1st question.

Also I would like to know,

1) In VSS I have configured 1 VLAN interface for server gateway. But in N5K do I have to configure at 2 switch seperately?

2) In WAN routing VSS shows as 1 device. Does this 2 N5K will show as 2 seperate hops or L3 vPC will allow them to act as a single device while traceroute from one end to another end.

I am not sure if my questiones are so elementary level or not. As I am very new to this technology I would like you all to get me some suggestions or documention or links regarding this design. I am also attaching the diagram.

BR//

Adnan

[AdnanShahid]

Hi Jerry,

Very nice to inform you that we have just tested the routing scenerio and it worked just fine. Thank you so much for your suggestion and cooperation.

During our activity we have solved a problem of VPC PEER. In our previous configuration the VPC keep-alive link was communicating over Vlan 3 and via ethernet 1/3 with trunk. But this creates a problem and our VPC peer is not getting up if I restart any one of the peer switches. Later we found that when the Peer switch is up than all its SVI's remain shutdown untill its peer link is up and peer link is not getting up as its VLAN 3 is down. Hence peer is not forming. For this we have shutdown the VLAN 3 and configure the IP on ethernet1/3. After that the problem resolved and peer is forming automatically after any peer switch rebooted.

Now, I would like to let me know one important routing issue that I hav found during redundancy testing. Also I would request you to suggest me for few more features.

1) Routing Issue: During our activity we found that if we shutdown or plugout the L3 link between N5K-1 and L3-Uplink-Switch, then routing become as follows between the 2 pc (PC1 and PC2).

     - PC1-PC2: PC1>L3SWGW>N5K-2>PC2. - This is fine.

     - PC2-PC1: PC2>N5K-1>L3LinkbetweenN5K-1&N5k-2>N5K-2>L3SW>PC2 - (!!)

Now, my queries are,

     - Why PC2 to PC1 is using L3 link between 2 5K instead of VPC-Peer Link. It could have been much faster?

     - Why HSRP is allowing to keep N5K-1 as gateway where as traffic from PC2 can easily go to N5K-2 and then use the L3 link of N5K-2 to reach the L3UplinkSW. In that case, PC1-PC2 and PC2-PC1 traffice would follow the same routed pattern.

                             [N5K------------1]

                             /  |        | ||       \

     [PC1] -- [L3SW]    |        | ||          [N2K] -- [PC2]

                             \  |        | ||       /

                               [N5K------------2]

2) VPC-Peer Role Issue: In our VPC peer we have seen that N5K-2 role is "Primary" and N5K-1 role is "Secondary:. I try to make N5K as Primary with Role Priority. Then it shows "Secondary, Operational Primary" in N5K-1 and "Primary, Operational Secondary" in N5K-2. I am not sure how it has formed Primary/Secondary and whether it has any major impact or not??... Kindly need you suggestion.

3) Suggestion on STP: I am not sure yet whether there is any STP bottleneck here or not. How can I verify it? Is there any best practices regarding how to check and how to verify. Please suggest.

4) Authentication and Configuration in HSRP: It would be nice if you can suggest,

     - Should we implement authentication in HSRP? Will it hamper performance or increase latency or increase process utilization? Is is recommended?

     - Is our HSRP configuration is ok? Specially Priority and Preempt?

5) Authentication in Eigrp: Should we implement authentication in Eigrp? Will it hamper performance or increase latency or increase any process utilization? Is is recommended by cisco?

Thank you so much again to give me such nice understading and insights to build our network. Really appreciate you cooperation in this.

We will test one more scenario (L2 and L3 together with 2 different WAN link with L3SW) before proceed for final deployment on 9-Aug-2012. Will let you know too.

Thanks again.

Regards,

Adnan

[Jerry Ye]

1) This is the fundamental concept of how HSRP works in a vPC environment. Both HSRP routers will forward traffic upstream. If the up stream L3 link(s) is broken, the N5K<->N5K L3 link will be used as last resort. Why it is not using the L2 vPC peer-link? It is because vPC PL is L2 and not going to forward L3 traffic (speed here doesn't make any difference).

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/design_guide_c07-625857.pdf

Page 23 explains how HSRP and vPC works together.

2) This is normal about the role. vPC doesn't support role preemption. In terms of the impact, you have to understand how vPC works first. If vPC PL failed, "Secondary" switch will take down its vPC POs, hence, if role changed, instead of N5K2, now N5K1 will take down vPC PO during PL failure.

3) I am not sure what you are talking about? STP bottleneck? It is configured as vPC and all gateway and links will forward traffic. I guess you can verify if there is any STP BLK ports.

4) It is upto your corporate policy to implement HSRP authentication or not. I will not make any suggestion here.

5) It is upto your corporate policy to implement EIGRP authentication or not. I will not make any suggestion here.

Regards,

jerry

[AdnanShahid]

Hi Jerry,

Thanks for your reply.

1) Routing with HSRP: OK. It seems that our test bed scenario is working exactly as written in the document.

2) VPC-Peer Role Issue: OK. I will make my vPC "Primary" switch as the N5K-1 and also make HRSP active on that switch. This way it will be easy for us to understood: vPC Primary Swtich <=> N5K-1 <=> HSRP active gateway.

3) STP: Actually the design is quite standard in terms of integration between two N5K and fex with N2K. I just wanted to know if there is any STP issue that I might need to be taken care of in this design.

4, 5) Authentication in HSRP and Eigrp: Currently there is no such policy that I have to use authentication. However considering the security I would like to implement this. Now I just wanted to know, if I implement this than will there be any chance of any performance issue (such as high process utilization in the switch or high latency in traffi etc.) might happen or not. If there is no chance of such issue, then we would like to implement it.

However, thank you soo much for your reply and suggestions, otherwise we couldn't have finish the configuration so early.

Regards,

Adnan

[Jerry Ye]

3) Nexus 2000 is not a switch, hense, no STP is running between them.

4) Protocol authentication should not cause performance.

Regards,

jerry

[AdnanShahid]

Thanks a lot Jerry. I will check and let you know.

Regards,

Adnan