Replacing 6500 VSS with Nexus 7K

beausoleilb1 · ‎09-02-2010

A basic overview of our current network design for core and distribution is as follows:

Core:

6506-E chassis in VSS

Distribution:

6509-E chassis single sup connected to each of the 6506 chassis in the core.

We are looking to replace the VSS in the core with Nexus 7K switches. Since the Nexus switches are not virtual by nature, I have to consider a HA protocol like HSRP or GLBP. Currently our links between the Core and Dist are L3 Point-To-Point. We wish to maintain this configuration in the new design. Since the two links on the Dist side are port-channels we currently have 2Gbps throughput. If I use HSRP with LACP one link goes into suspended mode, therefore only giving us 1Gbps throughput. With GLBP we continue to have 2Gbps. From what I can tell, ECMP (Equal-cost multipath) is enabled and should be in effect at this point, but I cannot find a way to prove this since I do not have another 6509 for distribution (my test distribution switch is just a 3560).

My question is this...

What is the best practice way to go in this situation? Should we allow STP to provide a look free environment and use priorities to determine the path (using HSRP and LACP) or should I use GLBP and have an active-active scenario from DIST to Core? Any suggestions on a solid way to test that load balancing is in fact working properly?

Excuse me if I misspeak on anything here as this is my first solo attempt at replacing a core. I've been living in the distribution and access layers for many years now.

Thank you in advance for any and all feedback.

Brian

Jon Marshall · ‎09-02-2010

The Nexus switches do support VPC (Virtual port channel - think of MEC on VSS) and VDC (virtual switches within the same chassis). Also HSRP has been modified on the Nexus 7K switches so that in effect your pair of Nexus switches become active/active in HSRP terms. It might be worth you having a read of some of the Nexus white papers which specifically cover Nexus/6500 intergration -

Nexus whitepapers.

Definitely have a read of the last white paper in the list about VPC and HSRP interaction.

Jon

beausoleilb1 · ‎09-02-2010

I do have a VDC running with a vPC link between them for the Core switches. I guess I failed to mention that in the post... I tried to provide as much detail as I could. When I configure HSRP according to the guides, it recommends ACTIVE mode for the interfaces in the port-channel rather than ON mode, which then enabled LACP on the interfaces, which in turn suspends one of the interfaces in the channel, thus creating an active-standby scenario. With GLBP the guides recommends ON mode for the interfaces in the port-channel which then disables LACP, thus creating an active-active scenario. I have read many docs and papers, and searched the web for best practices sample configs, but since the Nexus is a fairly new product, i'm sure there is a limited amount of information out there yet. I will also continue to investigate.

Thanks for your reply.

Chad Peterson · ‎09-02-2010

When using LACP you shouldn't get into an active/standby scenerio. There is something wrong here.

Can you send some output to show whats going on? What is attached to each link...how is it connected etc.

beausoleilb1 · ‎09-03-2010

Chad-

A quick overview of the topology for this lab/prep setup:

Two Nexus 7010 switches with a VDC named Core and vPC between them. Our normal distribution switch is a 6509 but since I do not have a spare I am trying to make do with a spare 3560 running the latest IOS. From the 3560 I have a L3 1Gb link to each of the N7K switches in a port channel. The default route is the GLBP address of this port channel, and OSPF is the routing protocol. I'm not sure how to tell which side of the link is placing the interface in Suspended mode. Hope some of the outputs below provide some details and a path.

What I am trying to accomplish is replacing the 6500 VSS in the core with the Nexus 7K switches while keeping the L3 portchannels i currently have in place from the distribution to the core, hopefully providing a load balance scenario.

N7K1-Core# show run int po51

interface port-channel51

no ip redirects

ip address 10.95.254.42/29

ip router ospf 51 area 0.0.0.0

glbp 51

ip 10.95.254.41

N2K1-Core# show run int po51

interface port-channel51

ip address 10.95.254.43/29

ip router ospf 51 area 0.0.0.0

glbp 51

ip 10.95.254.41

ENPBX3-Lab#show run int po2

interface Port-channel2

no switchport

ip address 10.95.254.44 255.255.255.248

N7K1-Core# show port-channel summary | i 51

51 Po51(RD) Eth LACP Eth3/3(s)

N7K2-Core# show port-channel summary | i 51

51 Po51(RU) Eth LACP Eth3/3(P)

DIST-Test# show etherchannel summary | i Po2

2 Po2(RU) LACP Gi0/1(s) Gi0/2(P)

DIST-Test#show etherchannel detail

Group: 2

----------

Group state = L3

Ports: 2 Maxports = 16

Port-channels: 1 Max Port-channels = 16

Protocol: LACP

Minimum Links: 0

Ports in the group:

-------------------

Port: Gi0/1

------------

Port state = Up Cnt-bndl Suspend Mstr Not-in-Bndl

Channel group = 2 Mode = Active Gcchange = -

Port-channel = null GC = - Pseudo port-channel = Po2

Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.

A - Device is in active mode. P - Device is in passive mode.

Local information:

LACP port Admin Oper Port Port

Port Flags State Priority Key Key Number State

Gi0/1 SA susp 32768 0x2 0x2 0x1D1 0x5

Partner's information:

LACP port Admin Oper Port Port

Port Flags Priority Dev ID Age key Key Number State

Gi0/1 FA 32768 a8b1.d457.3142 7s 0x0 0x32 0x303 0xF

Age of the port in the current state: 0d:00h:42m:45s

Port: Gi0/2

------------

Port state = Up Mstr Assoc In-Bndl

Channel group = 2 Mode = Active Gcchange = -

Port-channel = Po2 GC = - Pseudo port-channel = Po2

Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.

A - Device is in active mode. P - Device is in passive mode.

Local information:

LACP port Admin Oper Port Port

Port Flags State Priority Key Key Number State

Gi0/2 SA bndl 32768 0x2 0x2 0x1D2 0x3D

Partner's information:

LACP port Admin Oper Port Port

Port Flags Priority Dev ID Age key Key Number State

Gi0/2 SA 32768 18ef.63e4.6dc2 29s 0x0 0x32 0x303 0x3D

Age of the port in the current state: 0d:00h:29m:24s

Port-channels in the group:

---------------------------

Port-channel: Po2 (Primary Aggregator)

------------

Age of the Port-channel = 2d:00h:24m:11s

--More-- 2010 Sep 4 00:49:25 N7K2-Core %$ VDC-2 %$ %ARP-2-DUP_SRC_IP: arp [4119] Source address of packet received from a8b1.d457.3142 on Vlan10(port-channel200) is duplicate of local, 10.95.254.198

Logical slot/port = 2/2 Number of ports = 1

HotStandBy port = null

Passive port list = Gi0/1 Gi0/2

Port state = Port-channel L3-Ag Ag-Inuse

Protocol = LACP

Port security = Disabled

Ports in the Port-channel:

Index Load Port EC state No of bits

------+------+------+------------------+-----------

0 00 Gi0/2 Active 0

Time since last port bundled: 0d:00h:29m:30s Gi0/2

Time since last port Un-bundled: 0d:00h:29m:41s Gi0/2

Thanks for taking a look.

-Brian

drussell · ‎03-12-2012

Did you find out what caused the problem? I had something similar, also with a 3560 and Nexus VPC.

gnijs · ‎05-18-2012

Just my 2 cents: i think this is because you created a routed portchannel. Just create a trunked portchannel with VLAN 51 for example (switchport trunk allowed vlan 51). then put ip config on "int vlan 51".

The portchannel to the acess should come up with P/P (both links present)