Nexus 3048 vPC (10G guidelines)

[Clayton Howarth]

Hi,

I'm looking for some input on configuring vPC on the Nexus 3048.

I know that it's supported to use 1G interfaces for the vPC Peer-Link, but using 2x1G for the Peer-Link would make the Peer-Link a bottle-neck if the 10G ports are used in a vPC. What about using 2x10G ports for the Peer-Link and using the remaining 10G ports in one, or potentially two vPCs. Should that work or is it in any way not recommended? The reason I'm asking is that the 10G ports are called "Uplink" ports in the data sheets for the 3048.

We are planning to connect some servers to the 3048s using vPC with each server connected by 4x1G interface (2x1G for each switch), and then we want to connect a Netapp storage system with two controllers using 2x10G each (the controllers are active/passive, so you can think of it as two separate systems). We would connect controller A with vPC 1x10G to each switch, and controller B in the same way with vPC 1x10G to each switch. Does anyone see any problem using this setup?

Thank you, regards

Clayton

[Reza Sharifi]

Hi Clayton,

I have never used the 3000 series, but most likely you can use 2 of the uplink ports for vPC and the other 2 for Netapp controllers  You should also use 10Gig as this vPC doc states:

vPC peer link: The vPC peer link is the link used to synchronize states between the vPC peer devices. The vPC peer link carries control traffic between two vPC switches and also multicast, broadcast data traffic. In some link failure scenarios, it also carries unicast traffic. You should have at least two 10 Gigabit Ethernet interfaces for peer links.

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps11541/white_paper_c11-685753.html

HTH

[Clayton Howarth]

Thank you for your input Reza.

For non-scale deployments, they are now supporting 1G for the vPC:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus3000/sw/release/503_u3_1/n3k_rel_notes_503_u3_1.html#wp417209

But all the configuration guides are still saying 10G. I was looking for a configuration guide that stated 1G after the 5.0(3)U3(1) release, but they are all still saying 10G. In the latest release config guides, they've taken out the vPC chapter all together as far as I can see.

Best regards,

Clayton

[erinrjones]

Did you ever move forward with this?  I am thinking of doing something very similar w/ EMC 10GB iSCSI connections.  I would be interested in any thoughts you had, especially around the suitablility of the 3048 for the 10gb iSCSI SAN -> 1gb iSCSI client stepdown.    

[Clayton Howarth]

Hello Erin,

funny you should post just now. I posted the question in may, and it wasn't until yesterday that I got to try it out.

We used two of the 10G ports on the 3048 for the peer link. We connected one of the two remaining 10G ports from each 3048 to each Netapp controller.

Everything worked just fine when configuring, but the system is not yet in production because there's some data migration and new servers going in. But I think it'll work fine.

The 10G to 1G stepdown will be interresting. I think that the storage system is in a good place on the 10G ports, since multiple systems will access it at the same time. And since the switches are uplinked through a 2x1G PortChannel to a Gig-ethernet environment, I think it'll be all right.

[erinrjones]

Thanks for the reply.  I think this is one of those things where I am over engineering and worrying about problems that will never bite me based on teh load I am expecting.  I am going forward w/ the design, very similar to what you mentioned w/ 2 x 10gb cross connects and 2 x 10gb SAN connects per switch. 

I am still trying to work out the decision between 3048s and Catalyst 4948E, did you consider other switches?  What made you settle on the 3048s?

[NeverOutofTune]

I have a small data center that could use a couple of 10Gig ports and some 1Gig ports.  Since the VPC peer link should not normally handle unicast traffic if everything is dual homed then I would think that using a pair of 1Gs for the peer link would be sufficient.  

Is it possible to use more than a pair for the peer link?  Such as using 2 x 1G ports per switch?  I would like to leave my 10G ports available for other uses.