How to Uplink Stacked C2960S to C3560X

[julito4589]

I'm trying to decide between uplinking a pair of stacked C2960S switches that are part of a Hyper-V cluster environment to a C3560X switch OR just connect the Hyper-V hosts directly to the C3560X.

Here's some background of the environment:

- Hyper-V 2012 R2 environment

- Storage Network:

    - Two clustered Equallogics (8 eth connections each -- 4 active/4 passive) connected to the stacked C2960S

    - 4 Hyper-V hosts, each with quad NICs for iSCSI traffic connected to the stacked C2960S

- LAN Network

   - 4 Hyper-V hosts, each with an additional quad NIC for LAN traffic (Management, cluster, migration, public)  ==> to be connected to the rest of the LAN (directly to 3560X or through C2960S)

Originally I had considered connecting the Hyper-V hosts' LAN NICs directly to the C3560X. That would take 16 of the 48 ports (4 NICs X 4 hosts) in the C3560X, which is a core switch and has a bunch of other servers connected to it.

I have been thinking, however, that there's already a fair amount of traffic going through the C3560X and adding, for example, the cluster and VM migration to it may create some congestion. Instead, I've been considering connecting the Hyper-V hosts' LAN ports to the stacked C2960S, isolate the traffic with VLANs, and set up an uplink to the C3560X switch. The benefit of this is that, for example, cluster traffic and VM migration traffic would be handled in the C2960S, which have a smaller traffic load than the C3560X.

What implementation is more efficient?

Furthermore, if I were to uplink the C2960S switches to the C3560X, what is the best way to achieve this given the characteristics of the switches?

The complete model number for the C2960S is WS-2960S-48TS-L and for the C3560X is WS-C3560X-48T-S. There are SFP ports in the C2960S, but not in the C3560X. Is my best option here to team a couple of the interfaces on each type of switch and uplink that way?

Attached is a drawing with both options roughly outlined.

Thank you in advance for any recommendations.

[Reza Sharifi]

I like option 2.  This will keep traffic local to the 2960s when you are doing VM migrations. 

As for the uplinks, use 2 1Gig fiber interfaces and connect the 3560x to 2960s stack using a Portchannle. You also need an uplink module for the 3560x.

HTH

[julito4589]

Thanks for the input. Would you mind providing more info on the uplink module for the 3560X? There's an empty module there right now.

On the other hand, wouldn't I achieve about the same speeds if I use a PortChannel with to Gb interfaces and use CAT6? Unless the SPF modules are 10Gb, but the C2960S SPF are 1Gb

[Reza Sharifi]

If the distance is short enough (less than 100 meters) and you have available copper ports you can use copper.  This will also save you some money since you don't have to buy the uplink module any more.  If you decided to use the fiber for uplink here is the part number:

C3KX-NM-1G (Four GbE port network module)

http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-3750-x-series-switches/data_sheet_c78-584733.html

common practice is to use copper for end devices and fiber for uplink as it may give you more bandwidth to use.

HTH

[julito4589]

Yes, I have available ports on the 3560X and the distance is less than 100m. So I'll likely go with CAT6.

The key to option 2 however is correctly configuring the VLANs so that the iSCSI, cluster and live migration stay local to the C2960S stack and the management and public traffic go out to the 3560.

My doubt in that case is about the VLAN configs in the 3560. The PortChannel on both switches have to be set up as trunks to allow the management and public VLANs, but what after that?

There are some VLANs configured in the 3560, but this device is not doing L3 switching. The traffic is sent to a 3825 that does the VLAN translation (I inherited this config and plan to eventually set the 3560 to do the L3). Would I have to set the rest of the ports to be able to deal with the VLAN traffic coming through the PortChannel?

So for example, if I have port 1 and 2 in the 3560 configured for the Hyper-V uplink PortChannel and I have another server on port 3 that needs to communicate with the VMs, would I have to add the respective vlans to that port?

[Reza Sharifi]

ok, so in the current environment neither the 2960s or the 3560 are doing routing correct?  They are just layer-2 switches and all vlans terminate at 3825 and the 3825 connects to the provider (Internet) correct?

If that is the case than you have a couple of choices:

1-Keep it the way it is and add all the vlans to layer-2 switches and have the 3825 route between vlans.  The issue with this design is that inter-vlan communication has to go up to the router and come back down which will make things slow specially for your iSCSI and VM traffic.

2-Keep the 2960s as layer-2 and make the 3560 a layer-3 switch/router and terminate all the vlans/SVIs on it. This means for all vlans the inter-communication with have to go to the 3560x which is powerful enough to handle. You than have a trunk all vlans between the 2960s and the 3560 using a Portchannel. You also need a layer-3 link between the 3560 and the 3825. In this scenario you can have a vlan exist on both 2960s and 3560. For example, if you have vlan 10 that is already on the 2960s and you also need to have the same vlan on the 3560, you just configure a port on the 3560 as access port and add it to vlan 10.

HTH 

[julito4589]

>>ok, so in the current environment neither the 2960s or the 3560 are doing routing correct?  They are just layer-2 switches and all vlans terminate at 3825 and the 3825 connects to the provider (Internet) correct?

CORRECT on both counts

In choice 1 you mention that the "inter-vlan communication has to go up to the router and come back down which will make things slow specially for your iSCSI and VM traffic." While this is correct, is it not the case that for example, the iSCSI traffic will usually stay local to the 2960 and the ports that are tagged for that vlan. Same goes for the Cluster and Migration (VLANs) traffic. So in reality, the only (VLAN) traffic that would leave the 2960, traverse the 3560, and reach the 3825 is the management and public. Is this correct? In other words, in the trunk between the 2960 and the 3560 I was assuming that I'd add the vlans for management and public. Cluster and migration would not be part of that trunk.

In choice 2, that's actually I want to achieve eventually so the 3560 deals with the vlan routing. I'm not quite ready to migrate all the existing vlans though. So n00b question here. Can I have the 3560 act as a L3 switch for ONLY some of the VLANs (e.g. Hyper-V cluster and migration), but stay as a L2 for the rest of the vlans?

[Reza Sharifi]

Same goes for the Cluster and Migration (VLANs) traffic. So in reality, the only (VLAN) traffic that would leave the 2960, traverse the 3560, and reach the 3825 is the management and public. Is this correct?

Yes, as long as the other vlans don't need to reach the Internet than you don't have to add them to the trunk

Can I have the 3560 act as a L3 switch for ONLY some of the VLANs (e.g. Hyper-V cluster and migration), but stay as a L2 for the rest of the vlans?

Yes, that is possible as long as the layer-2 vlans don't need to reach the Internet. So the 3560 would serve as a layer-2 and layer-3 switch.

HTH

[julito4589]

Sorry if I re-iterate one of my last questions, but I want to make sure I understand your last statement.

Here's the current sh vlan in the 3560 (vlans names changed for privacy)

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Gi0/1, Gi0/4, Gi0/6, Gi0/7
                                                Gi0/8, Gi0/9, Gi0/10, Gi0/11
                                                Gi0/12, Gi0/13, Gi0/14, Gi0/15
                                                Gi0/16, Gi0/17, Gi0/18, Gi0/19
                                                Gi0/20, Gi0/24, Gi0/25, Gi0/26
                                                Gi0/27, Gi0/28, Gi0/30, Gi0/31
                                                Gi0/32, Gi0/33, Gi0/34, Gi0/35
                                                Gi0/36, Gi0/37, Gi0/38, Gi0/39
                                                Gi0/40, Gi0/41, Gi0/42, Gi0/43
                                                Gi0/44, Gi0/45, Gi0/46, Gi0/47
                                                Po1, Po2
6    Mickey                              active    Gi0/23
7    Minnie                    active
8    Donald                           active
9    Pluto                            active

So as you can see there are some VLANs configured on the 3560, but only one of the ports is tagged. The 3825 is doing most of the heavy lifting for those VLANs.

If I were to configure the 3560 to do inter-VLAN routing for the Hyper-V environment VLANs, I'd configure, say, interfaces Gi0/46, Gi0/47 with VLANs for management and public cluster traffic. But would configuring the 3560 to do L3 switching/routing for these vlans affect the existing ones (e.g. Mickey, Minnie, Donald, Pluto)?

Furthermore, what would happen with the traffic going through VLAN1? Some of the servers connected in those interfaces will need to communicate with the VMs in the Hyper-V environment which would be coming on VLANs other than VLAN1.