Nexus 1000v port-channels questions

[stheriault99]

Hi,

I’m running vCenter 4.1 and Nexus 1000v and about 30 ESX Hosts.

I’m using one system uplink port profile for all 30 ESX Host; On each of the ESX host I have 2 NICs going to a Catalyst 3750 switch stack (Switch A), and another 2 NICs going to another Catalyst 3750 switch stack (Switch B).

The Nexus is configured with the “sub-group CDP” command on the system uplink port profile like the following:

port-profile type ethernet uplink

vmware port-group

switchport mode trunk

switchport trunk allowed vlan 1,800,802,900,988-991,996-997,999

switchport trunk native vlan 500

mtu 1500

channel-group auto mode on sub-group cdp

no shutdown

system vlan 988-989

description System-Uplink

state enabled

And the port channel on the Catalyst 3750 are configured like the following:

interface Port-channel11

description ESX-10(Virtual Machine)

switchport trunk encapsulation dot1q

switchport trunk native vlan 500

switchport trunk allowed vlan 800,802,900,988-991

switchport mode trunk

switchport nonegotiate

spanning-tree portfast trunk

end

interface GigabitEthernet1/0/18

description ESX-10(Virtual Machine)

switchport trunk encapsulation dot1q

switchport trunk native vlan 500

switchport trunk allowed vlan 800,802,900,988-991

switchport mode trunk

switchport nonegotiate

channel-group 11 mode on

spanning-tree portfast trunk

spanning-tree guard root

end

interface GigabitEthernet1/0/1

description ESX-10(Virtual Machine)

switchport trunk encapsulation dot1q

switchport trunk native vlan 500

switchport trunk allowed vlan 800,802,900,988-991

switchport mode trunk

switchport nonegotiate

channel-group 11 mode on

spanning-tree portfast trunk

spanning-tree guard root

end

Now Cisco is telling me that I should be using MAC pinning when doing a trunk to two different stacks , and that each interface on 3750 should not be configured in a port-channel like above,  but should be configured as individual trunks.

First question: Is the above statement correct, are my uplinks configured wrong?  Should they be configured individually in trunks instead of a port-channel?

Second questions: If I need to add the MAC pinning configuration on my system uplink port-profile can I create a new system uplink port profile with the MAC pinning configuration and then move one ESX host (with no VM on them) one at a time to that new system uplink port profile? This way, I could migrate one ESX host at a time without outages to my VMs. Or is there an easier way to move 30 ESX hosts to a new system uplink profile with the MAC Pinning configuration.

Thanks.

[stephen.ehlas]

Hi

I have never heard of doing MAC pinning on a runk to a Cat 3750. As far as I know, mac-pinning is used on vPC links only, which can only run when you are connected to another upstream Nexus (2000, 5000, 5500, 7000 series).For a Cat 3750 I would use LACP as an etherchannel.

The config looks ok to me, although i am wondering why you have vlan1 allowed on the vmware port-group. It will either need to be enabled on the port channel, or better yet, disabled completl, as it may pose a security risk.

HTH

Steve

[sprasath]

Hello,

From what I understood, you have the following setup:

     - Each ESX host has 4 NICS

     - 2 of them go to a 3750 stack and the other 2 go to a different 3750 stack

     - all 4 vmnics on the ESX host use the same Ethernet port-profile

          - this has 'channel-group auto mode on sub-group cdp'

     - The 2 interfaces on each 3750 stack are in a port-channel (just 'mode on')

If yes, then this sort of a setup is correct. The only problem with this is the dependance on CDP. With CDP loss, the port-channels would go down.

'mac-pinning' is the recommended option for this sort of a setup. You don't have to bundle the interfaces on the 3750 for this and these can be just regular trunk ports. If all your ports are on the same stack, then you can look at LACP. The CDP option would not be supported in the future releases. In fact, it is supposed to be removed from 4.2(1)SV1(2.1) but I still see the command available (ignore 4.2(1)SV1(4) next to it) - I'll follow up on this internally:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_2_1_s_v_2_1_1/interface/configuration/guide/b_Cisco_Nexus_1000V_Interface_Configuration_Guide_Release_4_2_1_SV_2_1_1_chapter_01.html

For migrating, the best option would be as you suggested. Create a new port-profile with mac-pinning and move one host at a time. You can migrate VMs off the host before you change the port-profile and can remove the upstream port-channel config as well.

Thanks,

Shankar