11-02-2009 07:15 AM
Ok, we have two ESX 4 servers configured right now, dual CNAs connected to two separate upstream switches, 5020-A and 5020-B. We have the 1000V setup what we believe to be the proper way. All of the networking is connected to the 1000V - Service Console, VMotion, NFS, and all the VM VLANs. What we see are intermittent disconnects on our NFS datastores and service console connections 10-20 minutes after we add a second vmnic into the uplink. If we remove one of the vmnics and go back to a single connection, everything returns to normal.
I've included relevant parts from a show run and show spanning-tree from the 1000V... just looking for some insight on where to look next. Thanks.
Nexus1000V# show run
...
switchname Nexus1000V
vlan 1,60,160,235-236,610,669
...
port-profile system-uplink
capability uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan all
channel-group auto mode on sub-group cdp
no shutdown
system vlan 60,235-236,610,669
state enabled
port-profile ServiceConsole
vmware port-group
switchport mode access
switchport access vlan 60
no shutdown
system vlan 60
state enabled
port-profile NXNFS
vmware port-group
switchport mode access
switchport access vlan 669
no shutdown
system vlan 669
state enabled
...
Nexus1000V# show spanning-tree
VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
VLAN0060
Spanning tree enabled protocol rstp
Root ID Priority 32828
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32828 (priority 32768 sys-id-ext 60)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
lveth3/1 Desg FWD 4 128.1 Edge P2p
lveth3/4 Desg FWD 4 128.4 Edge P2p
lveth4/1 Desg FWD 4 128.1 Edge P2p
lveth4/5 Desg FWD 4 128.5 Edge P2p
VLAN0160
Spanning tree enabled protocol rstp
Root ID Priority 32928
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32928 (priority 32768 sys-id-ext 160)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
lveth3/5 Desg FWD 4 128.5 Edge P2p
lveth3/6 Desg FWD 4 128.6 Edge P2p
VLAN0235
Spanning tree enabled protocol rstp
Root ID Priority 33003
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 33003 (priority 32768 sys-id-ext 235)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
lveth4/6 Desg FWD 4 128.6 Edge P2p
VLAN0236
Spanning tree enabled protocol rstp
Root ID Priority 33004
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 33004 (priority 32768 sys-id-ext 236)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
lveth4/4 Desg FWD 4 128.4 Edge P2p
VLAN0610
Spanning tree enabled protocol rstp
Root ID Priority 33378
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 33378 (priority 32768 sys-id-ext 610)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
lveth3/2 Desg FWD 4 128.2 Edge P2p
lveth4/2 Desg FWD 4 128.2 Edge P2p
VLAN0669
Spanning tree enabled protocol rstp
Root ID Priority 33437
Address 0200.0c00.003c
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 33437 (priority 32768 sys-id-ext 669)
Address 0200.0c00.003c
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 2 128.4096 Edge P2p
Po2 Desg FWD 2 128.4097 Edge P2p
lveth3/3 Desg FWD 4 128.3 Edge P2p
lveth4/3 Desg FWD 4 128.3 Edge P2p
11-04-2009 10:48 PM
Hi,
Nexus 1K does not support Spanning Tree Protocol. The 2nd port which you added should be part of the port-channel.
The Spanning tree information below is that from N1K ? AFAIK we dont have this option.
The Port-channel if connected to the same Upstream switch then it should not have the CDP subgroup option. It can be either on or active or passive mode only.
11-04-2009 11:00 PM
+ Post the Interface configuration for each 5020 port where the CNA is attached.
Have you tried manually assigning the sub-group IDs?
As a test, for "VEM 3" for example
With both interfaces uplinked to the 1000v do the following:
int eth3/1
sub-group-id 0
int eth3/2
subgroup-id 1
+ After you do this post the output of "show int brief".
Thanks,
Robert
11-05-2009 07:34 AM
Thanks guys. I will get back to you once I'm able to relay from the customer, I appreciate the help!
-Kenny
12-03-2009 10:44 AM
Have you resolved this? I'm having a very similar issue with vPC-HM and would love to see what your fix was.
02-05-2010 08:49 PM
I experienced a similar problem which may be related. There is a current bug where multicast will publish MACs through a non-pinned subgroup. The current work around is to disable igmp snooping. Try disabling it globally. If your issue goes away, you may be hitting bug CSCte44240.
02-08-2010 10:43 AM
yep, i was hitting that bug too. unfortunately there is also a bug with disabling igmp snooping. We found that disabling it, whether globally or for specific vlans did not actually disable igmp snooping in all cases. cisco confirmed there is a bug. the workaround was basically to enable/disable it several times and force a system switchover (presuming you have a redundant VSM) until it starts working. I belive the bug has been fixed, but there has not been a patch/upgrade release date yet
02-09-2010 05:36 AM
That doesn't sound fun. Ours seemed to disable without issue. Do you happen to be running 1000v in production? We are but only with low impact servers for now.
02-09-2010 08:50 AM
no, this was in our lab where our company's software gets tested before it moves to production. unreliable multicast is a deal breaker for our application so i've had to migrate off the 1000v for the time being until its fixed.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide