07-25-2011 08:40 AM - edited 03-07-2019 01:23 AM
We tried today to add a third member to a C3750 stack (WS-C3750G-24T, 12.2(25)SEE2,C3750-IPBASE-M) but we forgot to poweroff the new member before to add the new stackwise cable. The result was that all ports of the 2nd member went down.
We unplugged the third Stackwise cable and powered off the third switch. Then rebooted the 2nd one.
But it is blocking at "Waiting for port download".
The state on maste is :
Switch# Role Mac Address Priority State
--------------------------------------------------------
*1 Master 001b.2b20.4080 15 Ready
2 Member 0000.0000.0000 0 Provisioned
After a long time, this 2nd member went to "Initializing" state and hung there but all the systems plugged on the stack became unreachable, even on master member !!
The master connexions recovered just after power off of the 2nd member.
We have plugged the most important connexions of 2nd member to a DRP switch, in wait for resolution of the problem.
But we are becoming crazy now and don't know what to do now.
Can anybody help ?
sw1#sh platform stack-manager all
Current
Switch# Role Mac Address Priority State
--------------------------------------------------------
*1 Master 001b.2b20.4080 15 Ready
2 Member 0000.0000.0000 0 Provisioned
Stack Port Status Neighbors
Switch# Port 1 Port 2 Port 1 Port 2
--------------------------------------------------------
1 Down Down None None
Stack Discovery Protocol View
==============================================================
Switch Active Role Current Sequence Dirty
Number State Number Bit
--------------------------------------------------------------------
1 TRUE Master Ready 001 FALSE
Stack State Machine View
==============================================================
Switch Master/ Mac Address Version Uptime Current
Number Member (maj.min) State
---------------------------------------------------------------------
1 Master 001b.2b20.4080 1.14 263308 Ready
Last Conflict Parameters
Switch Master/ Prio Default Image Uptime Mac Address
Number Member Config Type
-----------------------------------------------------------------------
1 Master 15 No 3 263051 001b.2b20.4080
2 Member 14 No 3 00000 001c.0e86.9080
Stack Discovery Protocol Counters
Messages Sent Messages Recvd
UP DOWN UP DOWN
--------------------------------------------------------
*1: 0126787969 0000008100 0126755494 0000008097
2: 0000000000 0000000000 0000000000 0000000000
3: 0000000000 0000000000 0000000000 0000000000
4: 0000000000 0000000000 0000000000 0000000000
5: 0000000000 0000000000 0000000000 0000000000
6: 0000000000 0000000000 0000000000 0000000000
7: 0000000000 0000000000 0000000000 0000000000
8: 0000000000 0000000000 0000000000 0000000000
9: 0000000000 0000000000 0000000000 0000000000
Stack Changes: 70
Misc Counters
Counter Up Down
---------------------------------------------------
Wrong Ver Number: Send: 0000000000 0000000000
Wrong Ver Number: Recv: 0000000000 0000000000
Missed Messages: 0000032730 0000000000
Orphaned Messages 0000000000 0000000000
Supressed Messages 0000002366 0000000000
No Available Messages 0252953259 0000015991
Link Present 0000000043 0000000001
Link Not Present 0000010433 0128685068
Link RxReset 0000104229 1286850671
Duplicates: 0000001928 0000000000
Switch Number of last duplicate: 2
Sequence Number Failures: 0000000000
Switch Number of last Failure: 256 Last Difference 0
Reciprocal Efficiency Changes: Upgrade 0 Downgrade 0
Resource Counters
-------------------------------------------
Chunk Alloc's 0001311644
Chunk Free's 0001311644
Enqueue Failures: 0000000000
Null Queue Failures: 0000000000
Chunk Alloc Errors: 0000000000
Stack State Machine Counters
Messages Sent Messages Recvd
-------------------------------------------------
*1: 0000000000 0000000000
2: 0000000000 0000000000
3: 0000000000 0000000000
4: 0000000000 0000000000
5: 0000000000 0000000000
6: 0000000000 0000000000
7: 0000000000 0000000000
8: 0000000000 0000000000
9: 0000000000 0000000000
07-25-2011 09:48 AM
hi, that is bummer! it seems the master cannot communicate to the secondary switch now. it looks like the switchfabric is stuck in the 1st few phases of election process. how is your switch stack cabled at this time ? can post the running configs of the pertitannt (misspelled) information?
07-26-2011 11:57 PM
Hi Chris.
That you for your answer. But does mean "pertitannt" ?
The config of master is :
#####################
version 12.2
no service pad
service timestamps debug datetime localtime
service timestamps log datetime localtime
service password-encryption
!
hostname sw1
!
enable secret 5 $1$GV4T$r30a38pYTJIwPlw2jGqFj/
!
username Master1 password 7 15350E1816256875
no aaa new-model
clock timezone CET 1
clock summer-time CEST recurring last Sun Mar 2:00 last Sun Oct 3:00
switch 1 provision ws-c3750g-24t
switch 2 provision ws-c3750g-24t
ip subnet-zero
!
!
!
!
errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause sfp-config-mismatch
errdisable recovery cause gbic-invalid
errdisable recovery cause l2ptguard
errdisable recovery cause psecure-violation
errdisable recovery cause dhcp-rate-limit
errdisable recovery cause unicast-flood
errdisable recovery cause vmps
errdisable recovery cause storm-control
errdisable recovery cause arp-inspection
errdisable recovery cause loopback
errdisable recovery interval 120
no file verify auto
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
spanning-tree vlan 1-4094 priority 24576
!
vlan internal allocation policy ascending
!
interface Port-channel1
description VERS_SW10
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface Port-channel2
description VERS_GROUPE
switchport access vlan 6
switchport mode access
!
interface Port-channel3
description VERS_SW2
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/1
description SW10
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet1/0/2
description ESX6_MNGT_VMOTION_vmnic3
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/3
description VPN1-7
switchport access vlan 7
switchport mode access
!
interface GigabitEthernet1/0/4
description VPN1-212
switchport access vlan 212
switchport mode access
!
interface GigabitEthernet1/0/5
description FW1-ETH2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-3,5,10
switchport mode trunk
!
interface GigabitEthernet1/0/6
description FW1-ETH3
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 4,6-8
switchport mode trunk
!
interface GigabitEthernet1/0/7
description FW1-ETH4
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 212,217
switchport mode trunk
!
interface GigabitEthernet1/0/8
description SW2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 3 mode on
!
interface GigabitEthernet1/0/9
description ESX3_MNGT_VMOTION_vmnic3
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/10
description ESX8_VM_vmnic0
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/11
description ESX4_MNGT_VMOTION_vmnic3
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/12
description S-EON1
switchport access vlan 2
switchport mode access
!
interface GigabitEthernet1/0/13
description ESX3_VM_vmnic0
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/14
description ESX8_MNGT_VMOTION_vmnic3
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/15
description VPN3-Interne
switchport access vlan 7
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet1/0/16
description VPN3-ISP
switchport access vlan 217
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet1/0/17
description ESX4_VM_vmnic0
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/18
description ESX4_VM_vmnic2
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/19
description ESX6_VM_vmnic0
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/20
description ESX6_VM_vmnic2
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/21
description ESX5_VM_vmnic0
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/22
description ESX7_VM_vmnic2
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet1/0/23
description GROUPE
switchport access vlan 6
switchport mode access
channel-group 2 mode on
!
interface GigabitEthernet1/0/24
description ISP_Completel
switchport access vlan 212
switchport mode access
speed 100
duplex full
!
interface GigabitEthernet2/0/1
description SW10
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet2/0/2
description ESX7_MNGT_VMOTION_vmnic3
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/3
description VPN2-7
switchport access vlan 7
switchport mode access
!
interface GigabitEthernet2/0/4
description VPN2-212
switchport access vlan 212
switchport mode access
!
interface GigabitEthernet2/0/5
description FW2-ETH2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-3,5,8,10
switchport mode trunk
!
interface GigabitEthernet2/0/6
description FW2-ETH3
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 4,6-8
switchport mode trunk
!
interface GigabitEthernet2/0/7
description FW2-ETH4
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 212,217
switchport mode trunk
!
interface GigabitEthernet2/0/8
description SW2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 3 mode on
!
interface GigabitEthernet2/0/9
description ESX3_MNGT_VMOTION_vmnic5
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/10
description ESX8_VM_vmnic2
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/11
description ESX4_MNGT_VMOTION_vmnic5
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/12
description ESX8_MNGT_VMOTION_vmnic5
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/13
description ESX6_MNGT_VMOTION_vmnic5
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/14
description ESX7_MNGT_VMOTION_vmnic5
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/15
description CHDAX-RT
switchport access vlan 8
switchport mode access
!
interface GigabitEthernet2/0/16
description VERS_SW11
switchport trunk encapsulation dot1q
!
interface GigabitEthernet2/0/17
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/18
description ESX3_VM_vmnic2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-3,5
switchport mode trunk
!
interface GigabitEthernet2/0/19
description ESX7_VM_vmnic0
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface GigabitEthernet2/0/20
switchport access vlan 2
switchport mode access
shutdown
spanning-tree portfast
!
interface GigabitEthernet2/0/21
switchport access vlan 2
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet2/0/22
description SIM-ETH1
switchport access vlan 2
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet2/0/23
description GROUPE
switchport access vlan 6
switchport mode access
channel-group 2 mode on
!
interface GigabitEthernet2/0/24
description ISP_OBS
switchport access vlan 217
switchport mode access
spanning-tree portfast
!
interface Vlan1
no ip address
shutdown
!
interface Vlan2
ip address 172.27.0.40 255.255.255.0
!
ip default-gateway 172.27.0.1
ip classless
no ip http server
!
logging trap debugging
logging facility local5
logging 172.27.0.9
access-list 10 permit 172.27.0.0 0.0.1.255 log
snmp-server group eon v3 auth notify *tv.FFFFFFFF.FFFFFFFF.FFFFFFFF.FFFFFFFF0F
snmp-server group monitoring v3 auth
snmp-server community C0u5cou5! RO
snmp-server location RMS Grenoble
snmp-server contact RMS,Admin
snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
snmp-server enable traps vlandelete
snmp-server enable traps envmon fan shutdown supply temperature status
snmp-server host 172.27.0.65 version 3 auth eon
!
control-plane
!
banner motd ^C
SW1 ^C
!
line con 0
login local
line vty 0 4
access-class 10 in
login local
line vty 5 15
login local
!
!
monitor session 1 source interface Gi1/0/5 - 6
ntp clock-period 36029833
ntp server 172.27.0.25
ntp server 172.27.0.27
end
07-28-2011 07:13 AM
Hi.
We solved the problem by a reboot of the whole stack.
Easy solution but not always easy to use when the stack is very critical.
07-28-2011 10:53 AM
Nice....Thanks for prompt solution response :-)
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide