3750 switch locking up issue

c.walsh · ‎09-01-2011

We have a customer with a stack of 2 x 3750x-24P switch's running IOS 12.2(58)SE1 feature set C3750E-IPBASEK9-M.

They report that all devices lose connectivity & that the only way to resolve the issue is to reboot the stack.

Has anyone seen this before, is it possibly a bug with the IOS.

Colin

IAN WHITMORE · ‎09-01-2011

It could be...or it could be something not configured correctly on the switches.

Would be good to see the config. It could be spanning-tree of course if it's not configured correctly or a number of other things like broadcast storms etc.

We would need to see logs, configs etc to have a better indication. Provide us with what you can.

HTH,

Ian

c.walsh · ‎09-01-2011

The logs were cleared when the switch was rebooted.

config...

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

no service password-encryption

!

hostname XXXXXXX

!

boot-start-marker

boot-end-marker

!

enable secret XXXXXXXX

!

username xxx privilege 15 password 0 yyyyyy

no aaa new-model

switch 1 provision ws-c3750x-24p

switch 2 provision ws-c3750x-24p

system mtu routing 1500

ip routing

ip dhcp excluded-address 192.168.250.129

ip dhcp excluded-address 192.168.250.254

ip dhcp excluded-address 192.168.250.1 192.168.250.4

ip dhcp excluded-address 192.168.250.253

ip dhcp excluded-address 192.168.250.252

ip dhcp excluded-address 192.168.250.200

!

ip dhcp pool Voice

network 192.168.250.0 255.255.255.128

default-router 192.168.250.1

option 150 ip 192.168.210.1 192.168.34.2

domain-name xxxxx.local

dns-server 192.168.100.140 192.168.100.142

!

ip dhcp pool Data

network 192.168.250.128 255.255.255.128

default-router 192.168.250.252

domain-name xxxxx.local

dns-server 192.168.100.140 192.168.100.142

!

vtp mode transparent

!

mls qos map policed-dscp 0 10 18 24 46 to 8

mls qos map cos-dscp 0 8 16 24 32 46 48 56

mls qos srr-queue input bandwidth 70 30

mls qos srr-queue input threshold 1 80 90

mls qos srr-queue input priority-queue 2 bandwidth 30

mls qos srr-queue input cos-map queue 1 threshold 2 3

mls qos srr-queue input cos-map queue 1 threshold 3 6 7

mls qos srr-queue input cos-map queue 2 threshold 1 4

mls qos srr-queue input dscp-map queue 1 threshold 2 24

mls qos srr-queue input dscp-map queue 1 threshold 3 48 49 50 51 52 53 54 55

mls qos srr-queue input dscp-map queue 1 threshold 3 56 57 58 59 60 61 62 63

mls qos srr-queue input dscp-map queue 2 threshold 3 32 33 40 41 42 43 44 45

mls qos srr-queue input dscp-map queue 2 threshold 3 46 47

mls qos srr-queue output cos-map queue 1 threshold 3 4 5

mls qos srr-queue output cos-map queue 2 threshold 1 2

mls qos srr-queue output cos-map queue 2 threshold 2 3

mls qos srr-queue output cos-map queue 2 threshold 3 6 7

mls qos srr-queue output cos-map queue 3 threshold 3 0

mls qos srr-queue output cos-map queue 4 threshold 3 1

mls qos srr-queue output dscp-map queue 1 threshold 3 32 33 40 41 42 43 44 45

mls qos srr-queue output dscp-map queue 1 threshold 3 46 47

mls qos srr-queue output dscp-map queue 2 threshold 1 16 17 18 19 20 21 22 23

mls qos srr-queue output dscp-map queue 2 threshold 1 26 27 28 29 30 31 34 35

mls qos srr-queue output dscp-map queue 2 threshold 1 36 37 38 39

mls qos srr-queue output dscp-map queue 2 threshold 2 24

mls qos srr-queue output dscp-map queue 2 threshold 3 48 49 50 51 52 53 54 55

mls qos srr-queue output dscp-map queue 2 threshold 3 56 57 58 59 60 61 62 63

mls qos srr-queue output dscp-map queue 3 threshold 3 0 1 2 3 4 5 6 7

mls qos srr-queue output dscp-map queue 4 threshold 1 8 9 11 13 15

mls qos srr-queue output dscp-map queue 4 threshold 2 10 12 14

mls qos queue-set output 1 threshold 1 100 100 50 200

mls qos queue-set output 1 threshold 2 125 125 100 400

mls qos queue-set output 1 threshold 3 100 100 100 400

mls qos queue-set output 1 threshold 4 60 150 50 200

mls qos queue-set output 1 buffers 15 25 40 20

mls qos

!

crypto pki trustpoint TP-self-signed-4270496768

enrollment selfsigned

subject-name cn=IOS-Self-Signed-Certificate-4270496768

revocation-check none

rsakeypair TP-self-signed-4270496768

!

crypto pki certificate chain TP-self-signed-4270496768

certificate self-signed 01

30820244 308201AD A0030201 02020101 300D0609 2A864886 F70D0101 04050030

31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274

69666963 6174652D 34323730 34393637 3638301E 170D3933 30333031 30303031

32375A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649

4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D34 32373034

39363736 3830819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281

8100CFA6 E52A650C 84ED05B0 DF9136ED 5230380A 78C7BAB9 DAB3D93B DDF2040F

82D34E4F A6FD7190 864DFEEA 0B6DE6DC 5ACA198B 1BB71B8C 5C66CD1C A59D87EA

487CA6CC E569914E 1F848137 72E53EFF A5185518 696E7B89 136088E5 6E730429

9F1CD323 3F85DBAA C077E078 C4AFB052 3D96EE92 1ADB861C 002886A3 D953CAE9

AD3F0203 010001A3 6C306A30 0F060355 1D130101 FF040530 030101FF 30170603

551D1104 10300E82 0C52494E 5F333735 305F5357 49301F06 03551D23 04183016

8014A289 9272F054 4DF7E329 C4D5AB0A 5902F9AA 7B5C301D 0603551D 0E041604

14A28992 72F0544D F7E329C4 D5AB0A59 02F9AA7B 5C300D06 092A8648 86F70D01

01040500 03818100 93A14230 BD779E81 866CF43A D8437886 5A1B04D9 13DFF432

82F2EF6E F6A6326D 2DEAB806 DD9FBE8D 5DDE60D4 AC1E28B0 E9249567 F292EA51

A54871C0 31B75CB8 0DBFD687 B2624BDC F15694A0 7B80C68D B22B960D F9FCA506

AC11D1E2 5253CA07 F5ED85BA A80E3CFA 380238B8 87B58C78 CD11DDDE BE2C1BE4

BBC65238 0B2FDDCF

quit

license boot level ipbase

license boot level ipbase switch 1

spanning-tree mode pvst

spanning-tree extend system-id

auto qos srnd4

!

vlan internal allocation policy ascending

!

vlan 2

name Voice

!

vlan 10

!

class-map match-all AUTOQOS_VOIP_DATA_CLASS

match ip dscp ef

class-map match-all AUTOQOS_DEFAULT_CLASS

match access-group name AUTOQOS-ACL-DEFAULT

class-map match-all AUTOQOS_VOIP_SIGNAL_CLASS

match ip dscp cs3

!

policy-map AUTOQOS-SRND4-CISCOPHONE-POLICY

class AUTOQOS_VOIP_DATA_CLASS

set dscp ef

police 128000 8000 exceed-action policed-dscp-transmit

class AUTOQOS_VOIP_SIGNAL_CLASS

set dscp cs3

police 32000 8000 exceed-action policed-dscp-transmit

class AUTOQOS_DEFAULT_CLASS

set dscp default

police 10000000 8000 exceed-action policed-dscp-transmit

!

interface FastEthernet0

no ip address

no ip route-cache cef

no ip route-cache

shutdown

!

interface GigabitEthernet1/0/1

description *** Video Conferencing ***

switchport mode access

duplex full

srr-queue bandwidth share 1 30 35 5

srr-queue bandwidth shape 10 0 0 0

priority-queue out

mls qos trust cos

auto qos trust

!

interface GigabitEthernet1/0/23

description *** MPLS WAN ***

switchport access vlan 10

srr-queue bandwidth share 1 30 35 5

srr-queue bandwidth shape 10 0 0 0

priority-queue out

mls qos trust cos

auto qos trust

!

interface GigabitEthernet1/0/24

description *** DSL RTR Fallback ***

!

interface GigabitEthernet2/0/1

description Link to Voice Gateway

switchport access vlan 2

mls qos trust dscp

!

interface GigabitEthernet2/0/2

switchport voice vlan 2

srr-queue bandwidth share 1 30 35 5

priority-queue out

mls qos trust device cisco-phone

mls qos trust cos

auto qos voip cisco-phone

spanning-tree portfast

service-policy input AUTOQOS-SRND4-CISCOPHONE-POLICY

!

all other ports configured the same as g2/0/2

!

interface Vlan1

ip address 192.168.250.252 255.255.255.128

!

interface Vlan2

description Voice

ip address 192.168.250.1 255.255.255.128

!

interface Vlan10

description *** BT MPLS ***

ip address 192.168.6.5 255.255.255.248

!

ip http server

ip http secure-server

!

ip route 0.0.0.0 0.0.0.0 192.168.6.1

!

ip access-list extended AUTOQOS-ACL-DEFAULT

permit ip any any

!

logging esm config

!

line con 0

line vty 0 4

password xxxxxx

login local

length 0

line vty 5 15

login

!

IAN WHITMORE · ‎09-01-2011

Can't see anything obvious in the config.

Checked the bugs as well for this model/version and there are only 43, none relating to this.

Can you provide also:

#sh spann

#sh int status

#sh int trunk

#sh cdp nei

and before the next reload, try and get a look at the logs. Also, do they say how often this occurs?

Might be best to put it all in a text file and attach.

Regards,

Ian

ankugarg · ‎09-01-2011

Is this happening consistently or happened only once...Also try to get the output of "sh logging",i addition to what Ian pointed.

Somasundaram Jayaraman · ‎09-01-2011

Hi,

Can you paste the output of show spanning-tree detail | i ieee|from|occur|is exec?

This ouptut will indicate from where we received the TCN's.

What is the cpu utlization of the switch?

++ Enable mac flap notification, so that in case of any loops in the network, we can figure it out.

conf t

mac address-table notification mac-move.

Hope this helps

Cheers

Somu

Rate helpful posts

Somasundaram Jayaraman · ‎09-01-2011

Hi,

Here is the link for troubleshooting common issues in 3750 switch.

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807ccc79.shtml

Hope this helps

Cheers

Somu

Rate helpful posts

Leo Laohoo · ‎09-01-2011

They report that all devices lose connectivity & that the only way to resolve the issue is to reboot the stack.

Have you tried shut/no shut all the ports?