cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2815
Views
40
Helpful
14
Replies

Issue with 5520 Wireless Controller Software Crash

tanner.zaitt
Level 3
Level 3

Hello, I am in trouble with WLC Controller 5520.

The issue is related to reading the system info information from the ssh console.

For example, on Tuesday the WLC Controller is rebooted of a software crash reason.
When we log in a few minutes later and we check the uptime of the WLC with "show sysinfo" command
and there the uptime was okay and we ignore the mail notification: 
System Up Time................................... 19 days 21 hrs 4 mins 46 secs for TESTWLC01.

But two days later we check it again and the output is different!

show sysinfo
Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.10.121.0
RTOS Version..................................... 8.10.121.0
Bootloader Version............................... 8.3.15.177
Emergency Image Version.......................... 8.3.141.0

OUI File Last Update Time........................ Tue Feb 06 10:44:07 UTC 2018

Build Type....................................... DATA + WPS

System Name...................................... TESTWLC01
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.2170
Redundancy Mode.................................. SSO
IP Address....................................... 192.168.1.20
IPv6 Address..................................... ::
System Up Time................................... 1 days 19 hrs 36 mins 6 secs
System Timezone Location......................... (GMT +1:00) Amsterdam, Berlin, Rome, Vienna
System Stats Realtime Interval................... 5

--More-- or (q)uit
System Stats Normal Interval..................... 180

Configured Country............................... Multiple Countries : BG
Operating Environment............................ Commercial (10 to 35 C)
Internal Temp Alarm Limits....................... 10 to 38 C
Internal Temperature............................. +24 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 4
Number of Active Clients......................... 85

OUI Classification Failure Count................. 141798

Memory Current Usage............................. 11
Memory Average Usage............................. 11
CPU Current Usage................................ 0
CPU Average Usage................................ 0

Flash Type....................................... Compact Flash Card
Flash Size....................................... 1073741824



The second check is not like the first about uptime information.
An additional check with  "show logging last-reset" show us real information about real software crash event

The output is :

show logging last-reset

(Cisco Controller) >show logging last-reset

!!!Message and Trap Logs from Previous Reset

--------------------------------------------- Last Reboot MsgLog & Traplog ---------------------------------------------
Sys Name: TESTWLC01
Model: AIR-CT5520-K9
Version: 8.10.121.0
Primary Boot Image: 8.10.121.0 (default) (active)
Backup Boot Image: 8.5.131.0
LastReset Reason: Software Crash
Timestamp: Tue Oct 6 14:35:36 2020
SystemUpTime: 27 days 22 hrs 52 mins 45 secs

And now I am confused ...

Where I can see really the true uptime of device after Software crash?

Is it possible to see the incorrect output of uptime in these two commands:
"show sysinfo" and "show logging last-reset" ?

And one last question is it possible to see logs only about software crash and reasons for these type issues? 
And if a reboot is initiated from a specific user is possible to figure out the right user? 

Thank you in advance!

Best regards!



1 Accepted Solution

Accepted Solutions

You are probably hitting a bug which is solved in the newer release 8.10.130.0.

Release notes:

https://www.cisco.com/c/en/us/td/docs/wireless/controller/release/notes/crn810mr3.html#resolved-caveats

Bug you probably hit:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvu23186

 

So I suggest upgrading to the newer release which should fix this bug. 

View solution in original post

14 Replies 14

Leo Laohoo
Hall of Fame
Hall of Fame
sh coredump summary

Hi. thank you, could you tell me where right we should check in this coredump files?
How is possible to read  the files?
I downloaded them and extracted them in Windows machine.

tanner.zaitt
Level 3
Level 3

The output is :


(Cisco Controller) >show coredump summary

Core Dump upload is disabled

Core Dump file on flash:

SW Version....................................... 8.10.121.0
Time Stamp....................................... Tue Oct 6 14:35:36 2020
File Size........................................ 68229995
File Name Suffix................................. _8.10.121.0_WLCTEST01_1601987736.gz


How is possible to read files in _8.10.121.0_WLCTEST01_1601987736.gz?

To minimize the time for TAC to be waiting for outputs, please prepare the following: 

  1. Controller Bundle
  2. sh tech
  3. sh config
  4. Core dump file

Send those to TAC so they can analyze the cause of the crash.

We couldn't analyze the situation alone?

 

 https://www.cisco.com/c/en/us/td/docs/wireless/controller/8-0/configuration-guide/b_cg80/b_cg80_chapter_0110110.html#upload-logs

 M.



-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
    When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

I get the files, but I can't read them

 

 - You mean unpacking .gz files ? Winzip can do that, if you are on linux use % gzip -d filename.

 M.



-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
    When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

I tried with Windows 10 machine and 7 zip and the files are numbers from 1 to 670.
With Notepad ++ and they are not readable.


With show tech I find only :

Reset reason : reset due to switch-driver crash ,(Version: 8.10.121.0)
Reset time : Tue Oct 6 14:41:31 2020
Trace back : 0x8f0454 0x1e06dea 0x1e07f46 0x8e8c85 0xf170 0x1366c4 0x129ef26 0x129efc2 0x129f43d 0x113b51d 0x113be29 0xf23bcc 0x236ef32 0x1e0fb97 0x7dff 0xe898d
Number of files present is: 9


 

 - In any case is the info from the crash-files destined for specialized analysis by cisco tac.

 M.



-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
    When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

Thank you but it is a little complicated to connect to Cisco TAC team, the WLC is under the support of Cisco, but I am not part of the local infrastructure, just I am part of a different support team with monitoring functions and I can't initiate a case in TAC.

Thanks a lot for your help!
I appreciate it.
Best regards!

You are probably hitting a bug which is solved in the newer release 8.10.130.0.

Release notes:

https://www.cisco.com/c/en/us/td/docs/wireless/controller/release/notes/crn810mr3.html#resolved-caveats

Bug you probably hit:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvu23186

 

So I suggest upgrading to the newer release which should fix this bug. 

Hi, thank you!

Yes, this will solve the issue!

The answer to my question is right CSCvu23186.

Best regards!



tanner.zaitt
Level 3
Level 3

Hi,

We found and new bug:

CSCvs49779
https://quickview.cloudapps.cisco.com/quickview/bug/CSCvs49779

Could you share information  about the upgrade procedure of  3x5520 WLC controllers configured with High Availability (SSO)?

Review Cisco Networking for a $25 gift card