Cisco AIR-LAP1142N-A-K9 keeps rebooting / loop

[greg.mermuys]

Hello,

I have a Cisco AIR-LAP1142N-A-K9 that keeps rebooting, I've done the factory reset (hold the mode button for 20-30 seconds until LED is red). 100+ other APs are up and running fine. Any ideas?

Thanks!

Greg

From console screen when booting the faulty AP:

...

flashfs[0]: 28 files, 9 directories

flashfs[0]: 0 orphaned files, 0 orphaned directories

flashfs[0]: Total bytes: 32385024

flashfs[0]: Bytes used: 7465472

flashfs[0]: Bytes available: 24919552

flashfs[0]: flashfs fsck took 19 seconds.

Reading cookie from system serial eeprom...Done

Base Ethernet MAC address: 70:81:05:03:81:30

Ethernet speed is 1000 Mb - FULL duplex

Loading "flash:/c1140-k9w8-mx.124-25e.JA1/c1140-k9w8-mx.124-25e.JA1"...################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################

File "flash:/c1140-k9w8-mx.124-25e.JA1/c1140-k9w8-mx.124-25e.JA1" uncompressed and installed, entry point: 0x4000

executing...

enet halted

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is

subject to restrictions as set forth in subparagraph

(c) of the Commercial Computer Software - Restricted

Rights clause at FAR sec. 52.227-19 and subparagraph

(c) (1) (ii) of the Rights in Technical Data and Computer

Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.

           170 West Tasman Drive

           San Jose, California 95134-1706

Cisco IOS Software, C1140 Software (C1140-K9W8-M), Version 12.4(25e)JA1, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2012 by Cisco Systems, Inc.

Compiled Wed 30-May-12 04:06 by prod_rel_team

Proceeding with system init

Proceeding to unmask interrupts

Initializing flashfs...

FLASH CHIP:  Numonyx P33

Checking for Over Erased blocks

......................................................................................................................................................................................................................................................

flashfs[2]: 28 files, 9 directories

flashfs[2]: 0 orphaned files, 0 orphaned directories

flashfs[2]: Total bytes: 32126976

flashfs[2]: Bytes used: 7465472

flashfs[2]: Bytes available: 24661504

flashfs[2]: flashfs fsck took 5 seconds.

flashfs[2]: Initialization complete.

flashfs[3]: 0 files, 1 directories

flashfs[3]: 0 orphaned files, 0 orphaned directories

flashfs[3]: Total bytes: 11999232

flashfs[3]: Bytes used: 1024

flashfs[3]: Bytes available: 11998208

flashfs[3]: flashfs fsck took 0 seconds.

flashfs[3]: Initialization complete....done Initializing flashfs.

00:00:08 UTC Fri Mar 1 2002: Unexpected exception to CPUvector 200, PC = 0x51C7B8  , LR = 0x51D228

-Traceback= 0x51C7B8 0x51D228 0x51D90C 0x509978 0x509D80 0x506AB8 0x191790 0x191844 0x57164 0x5746C 0x1A6410

CPU Register Context:

MSR = 0x00029230  CR  = 0x24000028  CTR = 0x005098AC  XER   = 0x00000000

R0  = 0x9400AE16  R1  = 0x013987B0  R2  = 0x00000000  R3    = 0x013988D0

R4  = 0x00000008  R5  = 0x013988E0  R6  = 0x00000000  R7    = 0x013988C8

R8  = 0x4B233FAE  R9  = 0x00000000  R10 = 0xBD912F2B  R11   = 0x009F845A

R12 = 0xB87D0700  R13 = 0x01000000  R14 = 0x00000000  R15   = 0x00000000

R16 = 0x00000000  R17 = 0x00000000  R18 = 0x00000000  R19   = 0x00000000

R20 = 0x00000000  R21 = 0x00000000  R22 = 0x00000000  R23   = 0x00000000

R24 = 0x00000000  R25 = 0x00000000  R26 = 0x00000000  R27   = 0x00C51FB8

R28 = 0x013988E0  R29 = 0x00000000  R30 = 0x013988D8  R31   = 0x013988D0

=== Flushing messages (00:00:09 UTC Fri Mar 1 2002) ===

Buffered messages:

Queued messages:

*Mar  1 00:00:09.428: %SYS-3-LOGGER_FLUSHING: System pausing to ensure console debugging output.

*Mar  1 00:00:09.302: %SYS-4-PUPDATECLOCK: Periodic Clock update with ROMMON failed, because size left in ROMMON (4294967295), size needed (29), error code (-1)

Nested exception_rom_monitor call (2 times)

Unexpected exception to CPUvector 200, PC = 51C7B8

-Traceback= 0x51C7B8 0x51D228 0x51D90C 0x509978 0x509D80 0x506AB8 0x191790 0x191844 0x57164 0x5746C 0x1A6410

Exception (200)! Machine Check

ESR: 0x00000000, BESRA: 0x00000000, BESRB: 0x00000000, BEAR: 0xFFFFFFFF

MSR: 0x00020030, SRR2: x002B31B0,   SRR3: 0x00020030

CPU Register Context:

Vector = 0x00000200  PC =  0x0051C7B8  MSR = 0x00029230  CR =  0x24000028

LR =  0x0051D228     CTR = 0x005098AC  XER = 0x00000000

R0 =  0x9400AE16     R1 =  0x013987B0  R2 =  0x00000000  R3 =  0x013988D0

R4 =  0x00000008     R5 =  0x013988E0  R6 =  0x00000000  R7 =  0x013988C8

R8 =  0x4B233FAE     R9 =  0x00000000  R10 = 0xBD912F2B  R11 = 0x009F845A

R12 = 0xB87D0700     R13 = 0x01000000  R14 = 0x00000000  R15 = 0x00000000

R16 = 0x00000000     R17 = 0x00000000  R18 = 0x00000000  R19 = 0x00000000

R20 = 0x00000000     R21 = 0x00000000  R22 = 0x00000000  R23 = 0x00000000

R24 = 0x00000000     R25 = 0x00000000  R26 = 0x00000000  R27 = 0x00C51FB8

R28 = 0x013988E0     R29 = 0x00000000  R30 = 0x013988D8  R31 = 0x013988D0

Stack trace:

PC = 0x0051C7B8, SP = 0x013987B0

Frame 00: SP = 0x013987C0    PC = 0x00C51D2C

Frame 01: SP = 0x01398860    PC = 0x0051D228

Frame 02: SP = 0x01398868    PC = 0x0051D90C

Frame 03: SP = 0x013988E8    PC = 0x00509978

Frame 04: SP = 0x01398908    PC = 0x00509D80

Frame 05: SP = 0x01398910    PC = 0x00506AB8

Frame 06: SP = 0x01398918    PC = 0x00191790

Frame 07: SP = 0x01398938    PC = 0x00191844

Frame 08: SP = 0x01398958    PC = 0x00057164

Frame 09: SP = 0x01398A60    PC = 0x0005746C

Frame 10: SP = 0x00000000    PC = 0x001A6410

Cisco IOS Software, C1140 Software (C1140-K9W8-M), Version 12.4(25e)JA1, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2012 by Cisco Systems, Inc.

Compiled Wed 30-May-12 04:06 by prod_rel_team

ap uptime is 12 seconds

cisco AIR-LAP1142N-A-K9    (PowerPC405ex) processor (revision A0) with 65526K/65536K bytes of memory.

Processor board ID FGL1533S4NX

PowerPC405ex CPU at 586Mhz, revision number 0x147E

using  eeprom values

WRDTR,CLKTR: 0x85000800 0x40000000

RQDC ,RFDC : 0x80000035 0x0000020c

using ÿÿÿÿ ddr static values from serial eeprom

ddr init done

Running Normal Memtest...

Passed.

IOS Bootloader - Starting system.

FLASH CHIP:  Numonyx P33

Checking for Over Erased blocks

......................................................................................................................................................................................................................................................

Xmodem file system is available.

DDR values used from system serial eeprom.

WRDTR,CLKTR: 0x85000800, 0x40000000

RQDC, RFDC : 0x80000035, 0x0000020c

PCIE0: link is up.

PCIE0: VC0 is active

PCIE1: link is up.

PCIE1: VC0 is active

PCIEx: initialization done

flashfs[0]: 28 files, 9 directories

flashfs[0]: 0 orphaned files, 0 orphaned directories

flashfs[0]: Total bytes: 32385024

flashfs[0]: Bytes used: 7465472

flashfs[0]: Bytes available: 24919552

flashfs[0]: flashfs fsck took 19 seconds.

Reading cookie from system serial eeprom...Done

Base Ethernet MAC address: 70:81:05:03:81:30

Ethernet speed is 1000 Mb - FULL duplex

Loading "flash:/c1140-k9w8-mx.124-25e.JA1/c1140-k9w8-mx.124-25e.JA1"...#########################################################################################################################################################################################################################################################

....

[Leo Laohoo]

Unexpected exception to CPUvector 200, PC = 51C7B8
-Traceback= 0x51C7B8 0x51D228 0x51D90C 0x509978 0x509D80 0x506AB8 0x191790 0x191844 0x57164 0x5746C 0x1A6410

CPUvector 200.  On an AP.  Now this is an error message I've never seen before on an AP before. 

Can I ask you if you can do the following:

1.  Power down the AP;

2.  Hold the Mode button and keep holding it down;

3.  Power up the AP for about 30 seconds. 

I want to ask if you can get the AP into ROMmon.  Then post the output to the command "dir flash:". 

I've got a strange feeling your AP is going to a vicious cycle called crash-boot-loop.

Normally seen in routers and switches, when an appliance goes into a crash-boot-loop, the appliance crashes.  IOS protocol states that every time an appliance crashes, it is mandated to do a "dump" of the crash information into the flash of the appliance (filename is usually called "crashinfo").  However, if the flash memory is full and the appliance attempts to create a crashinfo file/subdirectory, would you believe, the appliance CRASHES again, hence it's a vicious cycle. 

[greg.mermuys]

Leo,

Thanks, here is the ouput of the dir flash: command. I don't see a "crashinfo" file anywhere and it looks like the flash file system has approx 24mb available.

Greg

ap:

ap: dir flash:

Directory of flash:/

2    -rwx  265                      lwapp_mobileconcierge.cfg

3    -rwx  6706                     event.log

4    -rwx  6168                     private-multiple-fs

38   drwx  384                      c1140-k9w8-mx.124-25e.JA1

6    drwx  0                        configs

7    drwx  128                      c1140-rcvk9w8-mx

5    -rwx  209                      env_vars

24919552 bytes available (7465472 bytes used)

ap:

[Scott Fella]

Try one of these methods

debug lwapp console cli
debug capwap console cli
erase /all nvram

*******************************************************
Example:

Entered the username and password
debug capwap console cli
erase /all nvram:
[confirm]
undebug all
reload
[confirm]

*******************************************************
Hold reset for over 30 seconds to get to ROMMON

ap:delete flash:private-config
ap:delete flash:private-multiple-fs
ap:boot

Your AP will be factory default


Sent from Cisco Technical Support iPhone App

[greg.mermuys]

Thanks Scott,  I've tried this but after booting, it will still crash.

I've managed to delete the "c1140-k9w8-mx.124-25e.JA1" image that the AP downloads from the controller and just boot off the factory installed "c1140-rcvk9w8-mx" image. This works and the AP is up and running without the boot/crash/loop but of course as soon as I plug in the AP to the network, it finds the controller through discovery and downloads the new image and the the boot/crash/loop appears again. Any more ideas?

Thanks,

Greg

[Leo Laohoo]

What firmware is your WLC running at?

[greg.mermuys]

WLC is running v7.2.110.0

[Scott Fella]

I would clear the flash and only have these files

6    drwx  0                        configs

7    drwx  128                      c1140-rcvk9w8-mx

5    -rwx  209                      env_vars

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

[greg.mermuys]

I have cleared all the files from flash except the two directories and the one file as mentioned above and rebooted.

AP connecteed to the controller via DHCP/discovery process and downloaded the "c1140-k9w8-mx.124-25e.JA1" image and rebooted. After reboot, the same thing... crashes and into boot/crash loop.

Greg

[Scott Fella · Accepted solution]

Might be that the AP is going bad.  You have 100+ 1142's and this one is causing you issues.  You have reset it, deleted the files from flash and you still see a crash during boot.  I would just replace that AP.

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

[greg.mermuys]

Thanks for your help everyone, I managed to get the AP replaced with another one and RMA the old faulty one.

[Leo Laohoo]

Go to 7.4.110.0 and make sure you upgrade the FUS code of the WLC.

[Saravanan Lakshmanan]

Couldn't fine a matching bug id, however have replaced the 1140 for the exact same issue.