Cisco 3640 Crash after 20 hours

[rom]

123

[dixho]

Hi,

Can you post the output of show version please? The attached crash.txt does not show software version and feature set of the 3640.

Regards

Dixon

[rom]

Cisco Internetwork Operating System Software
IOS (tm) 3600 Software (C3640-IS-M), Version 12.2(21b), RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Wed 31-Mar-04 14:19 by pwade
Image text-base: 0x60008930, data-base: 0x61136000

ROM: System Bootstrap, Version 11.1(15700) [ptong 105], INTERIM SOFTWARE

ramallo uptime is 10 hours, 6 minutes
System returned to ROM by reload
System restarted at 07:41:18 AST Mon Nov 8 2010
System image file is "slot0:c3640-is-mz.122-21b.bin"

cisco 3640 (R4700) processor (revision 0x00) with 123904K/7168K bytes of memory.
Processor board ID 03869202
R4700 CPU at 100Mhz, Implementation 33, Rev 1.0
Bridging software.
X.25 software, Version 3.0.0.
SuperLAT software (copyright 1990 by Meridian Technology Corp).
Primary Rate ISDN software, Version 1.1.
2 Ethernet/IEEE 802.3 interface(s)
1 FastEthernet/IEEE 802.3 interface(s)
4 Serial network interface(s)
1 Channelized T1/PRI port(s)
DRAM configuration is 64 bits wide with parity disabled.
125K bytes of non-volatile configuration memory.
4096K bytes of processor board System flash (Read/Write)
16384K bytes of processor board PCMCIA Slot0 flash (Read/Write)
16384K bytes of processor board PCMCIA Slot1 flash (Read/Write)

Configuration register is 0x3922

[Leo Laohoo]

%Software-forced reload

Unexpected exception, CPU signal 23, PC = 0x60419A44

I agree.  Could be your IOS.

[rom]

The most funny thing is that this router has been running flawleslly almost 1 year....but still it cou

ld be.

[rom]

Cisco Internetwork Operating System Software
IOS (tm) 3600 Software (C3640-IS-M), Version 12.2(21b), RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Wed 31-Mar-04 14:19 by pwade
Image text-base: 0x60008930, data-base: 0x61136000

ROM: System Bootstrap, Version 11.1(15700) [ptong 105], INTERIM SOFTWARE

ramallo uptime is 10 hours, 6 minutes
System returned to ROM by reload
System restarted at 07:41:18 AST Mon Nov 8 2010
System image file is "slot0:c3640-is-mz.122-21b.bin"

cisco 3640 (R4700) processor (revision 0x00) with 123904K/7168K bytes of memory.
Processor board ID 03869202
R4700 CPU at 100Mhz, Implementation 33, Rev 1.0
Bridging software.
X.25 software, Version 3.0.0.
SuperLAT software (copyright 1990 by Meridian Technology Corp).
Primary Rate ISDN software, Version 1.1.
2 Ethernet/IEEE 802.3 interface(s)
1 FastEthernet/IEEE 802.3 interface(s)
4 Serial network interface(s)
1 Channelized T1/PRI port(s)
DRAM configuration is 64 bits wide with parity disabled.
125K bytes of non-volatile configuration memory.
4096K bytes of processor board System flash (Read/Write)
16384K bytes of processor board PCMCIA Slot0 flash (Read/Write)
16384K bytes of processor board PCMCIA Slot1 flash (Read/Write)

Configuration register is 0x3922

[dixho]

i decode the stack. The stack points to a memory corruption issue. In order 100% confirm the bug, we need to get the core dump just before the crash.

http://www.cisco.com/en/US/products/sw/iosswrel/ps1835/products_tech_note09186a00800c7d59.shtml

From looking at the decode, it shows that the router tried to translate LDAP trafffic. It may be CSCec72958. I cannot find any memory corruption issue caused by NTP.

[Timothy Stewart]

I agree with Dixon.  The crash was due to a memory corruption.  The IOS attempted to free a block of memory that was in fact not really the start of a block.  This seems to have occurred when NAT was translating a LDAP packet that was sourced from the inside to the outside.  During the translation NAT attempted to free a memory block but the pointer was incorrect.  CSCec72958 is the most likely defect so upgrading past it should resolved your problem.  You may also be able to workaround the issue by setting up a filter for NAT so that LDAP packets are not translated.

Tim

[rom]

"You may also be able to workaround the issue by setting up a filter for NAT so that LDAP packets are not translated."

How I can set up this?

[rom]

Thanks.

I did the configuration for the Crash Dump.

Let see what happens.

Thanks Again.

[Timothy Stewart · Accepted solution]

The easiest way to filter ldap is to install a inbound acl on the nat inside interface:

Interface Gig 0/0

ip access-group 100 in

!

access-list 100 deny tcp any any eq 389

access-list 100 deny udp any any eq 389

access-list 100 permit ip any any

The core dump will not help in this case.  The partial crashinfo you provided was more than enough to identify the correct defect therefore there is no need for a core dump.

Tim