cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
8501
Views
0
Helpful
5
Replies

Crash Errors and Traceback condition Switch 3750 Stack

Luis Salazar
Level 1
Level 1

Hi,

Hoping someone can help me make some sense of this:

EEHYP_CS_801#sh logging

Syslog logging: enabled (0 messages dropped, 0 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.

No Inactive Message Discriminator.

    Console logging: disabled

    Monitor logging: level debugging, 236 messages logged, xml disabled,

                     filtering disabled

    Buffer logging:  level debugging, 736199 messages logged, xml disabled,

                     filtering disabled

    Exception Logging: size (4096 bytes)

    Count and timestamp logging messages: disabled

    File logging: file flash:mylog,

        max size 4096, min size 0,

        level critical, 418 messages logged

    Persistent logging: disabled

No active filter modules.

    Trap logging: level informational, 776947 message lines logged

        Logging to 172.21.100.250  (udp port 514,  audit disabled,

              authentication disabled, encryption disabled, link up),

              776878 message lines logged,

              0 message lines rate-limited,

              0 message lines dropped-by-MD,

              xml disabled, sequence number disabled

              filtering disabled

        Logging to 10.64.100.3  (udp port 514,  audit disabled,

              authentication disabled, encryption disabled, link up),

              776878 message lines logged,

              0 message lines rate-limited,

              0 message lines dropped-by-MD,

              xml disabled, sequence number disabled

              filtering disabled

        Logging to 10.161.100.246  (udp port 514,  audit disabled,

              authentication disabled, encryption disabled, link up),

              776878 message lines logged,

              0 message lines rate-limited,

              0 message lines dropped-by-MD,

              xml disabled, sequence number disabled

              filtering disabled

        Logging to 172.21.100.69  (udp port 514,  audit disabled,

              authentication disabled, encryption disabled, link up),

              776878 message lines logged,

              0 message lines rate-limited,

              0 message lines dropped-by-MD,

              xml disabled, sequence number disabled

              filtering disabled

        Logging to 172.21.100.66  (udp port 514,  audit disabled,

              authentication disabled, encryption disabled, link up),

              776880 message lines logged,

              0 message lines rate-limited,

              0 message lines dropped-by-MD,

              xml disabled, sequence number disabled

              filtering disabled

        Logging to 10.161.100.111  (udp port 514,  audit disabled,

              authentication disabled, encryption disabled, link up),

              776880 message lines logged,

              0 message lines rate-limited,

              0 message lines dropped-by-MD,

              xml disabled, sequence number disabled

              filtering disabled

Log Buffer (4096 bytes):

64EF8 (EEHYP_CS_801-1)

Jun 28 11:56:50: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:56:55: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:00: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:05: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:10: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:15: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:20: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:25: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:30: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:35: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:40: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:45: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:50: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:57:55: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:00: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:05: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:10: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:15: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:20: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:25: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

Jun 28 11:58:30: %TCAMMGR-3-MOVE_ERROR: cam entry move from index 1063 to index 1185 failed (EEHYP_CS_801-1)

-Traceback= D667B8 D66F04 41463C 40FFF8 411834 423A6C A6E428 A64EF8 (EEHYP_CS_801-1)

EEHYP_CS_801#

1. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED (EEHYP_CS_801-1) *

2. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED (EEHYP_CS_801-1) *

3. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED (EEHYP_CS_801-1) *

4. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED (EEHYP_CS_801-1) *

5. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED System previously crashed with the following message: (EEHYP_CS_801-1) *

6. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(50)SE1, RELEASE SOFTWARE (fc2) (EEHYP_CS_801-1) *

7. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Copyright (c) 1986-2009 by Cisco Systems, Inc. (EEHYP_CS_801-1) *

8. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED (EEHYP_CS_801-1) *

9. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Compiled Mon 06-Apr-09 08:19 by amvarma (EEHYP_CS_801-1) *

10. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Debug Exception (Could be NULL pointer dereference) Exception (0x2000)! (EEHYP_CS_801-1) *

11. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED SRR0 = 0x015B0204 SRR1 = 0x00029230 SRR2 = 0x006B2E9C SRR3 = 0x00021000 (EEHYP_CS_801-1) *

12. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED ESR = 0x00000000 DEAR = 0x00000000 TSR = 0x8C000000 DBSR = 0x10000000 (EEHYP_CS_801-1) *

13. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED CPU Register Context: (EEHYP_CS_801-1) *

14. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Vector = 0x00002000 PC = 0x00A6C49C MSR = 0x00029230 CR = 0x22000028 (EEHYP_CS_801-1) *

15. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED LR = 0x00A6C49C CTR = 0x0022C850 XER = 0x60000006 (EEHYP_CS_801-1) *

16. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R0 = 0x00A6C49C R1 = 0x04399374 R2 = 0x00000000 R3 = 0x00000000 (EEHYP_CS_801-1) *

17. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R4 = 0x015B0204 R5 = 0x00029230 R6 = 0x00029230 R7 = 0xBEEFCAFE (EEHYP_CS_801-1) *

18. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R8 = 0x00000000 R9 = 0x025C0000 R10 = 0x0251D838 R11 = 0x0251D308 (EEHYP_CS_801-1) *

19. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R12 = 0x68A321FD R13 = 0x00110000 R14 = 0x001DB47C R15 = 0x00000000 (EEHYP_CS_801-1) *

20. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x00000000 (EEHYP_CS_801-1) *

21. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R20 = 0x00000000 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x00000000 (EEHYP_CS_801-1) *

22. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED R24 = 0x00000000 R25 = 0x00000000 R26 = 0x00000000 R27 = 0x00000000 (EEHYP_CS_801-1) *

23. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35

PLATFORM 1 CRASHED R28 = 0x00000000 R29 = 0x00000000 R30 = 0x00000000 R31 = 0x00000003 (EEHYP_CS_801-1) *

24. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Stack trace: (EEHYP_CS_801-1) *

25. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED PC = 0x00A6C49C, SP = 0x04399374 (EEHYP_CS_801-1) *

26. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 00: SP = 0x04399384 PC = 0x00A6C49C (EEHYP_CS_801-1) *

27. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 01: SP = 0x04399394 PC = 0x00A642EC (EEHYP_CS_801-1) *

28. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 02: SP = 0x043993AC PC = 0x00A6F160 (EEHYP_CS_801-1) *

29. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 03: SP = 0x043993F8 PC = 0x00A6290C (EEHYP_CS_801-1) *

30. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 04: SP = 0x04399490 PC = 0x00D667B8 (EEHYP_CS_801-1) *

31. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 05: SP = 0x04399518 PC = 0x00D66F04 (EEHYP_CS_801-1) *

32. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 06: SP = 0x04399538 PC = 0x00A68208 (EEHYP_CS_801-1) *

33. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 07: SP = 0x04399558 PC = 0x015B0AC8 (EEHYP_CS_801-1) *

34. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 08: SP = 0x043995B8 PC = 0x001DB510 (EEHYP_CS_801-1) *

35. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 09: SP = 0x043995C0 PC = 0x00A6E428 (EEHYP_CS_801-1) * 36. EEHYP_CS_801 192.168.171.6 Jun 28 2011 06:55:35 PLATFORM 1 CRASHED Frame 10: SP = 0x00000000 PC = 0x00A64EF8 (EEHYP_CS_801-1) *

!

!

!

!

------------------ show version ------------------

Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(50)SE1, RELEASE SOFTWARE (fc2)

Copyright (c) 1986-2009 by Cisco Systems, Inc.

Compiled Mon 06-Apr-09 08:19 by amvarma

Image text-base: 0x00003000, data-base: 0x01900000

ROM: Bootstrap program is C3750 boot loader

BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(44)SE5, RELEASE SOFTWARE (fc1)

EEHYP_CS_801 uptime is 10 weeks, 5 days, 14 hours, 32 minutes

System returned to ROM by power-on

System restarted at 21:12:09 GMT Wed Apr 13 2011

System image file is "flash:/c3750-ipbasek9-mz.122-50.SE1.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C3750-24TS (PowerPC405) processor (revision M0) with 118784K/12280K bytes of memory.

Processor board ID FDO1136Z94J

Last reset from power-on

6 Virtual Ethernet interfaces

96 FastEthernet interfaces

8 Gigabit Ethernet interfaces

The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address       : 00:1D:A2:54:CA:80

Motherboard assembly number     : 73-9677-10

Power supply part number        : 341-0034-01

Motherboard serial number       : FDO11360HY5

Power supply serial number      : DTH112311E8

Model revision number           : M0

Motherboard revision number     : B0

Model number                    : WS-C3750-24TS-S

System serial number            : FDO1136Z94J

Top Assembly Part Number        : 800-25857-02

Top Assembly Revision Number    : F0

Version ID                      : V05

CLEI Code Number                : CNMV100CRE

Hardware Board Revision Number  : 0x01

Switch Ports Model              SW Version            SW Image                

------ ----- -----              ----------            ----------              

     1 26    WS-C3750-24TS      12.2(50)SE1           C3750-IPBASEK9-M        

*    2 26    WS-C3750-24TS      12.2(50)SE1           C3750-IPBASEK9-M        

     3 26    WS-C3750-24TS      12.2(50)SE1           C3750-IPBASEK9-M        

     4 26    WS-C3750V2-24TS    12.2(50)SE1           C3750-IPBASEK9-M        

Switch 01

---------

Switch Uptime                   : 4 hours, 51 minutes

Base ethernet MAC Address       : 00:1D:A2:41:E3:00

Motherboard assembly number     : 73-9677-10

Power supply part number        : 341-0034-01

Motherboard serial number       : FDO11360HP2

Power supply serial number      : DTH112311C3

Model revision number           : M0

Motherboard revision number     : B0

Model number                    : WS-C3750-24TS-S

System serial number            : FDO1136Z93W

Top assembly part number        : 800-25857-02

Top assembly revision number    : F0

Version ID                      : V05

CLEI Code Number                : CNMV100CRE

Switch 03

---------

Switch Uptime                   : 10 weeks, 5 days, 14 hours, 32 minutes

Base ethernet MAC Address       : 00:1E:49:BD:0A:00

Motherboard assembly number     : 73-9677-10

Power supply part number        : 341-0034-01

Motherboard serial number       : FDO1145074X

Power supply serial number      : DTH113910A1

Model revision number           : M0

Motherboard revision number     : B0

Model number                    : WS-C3750-24TS-S

System serial number            : FDO1145Y1EB

Top assembly part number        : 800-25857-02

Top assembly revision number    : F0

Version ID                      : V05

CLEI Code Number                : CNMV100CRE

Switch 04

---------

Switch Uptime                   : 2 days, 1 hour, 11 minutes

Base ethernet MAC Address       : B4:A4:E3:8F:14:00

Motherboard assembly number     : 73-11764-08

Power supply part number        : 341-0328-02

Motherboard serial number       : FDO143615TZ

Power supply serial number      : DCA1428M2AQ

Model revision number           : H0

Motherboard revision number     : A0

Model number                    : WS-C3750V2-24TS-S

System serial number            : FDO1437X0L8

Top assembly part number        : 800-31047-04

Top assembly revision number    : B0

Version ID                      : V04

CLEI Code Number                : COMP800ARA

Configuration register is 0xF

1 Accepted Solution

Accepted Solutions

Looks like it's a memory leak and could be attributed to Cisco Bug ID CSCsk84233.  Try disabling EIGRP and see if this helps.  Otherwise, you can try 12.2(55)SE3.

Now looking at your first post there's a possibility that your switches were the first generation of 3750 with 16mb of flash.  If this is the case then download the IOS with the "lm" prefix so you can archive it and fit into your switch.  The IOS BIN file is the same size but the HTML stuff is cut down to fit.

View solution in original post

5 Replies 5

Leo Laohoo
Hall of Fame
Hall of Fame

This doesn't give me any information.  Can you post the crashinfo file that could be found in switch stack member 1?

Hi, I attached 2 files for more information. Please let me know if you can open and view this files.

Thanks a lot.

Regards,

Your crashinfo file is invalid.  The crashinfo was taken when your switch was running 12.2(25)SEB4. 

Hi, I'm sorry ...

I attached the right files. Please let me know if you can open and view this files.

Also I Attached a file, where you can see the changes apply to the swicth in the moment of Crash.

Thanks a lot.

Regards,

Looks like it's a memory leak and could be attributed to Cisco Bug ID CSCsk84233.  Try disabling EIGRP and see if this helps.  Otherwise, you can try 12.2(55)SE3.

Now looking at your first post there's a possibility that your switches were the first generation of 3750 with 16mb of flash.  If this is the case then download the IOS with the "lm" prefix so you can archive it and fit into your switch.  The IOS BIN file is the same size but the HTML stuff is cut down to fit.

Review Cisco Networking for a $25 gift card