cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5441
Views
10
Helpful
17
Replies

ME-3800X-24FS-M

ruslank
Level 1
Level 1

Hi,

I got on ME-3800X-24FS-M some strange messages and can't find any documents what is mean:

May  2 2011 17:18:27.552 EET: platform assert failure: 0: ../src-nile/src-asic-nile/nile_adjmgr.c: 6573: adjmgr_l3_switch_nh_to_ucast
May  2 2011 17:18:27.552 EET: -Traceback= 422970 53D924 D93EBC B402A0 B36C60 41E524 420794 B33F14 B35804 DC8EA4 DBF92C
May  2 2011 22:07:11.352 EET: platform assert failure: 0: ../src-nile/src-asic-nile/nile_adjmgr.c: 6509: adjmgr_l3_switch_nh_to_flood
May  2 2011 22:07:11.352 EET: -Traceback= 422970 53DE3C D93F58 B40910 B3745C 421C40 4222E0 DC8EA4 DBF92C

Installed software:

Cisco IOS Software, ME380x Software (ME380x-UNIVERSAL-M), Version 12.2(52)EY1, RELEASE SOFTWARE (fc1)

ROM: Bootstrap program is WHALES boot loader
BOOTLDR: ME380x Boot Loader (ME380X-HBOOT-M) Version 12.2(52r)EY2, RELEASE SOFTWARE (fc1)

3 Accepted Solutions

Accepted Solutions

garhall
Level 1
Level 1

Hi,

After decoding te traceback in the error message I can confirm this is being investigated in a Cisco internal bug:

CSCtj22513 platform assert failure traceback seen on modifying interface

This is fixed through:

CSCtk56241 crash noticed after nile assert error messages

This issue is fixed in IOS version 12.2(52)EY2 which should be available on Cisco.com soon. You will be able to download the image from the following link:

http://www.cisco.com/cisco/software/navigator.html?mdfid=283052113&i=rm

The issue itself should not cause any issue (the crash will not occur in your version).

To prevent this upgrade to the mentioned version when it's available.

Thanks,

/gary

View solution in original post

Hi,

No problem, happy to help. If you could mark this question as 'answered' that would be great, thanks.

/gary

View solution in original post

Hi,

It seems that the same bug is striking back.

It turned out that CSCtk56241 was fixing assert messages and traffic loss problems but it appears that the fix made for that bug does not cover all possible triggers.

Development team has recently given to a TAC customer hitting the same issue with EY2 and engineering image which will supposdely fix the issue for good or in the worst scenario help troubleshoot the problem further.

Likely EY3 will contain the final fix.

Anyway you'd better monitor the bug to see when it gets updated with the final resolution notes.

Riccardo




View solution in original post

17 Replies 17

garhall
Level 1
Level 1

Hi,

After decoding te traceback in the error message I can confirm this is being investigated in a Cisco internal bug:

CSCtj22513 platform assert failure traceback seen on modifying interface

This is fixed through:

CSCtk56241 crash noticed after nile assert error messages

This issue is fixed in IOS version 12.2(52)EY2 which should be available on Cisco.com soon. You will be able to download the image from the following link:

http://www.cisco.com/cisco/software/navigator.html?mdfid=283052113&i=rm

The issue itself should not cause any issue (the crash will not occur in your version).

To prevent this upgrade to the mentioned version when it's available.

Thanks,

/gary

Thank you.

Hi,

No problem, happy to help. If you could mark this question as 'answered' that would be great, thanks.

/gary

ruslank
Level 1
Level 1

Hi,

After upgrading software to version 12.2(52)EY2:

Cisco IOS Software, ME380x Software (ME380x-UNIVERSALK9-M), Version 12.2(52)EY2, RELEASE SOFTWARE (fc1)

BOOTLDR: ME380x Boot Loader (ME380X-HBOOT-M) Version 12.2(52r)EY2, RELEASE SOFTWARE (fc1)

License Level: MetroIPServices

I got very similiar log messages:

Jun 30 2011 10:53:34.108 EET: platform assert failure: 0: ../src-nile/src-asic-nile/nile_adjmgr.c: 6662: adjmgr_l3_switch_nh_to_ucast

Jun 30 2011 10:53:34.108 EET: -Traceback= 4252E4 540460 DB0FA0 B4472C B3B0EC 420E98 423108 B383A0 B39C90 DE6088 DDCB10

Jun 30 2011 10:53:34.108 EET: Assert adjmgr_l3_switch_nh_to_ucast mac (0013.8f24.020a) vlan 1553

Jun 30 2011 11:47:00.571 EET: platform assert failure: 0: ../src-nile/src-asic-nile/nile_adjmgr.c: 6585: adjmgr_l3_switch_nh_to_flood

Jun 30 2011 11:47:00.571 EET: -Traceback= 4252E4 5409F4 DB1050 B44D9C B3B8E8 4245B4 424C54 DE6088 DDCB10

Jun 30 2011 11:47:00.571 EET: Assert  adjmgr_l3_switch_nh_to_flood mac (0013.7737.f95f) vlan 1851

This is the same BUG or second one?

Thank you.

Hi,

It seems that the same bug is striking back.

It turned out that CSCtk56241 was fixing assert messages and traffic loss problems but it appears that the fix made for that bug does not cover all possible triggers.

Development team has recently given to a TAC customer hitting the same issue with EY2 and engineering image which will supposdely fix the issue for good or in the worst scenario help troubleshoot the problem further.

Likely EY3 will contain the final fix.

Anyway you'd better monitor the bug to see when it gets updated with the final resolution notes.

Riccardo




Hi,

I installed version 12.2(52)EY2a and this error message is gone!!!

Now with version 12.2(52)EY2a I got new crash:

Aug 17 2011 14:33:00.978 EET: %PLATFORM_NCEF-3-ADJ:

Null interface handle for if_number 4237 Adj hdl 175903056,Midchain Adj NO

-Traceback= DB060C DA0C68 DA3B90 D9C00C 12DB698 E79F74 E7D408 E7D8F8 17DB8D4 1458440 145A81C 145C008 145C370 145D50C 17990E4 1798630

Maybe you can help to understand what is it mean?

Hi,

Did the switch completely crash or did you just see that traceback but the switch did not reload?

If it did not crash I need the show log showing the messages before the traceback (including the traceback) and a show ver.

If it crashed I need the crashinfo file.

Where you configuring some command related to IP PIM by any chance? What exactly?

I might have an idea already but I need to double check.

Riccardo

Hi,

Sorry for delay in the answer, but I update to new software and have the same problem:

Jan 12 2012 03:16:51.173 EET: %PLATFORM_NCEF-3-ADJ:

Null interface handle for if_number 4239 Adj hdl 174736464,Midchain Adj NO

-Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

Jan 12 2012 03:26:54.829 EET: %PLATFORM_NCEF-3-ADJ:

Null interface handle for if_number 4239 Adj hdl 173238512,Midchain Adj NO

-Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

Jan 12 2012 06:08:09.099 EET: %PLATFORM_NCEF-3-ADJ:

Null interface handle for if_number 4239 Adj hdl 170613168,Midchain Adj NO

-Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

Jan 12 2012 08:13:58.530 EET: %PLATFORM_NCEF-3-ADJ:

Null interface handle for if_number 4239 Adj hdl 174753264,Midchain Adj NO

-Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

My software version:

# sh version

Cisco IOS Software, ME380x Software (ME380x-UNIVERSALK9-M), Version 12.2(52)EY3a, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Fri 21-Oct-11 00:26 by prod_rel_team

License Level: MetroIPServices

License Type: Permanent

Next reload license Level: MetroIPServices

Traceback is appears after restart cisco.

Thank you.

please see my previous message

Hi,

>Did the switch completely crash or did you just see that traceback but the switch did not reload?

Switch is working and I don't understand that is not work correctly.

>If it did not crash I need the show log showing the messages before the traceback (including the traceback) and a show ver.

All messages after reload:

Jan 12 02:00:01 ME3800x 52633: Jan 12 2012 02:00:00.273 EET: %SYS-5-RELOAD: Reload requested by ruslan on vty1 (78.158.0.65). Reload reason: Reason unspecified

Jan 12 02:06:03 ME3800x 614: Jan 12 2012 02:06:02.681 EET: %DHCP_SNOOPING-6-AGENT_OPERATION_SUCCEEDED: DHCP snooping database Read succeeded.

Jan 12 02:06:07 ME3800x 615: Jan 12 2012 02:06:06.779 EET: %SPANTREE-5-TOPOTRAP: Topology Change Trap for vlan 1

Jan 12 02:06:07 ME3800x 616: Jan 12 2012 02:06:07.392 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan851

Jan 12 02:06:07 ME3800x 617: Jan 12 2012 02:06:07.392 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan852

Jan 12 02:06:07 ME3800x 618: Jan 12 2012 02:06:07.392 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan951

Jan 12 02:06:07 ME3800x 619: Jan 12 2012 02:06:07.392 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan952

Jan 12 02:06:07 ME3800x 620: Jan 12 2012 02:06:07.392 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan953

Jan 12 02:06:08 ME3800x 621: Jan 12 2012 02:06:07.392 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan954

Jan 12 02:06:08 ME3800x 622: Jan 12 2012 02:06:07.785 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan951, changed state to up

Jan 12 02:06:08 ME3800x 623: Jan 12 2012 02:06:07.785 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan952, changed state to up

Jan 12 02:06:08 ME3800x 624: Jan 12 2012 02:06:07.785 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan953, changed state to up

Jan 12 02:06:08 ME3800x 625: Jan 12 2012 02:06:07.785 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan954, changed state to up

Jan 12 02:06:08 ME3800x 626: Jan 12 2012 02:06:08.013 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan851, changed state to up

Jan 12 02:06:09 ME3800x 627: Jan 12 2012 02:06:08.013 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan852, changed state to up

Jan 12 02:06:10 ME3800x 628: Jan 12 2012 02:06:09.869 EET: %SPANTREE-5-TOPOTRAP: Topology Change Trap for vlan 48

Jan 12 02:06:10 ME3800x 629: Jan 12 2012 02:06:10.420 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan1058

Jan 12 02:06:10 ME3800x 630: Jan 12 2012 02:06:10.876 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1058, changed state to up

Jan 12 02:06:12 ME3800x 631: Jan 12 2012 02:06:11.426 EET: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 78.158.8.1 on interface Vlan47

Jan 12 02:06:12 ME3800x 632: Jan 12 2012 02:06:11.788 EET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan47, changed state to up

Jan 12 03:16:52 ME3800x 633: Jan 12 2012 03:16:51.173 EET: %PLATFORM_NCEF-3-ADJ:

Jan 12 03:16:52 ME3800x 634: Null interface handle for if_number 4239 Adj hdl 174736464,Midchain Adj NO

Jan 12 03:16:52 ME3800x 635: -Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

Jan 12 03:26:55 ME3800x 638: -Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

Jan 12 06:08:10 ME3800x 639: Jan 12 2012 06:08:09.099 EET: %PLATFORM_NCEF-3-ADJ:

Jan 12 06:08:10 ME3800x 640: Null interface handle for if_number 4239 Adj hdl 170613168,Midchain Adj NO

Jan 12 06:08:10 ME3800x 641: -Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

Jan 12 08:13:59 ME3800x 642: Jan 12 2012 08:13:58.530 EET: %PLATFORM_NCEF-3-ADJ:

Jan 12 08:13:59 ME3800x 643: Null interface handle for if_number 4239 Adj hdl 174753264,Midchain Adj NO

Jan 12 08:13:59 ME3800x 644: -Traceback= DB23C0 DA2A1C DA5944 D9DDC0 12DD3FC E7BD28 E7F1BC E7F6AC 17DD62C 145A1A4 145C580 145DD6C 145E0D4 145F270 179AE3C 179A388

#sh version

Cisco IOS Software, ME380x Software (ME380x-UNIVERSALK9-M), Version 12.2(52)EY3a, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Fri 21-Oct-11 00:26 by prod_rel_team

Image text-base: 0x00003000, data-base: 0x0221CDF4

ROM: Bootstrap program is WHALES boot loader

BOOTLDR: ME380x Boot Loader (ME380X-HBOOT-M) Version 12.2(52r)EY2, RELEASE SOFTWARE (fc1)

3800-fst-zad20 uptime is 12 hours, 53 minutes

System returned to ROM by power-on

System restarted at 02:01:05 EET Thu Jan 12 2012

System image file is "flash:/me380x-universalk9-mz.122-52.EY3a/me380x-universalk9-mz.122-52.EY3a.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

License Level: MetroIPServices

License Type: Permanent

Next reload license Level: MetroIPServices

cisco ME-3800X-24FS-M (PowerPC8572) processor (revision A0) with 1015808K/32760K bytes of memory.

Processor board ID FOC1511X2C9

Last reset from power-on

180 Virtual Ethernet interfaces

25 Gigabit Ethernet interfaces

2 Ten Gigabit Ethernet interfaces

The password-recovery mechanism is enabled.

1536K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address       : 44:E4:D9:9A:54:80

Motherboard assembly number     : 73-12068-07

Motherboard serial number       : FOC151001TD

Model revision number           : A0

Motherboard revision number     : A0

Model number                    : ME-3800X-24FS-M

System serial number            : FOC1511X2C9

Top Assembly Part Number        : 800-31465-01

Top Assembly Revision Number    : A0

Version ID                      : V01

CLEI Code Number                : IPMSW00DRA

Configuration register is 0xF

>If it crashed I need the crashinfo file.

Switch is working

>Where you configuring some command related to IP PIM by any chance? What exactly?

Yes we are using multicast routing

Example of configuration:

interface Vlan101

ip address 78.158.1.1 255.255.255.248

no ip redirects

no ip unreachables

no ip proxy-arp

ip pim passive

end

interface Vlan3001

ip address 78.158.0.73 255.255.255.252

no ip proxy-arp

ip pim sparse-mode

ip ospf network point-to-point

ip ospf hello-interval 1

hold-queue 4096 in

hold-queue 4096 out

end

#sh ip igmp ssm-mapping

SSM Mapping  : Enabled

DNS Lookup   : Enabled

Mcast domain : stream.fastlink.lt

Name servers : 78.158.0.2

               78.158.0.65

can you tell me the last command that was entered right before the traceback (i.e. interface svi removed, pim related command added or removed etc.)? Please also check if you have scripts running that make configuration changes.

Riccardo

Hi,

I'm post full log of cisco after restart. No changes in the configuration, no connection to managment.

Sheduled reload at 02:00:

Jan 12 02:00:01 ME3800x 52633: Jan 12 2012 02:00:00.273 EET:  %SYS-5-RELOAD: Reload requested by ruslan on vty1 (78.158.0.65). Reload  reason: Reason unspecified

First error:

Jan 12 03:16:52 ME3800x 633: Jan 12 2012 03:16:51.173 EET: %PLATFORM_NCEF-3-ADJ:

the commands which are typed are not showing in the log.

Are you 100% sure that you or anybody else (scripts included) did not make any single change from the time the switch was reloaded and the time the first traceback was printed?

Riccardo

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: