iprouting .iosproc error on 6509

[Vinod Hira]

We had a major network outage due the followong error which was found in logs

%DUAL-3-INTERNAL: EIGRP-IPv4:(20) 100: Internal Error : iprouting.iosproc : (PID=16432, TID=6) : -Traceback=(s72033_rp-adventerprisek9_wan-4-dso-n.so+0x1E7A4C) ([40:0]+0x1969EC) ([40:0]+0x1919A0) ([40:0]+0x192378) ([40:0]+0x1B380C) ([40:0]+0x1B332C) ([40:0]+0x1A8FE4([40:0]+0x1A9B2C) ([40:0]+0x1CA560) ([30:-9]1+0xB5360) ([40:0]+0xB5338) .

Actually after log search on Google we have manage to restart iprouting.iosproc.

Now my request tp cisco community that why this could happen and also why this error message didnot logged in to the syslog sever ?

Thank you all for solution in advance.

[Reza Sharifi]

Hi,

Do you see a crash file on your flash?

dir flash

or sh flash

What IOS version are you running?

HTH

[Richard Burts · Accepted solution]

There are several indications in what you post that suggest that this is a software bug. First I would note that the error message describes an internal error in EIGRP. Also the fact that there is traceback in the message is an indication of a software bug. We do not have sufficient information to identify the specific error. If this router is on a maintenance contract then I suggest that you open a case with Cisco TAC.

HTH

Rick

Sent from Cisco Technical Support iPad App

[Vinod Hira]

Hi Reza & Richard

Thank you for the reply, it will be very usefull. We do have maintenance contatract with IBM so we will have to log the fault with them.

The IOS version is:

sup-bootdisk:/s72033-adventerprisek9_wan-vz.122-33.SXI.bin

This 6509 is one of the distribution device in our network and untill we get a solution from IBM, I would like the suggession from community to mitigate this issue. We have another 6509 running in pair with this troublesome which has a  mirror configuration. So, because of that one gone down, other one should update the routing table to core but that din't happened and that's why half of the network went down. I know this could be a configuration or design issue which I will have look at.

Now as I have metioned in my first post that this internal error was in log but why it didn't go in syslog server ? Due to this error there were ton's of NBRCHANGE log went into our syslog server Stateseekar except this one. So could anyone suggest the log config that can send this type of alert ?

Thank you all again.

[Reza Sharifi]

Hi,

Here is some good info on how to get the crash file from 6500 or 7600 series routers.

https://supportforums.cisco.com/docs/DOC-19727

Also, your IOS version (SXI) is at least 4 years old and this may be a good time to upgrade.

HTH

[Vinod Hira]

Hi Reza

Thanks again but unfortunatly there was no crashinfo for that particular incident or on that day. On the  day of the incident this 6509 not actually crashed but only EIGRP routing process was stopped. When we have check this device on that day, it has all the routes in table and all the eigrp neighbors were pingable but routing update didn't send to neighbors. Have a look this bootflash:

-#- ED ----type---- --crc--- -seek-- nlen -length- ---------date/time--------- name

1   .. unknown      D743047E   DDD80   34   384253 Oct 30 2008 12:29:36 +13:00 crashinfo_ios-base-20081029-232932

2   .. unknown      69B9BA3A  139330   34   374063 Oct 30 2008 18:00:26 +13:00 crashinfo_ios-base-20081030-050022

3   .. unknown      F5713ABE  1985F4   34   389697 Oct 31 2008 12:09:24 +13:00 crashinfo_ios-base-20081030-230920

4   .. unknown      41BDB80C  1F9578   34   397060 Oct 31 2008 16:56:19 +13:00 crashinfo_ios-base-20081031-035615

5   E. unknown      FFFFFFFF  21ECF8   34 -      1 Feb 11 2010 06:13:55 +13:00 crashinfo_ios-base-20100210-171348

6   .. config       BE133801  22ECDC   12    65380 Jun 1 2011 01:29:31 +12:00 VLANSTP.conf

7   .. config       BE1341FF  23ECC0   10    65380 Jun 1 2011 01:52:08 +12:00 Changedone

8   .. config       97E8F5C2  24EC18   15    65240 Jun 15 2011 18:39:23 +12:00 vlan-pre-change

---------------------------------

These were old anyway.

I think you right, it would be qood time to upgrade.

Also I am looking for an answer or solution that why this type of error not been sent to syslog server ?

Cheers all.