-traceback after ios upgrade, no other logs 2911

[Tboss99]

Hi,

Im in the process of upgrading out routers, and ive upgraded a couple of them to 15.6-3.M7 without issue. (c2900-universalk9-mz.SPA.156-3.M7)

However my first attempt with 15.6-3.M7 went horribly wrong. As there is very little to go on, i guess this is a longshot, but checking in case someone else encountered this.

- Router came back up and I could access it remotely over DMVPN tunnel

- Users behind behind lost all external network access, (internet and intranet)
- Router kept throwing this is repeat, but nothing else. And no crash files of any kind generated.

Nov 27 10:34:39.198 CET: -Traceback= 36D3E928z 36CCB778z 30032B5Cz 30032B40z

Nov 27 10:34:55.455 CET: -Traceback= 36D3E928z 36D67148z 36D67228z 36D484CCz 36D5A5A8z 36D5AA40z 36D5AAF8z 36D4B9ACz 36D4F4DCz 30032B5Cz 30032B40z

Just these two events and nothing else. No file generated, nothing. No other event at all in the log,
I reloaded back to the old image and it was fine again.

Upgrade from
c2900-universalk9-mz.SPA.152-4.M5.bin

And nothing useful on the flash

Directory of flash0:/

1 -rw- 30836 Sep 18 2019 14:34:44 +01:00 TESTSDWAN.CFG
2 -rw- 3064 Apr 2 2014 12:11:50 +01:00 cpconfig-29xx.cfg
3 -rw- 36970 Sep 21 2020 12:07:30 +01:00 beforeISR2change.cfg
4 drw- 0 Apr 2 2014 12:12:22 +01:00 ccpexp
245 -rw- 2464 Apr 2 2014 12:14:06 +01:00 home.shtml
246 -rw- 110617124 Nov 26 2020 15:10:58 +01:00 c2900-universalk9-mz.SPA.156-3.M9.bin
247 -rw- 99098648 Sep 20 2013 22:11:38 +01:00 c2900-universalk9-mz.SPA.152-4.M5.bin
248 -rw- 25415 Sep 5 2014 15:51:30 +01:00 <routername>.CFG
249 -rw- 37249 Nov 27 2020 10:21:34 +01:00 backup1.cfg

Current show ver

Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.2(4)M5, RELEASE SOFTWARE (fc2)

Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Fri 13-Sep-13 14:59 by prod_rel_team

ROM: System Bootstrap, Version 15.0(1r)M16, RELEASE SOFTWARE (fc1)

[balaji.bandi]

Looks for me a Bug, can you try any other upper version ? - is that works ?

[Tboss99]

Thanks,

Well its not really straight forward for me to test other versions. I do know 15.6-3.M7 worked ok on other routers (same hardware) with similar config. This one is slightly different than other branch routers as it has a SD-wan connection to HK.

The problem is that im remotely managing it from Europe, the router is in Shanghai so needs planning and someone standing by locally with a 4g connection and console cable in case it goes wrong.

Is there any point opening a tac case with so little data to go on?

But id be happy to take suggestions for different version. We mainly just want to make sure we get the latest security fixes in place. We don't need new features etc. Just stability and security.

[Georg Pauwen]

Hello,

--> Router came back up and I could access it remotely over DMVPN tunnel

--> Users behind behind lost all external network access, (internet and intranet)

Post the full running configuration, maybe we can spot something. The traceback might not mean much and could just be informational. Also, post the output of 'show flash', maybe your flash is (close to) full.

[Tboss99]

I appreciate that. Thanks.

There is about 44mb left on the flash.

See attached.

[Georg Pauwen]

Hello,

the first thing to notice is that you have about 200 static routes in your configuration. Check if you can summarize some of them, or possibly even delete them if they are not needed.

Also, you might want to delete the old image file in flash (backup the image to a TFTP server first), c2900-universalk9-mz.SPA.152-4.M5.bin, that will free up a lot of flash memory.

[Tboss99]

Thanks.

But that does not solve the crashes and lack of LAN connectivity. I would think 44mb should be plenty, at least it has on other routers we've upgraded. Deleting the old image is something we do after about a week of stable operation on the new version. We need be able to quickly revert. And glad we did in this case as any LAN tftp was not reachable due to the issue.

An good recommendation as to what build we should go for instead, that still has the security fixed. Or at least most of them?

[Richard Burts]

A traceback is always a sign of a software problem. We do not have the ability to evaluate the traceback and identify the issue, but TAC does. I would think it might be worth while opening a case with TAC and see what they have to say.