Regretting 6807

[mfwilkinson]

Just recently migrated from a 6509 core with typical 3500 distribution and access switchs to a 6807 core with FEXes replacing all our other switches. Today we just got hit several (4) times by bug CSCup70579, each time this forced a reload of the system. This brought our entire company down, core, FEXes, phones, Wi-Fi, Internet, all of it. With all the FEXes (15) these reloads took perhaps a bit more than 20 minutes each time. This made it extremely difficult to troubleshoot and hit both the Sups (active and standby in the system). This was absolutely crushing to the company, and a loss of faith in our networking staff. I've never had such a negative experience with a Cisco product prior to this...the previous 6509 ran absolutely flawless and continuously for many years.

We were not made aware of the bug prior to getting hit with it. The TAC staff had a very difficult time pinpointing the problem (until it was pushed to engineering). Cisco is aware that we own this product and being on SmartNet shouldn't we receive some kind of notice via email that a critical, reload causing, issue potentially exists in our system? The bug notice was released late September and we've been sitting on that timebomb till it hit us. Is the normal procedure to search through all the bugs on Cisco's site and determine if our hardware/software is potentially affected by each one? Perhaps we've just become used to the way RedHat, Microsoft, and EMC notify their customers of critical issues and we're beginning to expect the same from Cisco...

[Reza Sharifi]

Thank you for posting this, as this will help network engineers around the globe keep their eyes open on new products and IOS releases and to do a lot more research and testing before deploying anything new.  I am sorry about the issues you are facing with.  Unfortunately, Cisco doses not notify the customers of any known bugs in their software. The bugs get published in release notes and usually the customers find out the hard way they are facing an issue; hence this bug.

[paul driver]

Hello

considering the expense these company's spend and how vital the network is them-Cisco should review that procedure and inform their customers  of any potential outage that their software  could have on their systems especially a core network!

res

Paul

[Reuben Farrelly]

Unfortunately the release notes have been often incomplete and don't even begin to cover many of the serious bugs that are known about.

Here are two that I have come across in the past 12 months:

CSCul95289 - Console / VTY / SSH session hangs on show command (fails under normal operation and requires reload of the chassis to clear)

CSCun26289 - Problems transferring/answering calls when using headset with 79XX phone (breaks core functionality for every user who has a headset)

Neither of these were (or are) in the release notes, and both were serious defects.  At least these bugs were publically viewable though :-)

There also seems to be a push within the documentation teams to stop documenting defects in release notes entirely - the last two 15M release rebuilds have all replaced these with instructions on how to get the information out of Bug Toolkit directly.

Which is fine in principle, except Bug Toolkit is a PITA to get this sort of information out of quickly...even my grunty high end PC crawls while trying to scroll through the lists.....

[Leo Laohoo · Accepted solution]

Cisco is aware that we own this product and being on SmartNet shouldn't we receive some kind of notice via email that a critical, reload causing, issue potentially exists in our system?

Yes you can.  Have you heard of Cisco Notification Service?

[bigjess]

Have you configured COPP to temporarily prevent the BUG until the fix comes out?

[mfwilkinson]

Yes, thanks. we've applied the suggested workaround using COPP.

[Leo Laohoo]

Thank you for taking the time to rate our posts.  :)

[sgt2111usmc]

Is there any word on when the new code for this bug CSCup70579 is going to be released?  The last notification that I heard was this month.  

[mfwilkinson]

It's out. Just applied it a couple days ago.