topic Re: Signature Update S601 Pulled From Cisco.com in Network Security

Signature Update S601 Pulled From Cisco.com

nicksmi — Tue, 26 Mar 2019 00:20:11 GMT

Due to a bug in IPS software versions prior to 7.0.6, it is not advised to apply signature updates past S601 without having upgraded to 7.0.6 first. This issue is being tracked as CSCtn23051. If a customer has applied S601 without problems, there is no issue beyond having to upgrade to 7.0.6 to apply updates S602 and later. If you have not applied S601 yet, it is advised to upgrade to 7.0.6 before applying update S602 and later.

Thank you for your patience and understanding in this matter.

Nicholas Smith

Cisco IPS Signature Team

Signature Update S601 Pulled From Cisco.com

hartkl5277 — Wed, 12 Oct 2011 15:17:10 GMT

Is there a fix for the systems that have the S601 applied and not responding?

Re: Signature Update S601 Pulled From Cisco.com

nicksmi — Wed, 12 Oct 2011 15:36:02 GMT

Upgrade to 7.0(6) or 7.1. Update 6.2.4 also resolves the issue.

Re: Signature Update S601 Pulled From Cisco.com

hoylea — Wed, 12 Oct 2011 20:19:38 GMT

This is unacceptable. I've attempted to upgrade 4 devices to 7.0.6 today and three have failed.

You released signature update that only works on the latest code

...which has only been released for a month

... which automatically applies to all devices in the field

... without any ability to remove short of a reimage

... without sufficient (any?) testing on other versions

What if you released a 602 sig file that is actually 600? Would that roll back? If so, do that and release 601 as 603 when it's actually been tested!

EDIT: I have found that rebooting before attempting the upgrade seems to increase the chance of success.

Signature Update S601 Pulled From Cisco.com

nicksmi — Wed, 12 Oct 2011 21:42:45 GMT

Mr. Anderson,

We apologize for the problems you have been experiencing with our sensors. The root cause is a memory segmentation problem that is fixed in the 7.0(6) software release and cannot be reliably fixed via a signature update. Signature update S601 merely brought us to the threshold where the problem manifests itself more frequently. Our updates are tested and due to the nature of this problem its occurrence is fairly random and we did not observe it. We apologize for the problems this has caused and are reviewing our internal processes to avoid a reoccurrence.

We recomend that customers be at the latest signature update level to provide the max protection, and it is important to be at the latest code level as well to ensure reliable and effective functionality.

At this time, we are asking you to open a TAC case to document your upgrade woes and resolve them.

http://www.cisco.com/cisco/web/support/index.html

Thank you for your patience in this matter.

Nicholas Smith

Cisco IPS Signature Team

Re: Signature Update S601 Pulled From Cisco.com

whaydenscana — Thu, 13 Oct 2011 00:06:28 GMT

Hoyle,

I had the same problem, this is what I did to get it downgraded.

---------

Downgrade from console---

....unplug monitor port (prevent engine from running) .... reset IPS.....downgrade from command line to 600....go to gui turn off auto update....plug ports back in.

Downgrade from ssh---

shutdown monitor port (prevent engine from running) .... reset IPS.....downgrade from command line to 600....go to gui turn off auto update....plug ports back in.

At this point you should be able initiate the system upgrade.

Thanks,

Will

Re: Signature Update S601 Pulled From Cisco.com

hartkl5277 — Thu, 13 Oct 2011 14:16:50 GMT

Is the issue with a specific signature being loaded or something else in the package? I was thinking that if is just a signature that you could use the command line to retire the signature so that it is not loaded and then reset the system. Any thoughts or feedback? Seems like S602 is taking a long time to get released, going through a better QA process I hope.

Signature Update S601 Pulled From Cisco.com

Costin Vilcu — Thu, 13 Oct 2011 14:43:45 GMT

Nick, you said "Update 6.2.4 also resolves the issue".

Does that mean that also the 6.0.x, 6.1.x and 6.2.[1,2,3] are affected by this bug?

Thank you,

Re: Signature Update S601 Pulled From Cisco.com

nicksmi — Thu, 13 Oct 2011 15:03:41 GMT

The root issue is with the sensor software (CSCtn23051) and was addressed a number of months ago in the 7.0(5) SP. We reccomend upgrading to 7.0(6) due to other issues with 7.0(5).

S601 happens to fairly reliably reproduce the memory condition that triggers the bug. The only way to reliably fix the problem is to upgrade the sensor software.

Re: Signature Update S601 Pulled From Cisco.com

nicksmi — Thu, 13 Oct 2011 15:04:21 GMT

I belive that is correct. We reccomend that customers be at the latest sig update level to provide the max protection, and it is important to be at the latest code level as well to ensure reliable and effective functionality.

Re: Signature Update S601 Pulled From Cisco.com

twilcox — Thu, 13 Oct 2011 15:48:19 GMT

What is the reccommended course of action for those of us that have AIP-SSM's inside our firewalls?

Question: should we wait until 602 is released? and will the automatic upgrade work for them i.e. will they automatically update to 602?

I cannot upgrade any of our AIP-SSM's currently running 7.0.4 (E4) to the current release via the GUI either with ASDM or with IEV. This is pretty bad.

I would like to avoid downtime since they are inside redundant fw pairs. Upgrades of the AIP-SSM typically require a FW failover when upgrading, so waiting for a sig update to 602 is preferable if it will automatically update. Otherwise I need to know if this will not work in order to schedule downtime / change control to get this fixed.

From a customer support standpoint, if this was a know issue which it evidently was ... Cisco needs to tell customers beforehand to upgrade to the most recent code rev.

The fact that a sig update can kill off the IPS inside a FW is pretty lame. It also shows the lack of regression testing in your software process. You need to fix that but I guess since you guys outsource so much to foreign countries, I'm not surprised when stuff like this happens. I've learned how to suffer being a Cisco customer.

Re: Signature Update S601 Pulled From Cisco.com

nicksmi — Thu, 13 Oct 2011 17:29:17 GMT

Upgrading to S602 will not solve the problem.

Signature Update S601 aggravated a bug (CSCtn23051) existing in some older versions of the sensor software.

To resolve this issue, upgrade the sensor software to version 6.2(4) (released June 2011) or 7.0(5) (released May 2011) or greater (7.0(6) (released Sept 2011) is recommended). Sensors running 7.1(x) software are not affected.

A signature update will not resolve the problem. While the problem may not immediately manifest itself with future signature updates, unless the sensor is updated, the problem can occur, depending on the sensor configuration, the traffic being inspected, and the state of the memory.

For the best protection it is critical to apply the appropriate service packs to the sensor to ensure reliable and accurate operation.

Re: Signature Update S601 Pulled From Cisco.com

hoylea — Thu, 13 Oct 2011 17:49:15 GMT

Nicholas,

Thank you for taking the time to respond to this thread.

Re: Signature Update S601 Pulled From Cisco.com

nicksmi — Thu, 13 Oct 2011 20:44:47 GMT

UPDATE:

Signature Update S601 aggravated a bug (CSCtn23051) existing in some older versions of the sensor software.

A signature update will not resolve the problem. While the problem may not immediately manifest itself with future signature updates, unless the sensor is updated, the problem can occur, depending on the sensor configuration, the traffic being inspected, and the state of the memory.

For the best protection it is critical to apply the appropriate service packs to the sensor to ensure reliable and accurate operation.

Signature Update S601 Pulled From Cisco.com

yantiscompany — Fri, 14 Oct 2011 16:49:02 GMT

Wow, what a $#%# up. Glad I had my module set to bypass, or I would have been super pissed.

Here is what worked for me without any down time, since I'm a little lost without my precious GUI:

Putty into your ASA, and do "sh module", and you should see your ips module as 1. Run command "hw-module module 1 reset" to reset your ips module. I tried running the update without restarting the module, and it just got hung up until I restarted it. Download the upgrade 7.0(6) .pkg from cisco. After module is backup, telnet to the module. Run upgrade as stated in this article "http://www.cisco.com/en/US/docs/security/ips/6.2/configuration/guide/cli/cli_system_images.html#wp1142504 ". After module is backup you should be good to go. Module takes a while to restart, so be patient.

Not a pro, so please correct me if any of the above is wrong.

Re: Signature Update S601 Pulled From Cisco.com

twilcox — Fri, 14 Oct 2011 17:26:04 GMT

Thanks for the tip on the hw-module command that saved me a trip to one of our remote sites …

Re: Signature Update S601 Pulled From Cisco.com

hartkl5277 — Fri, 14 Oct 2011 21:21:41 GMT

One thing I have noticed in this thread is the only help being provided is from the community. NICKSMI seems to be the only Cisco representative responding to this thread and his comments make it sound like it is our fault for not running the latest update. But no were in the release notes of S601 does it say only supports 7.0.5 and above. After a coupe of days of this you would think that someone within Cisco would be able to supply some answers besides upgrade to 7.0.6. Some of us cannot because the S601 update screwed up the systems that bad. It would be nice just to hear Cisco admit that they screwed up on this one but hear it what you can do to recover. Come on and man up.

Re: Signature Update S601 Pulled From Cisco.com

m.vuckovic — Wed, 19 Oct 2011 20:04:34 GMT

A lot of work to do.

It seems that upgrades work OK but I have problems with one IPS 4240 device which hangs. All other non-problematic devices are

ASA-AIP type.

Any ideas why IPS ?

Thanks and best regards,

Marko

Re: Signature Update S601 Pulled From Cisco.com

twilcox — Wed, 19 Oct 2011 20:39:03 GMT

I had to open a TAC case because one of our AIP-SSM’s license files was corrupted. The other three AIP-SSMs I was able to fix and one of the suggestions was to use the session command from the firewall which save me a trip out to a remote site to do a reboot manually (like unplug the unit from the FW) so someone posted a note about resetting the IPS 4250 appliances from the console port which would be a good idea to try if you haven’t done so already.

All in all this issue cost me a couple of days of productive work. And I agree that the guys from Cisco on this page pretty much came across as blaming us as end users for not upgrading to the latest code. I mean we were a couple of revs back on 7.02 (e4) still in the same train of code; so I thought those comments were completely inappropriate especially since there was no prior warning or major service bulletin on the security bulletin that is emailed out; like the one I just received on S603 a few minutes ago. In other words, that is how you need to warn customers about major code bugs like this … that guy was acting like a corporate parrot, puppet and tool for blaming the customers. I know he pissed off a lot of customers.

Signature Update S601 Pulled From Cisco.com

m.vuckovic — Wed, 19 Oct 2011 21:07:23 GMT

Thank you very much for your comments. It seems that IPS 4240 is the only device that is having a problem, at least in my case. One sleepless night and tomorrow I'll open the case. For me whole week is strange so I'm in kind a 'just think positive' mantra but I can understand the frustration of many of you.