Buy or Renew
Buy or Renew
Duo Security forums now LIVE! Get answers to all your Duo Security questions. Learn more
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
4399
Views
0
Helpful
12
Replies

Both IPS 4260 is down - Application Failed Critical

Rene Rolsted
Level 1
Level 1

‎10-12-2011 03:59 AM - edited ‎03-10-2019 05:30 AM

Tonight was both my 4260 IPS appliance boxes, down.

Health Status for Failed Applications                   Red

Both IPS appliance have both interfaces down.

Gi eth 2/0 and Gi Eth 2/1 running in Inline-Vlan-Pair

When I reboot one of the boxe, both interface go up, but after 1-2 min they go down.

Now I have put both boxes in bypass mode and then both of their interfaces are up

I suspect that it's the new signature version 6.01 which is to blame.

Is there anyone who has experienced this?

UDVIK01_AWG_IPS# sh ver

Application Partition:

Cisco Intrusion Prevention System, Version 7.0(4)E4

Host:

    Realm Keys          key1.0

Signature Definition:

    Signature Update    S601.0                   2011-10-10

OS Version:             2.4.30-IDS-smp-bigphys

Platform:               IPS-4260-K9

Serial Number:         

Licensed, expires:      10-Nov-2011 UTC

Sensor up-time is 2:39.

Using 1892966400 out of 4100345856 bytes of available memory (46% usage)

system is using 17.4M out of 38.5M bytes of available disk space (45% usage)

application-data is using 47.5M out of 166.8M bytes of available disk space (30% usage)

boot is using 41.6M out of 68.6M bytes of available disk space (64% usage)

application-log is using 494.0M out of 513.0M bytes of available disk space (96% usage)

MainApp            B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500   Running

AnalysisEngine     B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500   Running

CollaborationApp   B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500   Running

CLI                B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500

Upgrade History:

* IPS-sig-S600-req-E4       21:15:10 UTC Wed Oct 05 2011

  IPS-sig-S601-req-E4.pkg   21:15:10 UTC Tue Oct 11 2011

Recovery Partition Version 1.1 - 7.0(4)E4

Host Certificate Valid from: 01-Nov-2010 to 01-Nov-2012

UDVIK01_AWG_IPS# sh health

Overall Health Status                                   Red

Health Status for Failed Applications                   Red

Health Status for Signature Updates                     Green

Health Status for License Key Expiration                Yellow

Health Status for Running in Bypass Mode                Red

Health Status for Interfaces Being Down                 Green

Health Status for the Inspection Load                   Green

Health Status for the Time Since Last Event Retrieval   Green

Health Status for the Number of Missed Packets          Green

Health Status for the Memory Usage                      Green

Health Status for Global Correlation                    Green

Health Status for Network Participation                 Green

Security Status for Virtual Sensor IDS-UDVIK   Green

Security Status for Virtual Sensor IPS-Udvik   Green

Security Status for Virtual Sensor vs0         Green

UDVIK02_AWG_IPS# sh ver

Application Partition:

Cisco Intrusion Prevention System, Version 7.0(4)E4

Host:

    Realm Keys          key1.0

Signature Definition:

    Signature Update    S601.0                   2011-10-10

OS Version:             2.4.30-IDS-smp-bigphys

Platform:               IPS-4260-K9

Serial Number:         

Licensed, expires:      10-Nov-2011 UTC

Sensor up-time is 3:47.

Using 1893654528 out of 4100345856 bytes of available memory (46% usage)

system is using 17.4M out of 38.5M bytes of available disk space (45% usage)

application-data is using 47.6M out of 166.8M bytes of available disk space (30% usage)

boot is using 41.6M out of 68.6M bytes of available disk space (64% usage)

application-log is using 494.0M out of 513.0M bytes of available disk space (96% usage)

MainApp            B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500   Running

AnalysisEngine     B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500   Running

CollaborationApp   B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500   Running

CLI                B-BEAU_704_2010_JUL_21_15_57_7_0_3_29   (Ipsbuild)   2010-07-21T15:59:36-0500

Upgrade History:

* IPS-sig-S600-req-E4       21:45:10 UTC Wed Oct 05 2011

  IPS-sig-S601-req-E4.pkg   21:45:10 UTC Tue Oct 11 2011

Recovery Partition Version 1.1 - 7.0(4)E4

Host Certificate Valid from: 01-Nov-2010 to 01-Nov-2012

UDVIK02_AWG_IPS# sh health

Overall Health Status                                   Red

Health Status for Failed Applications                   Red

Health Status for Signature Updates                     Green

Health Status for License Key Expiration                Yellow

Health Status for Running in Bypass Mode                Red

Health Status for Interfaces Being Down                 Green

Health Status for the Inspection Load                   Green

Health Status for the Time Since Last Event Retrieval   Green

Health Status for the Number of Missed Packets          Green

Health Status for the Memory Usage                      Green

Health Status for Global Correlation                    Green

Health Status for Network Participation                 Green

Security Status for Virtual Sensor IDS-Udvik   Green

Security Status for Virtual Sensor IPS-Udvik   Green

Security Status for Virtual Sensor vs0         Green

Labels:
0 Helpful
12 Replies 12

whaydenscana
Level 1
Level 1

‎10-12-2011 05:50 AM

I am having the same problem. I so far have 4 IPS down.  They all crashed after 601 signature update.  If you hadn't updated your signatures don't and turn off auto update.

I have two open TAC cases about this issue.

Also upgrading/downgrading isn't working right now....just freezes.....

this is bad.

0 Helpful

mik.gustafsson
Level 1
Level 1
In response to whaydenscana

‎10-12-2011 06:10 AM

Reset worked for me. Im on my last appliance now, 6 in total, and all are up and running again.

0 Helpful

Rene Rolsted
Level 1
Level 1
In response to mik.gustafsson

‎10-12-2011 06:16 AM

Hi Mikael,

I did a reset in CLI this morning

but this is not working for me

0 Helpful

Rene Rolsted
Level 1
Level 1
In response to whaydenscana

‎10-12-2011 06:13 AM

Hi William,

Okay, I'm waiting for Cisco comes up with a solution.

I have contacted Cisco Denmark with the error.

I had set myself to do a downgrade of the signature, but I wait.

thanks for your reply

0 Helpful

mik.gustafsson
Level 1
Level 1
In response to Rene Rolsted

‎10-12-2011 06:42 AM

Hi Rene,

All but one did come back to normal after a reset.

So not a 100% solution unfortunately.

0 Helpful

hartkl5277
Level 1
Level 1

‎10-12-2011 07:55 AM

Having the same issue with S601, reset did not work on either unit.  Glad I only updated in the non-production environment.  Hope a fix comes out soon.

0 Helpful

whaydenscana
Level 1
Level 1

‎10-12-2011 08:21 AM

Cisco respone..it is a bug.

"

Hi William,

I have got a response from the collaboration team. It seems that there is a bug , we have the following response from the developers

“Due to a bug in IPS software versions prior to 7.0.6, it is not advised to apply signature updates past S601 without having upgraded to 7.0.6 first.If a customer has applied S601 with no problems, there is no issue beyond having to upgrade to 7.0.6 to apply updates S602 and later.  If you have not applied S601 yet, it is advised to upgrade to 7.0.6 before applying update S602 and later.”

I am still finding some more details and would get back to you.

"

I am currently downgrading and having success. 

Downgrade from console

....unplug monitor port (prevent engine from running) .... reset IPS.....downgrade from command line to 600....go to gui turn off auto update....plug ports back in.

Downgrade from ssh

shutdown monitor port (prevent engine from running) .... reset IPS.....downgrade from command line to 600....go to gui turn off auto update....plug ports back in.

Thanks,

Will

0 Helpful

rrfield
Level 1
Level 1

‎10-12-2011 08:43 AM

In our environment this is limited to our single 4260 sensor.  We have multiple SSM-10 modules in ASA's and they are all functioning properly. 

Is this the case for others as well?

0 Helpful

whaydenscana
Level 1
Level 1
In response to rrfield

‎10-12-2011 09:34 AM

I am only working with 4240s here.  I am not familar with the modules in the ASA.

0 Helpful

hoyleanderson
Level 1
Level 1
In response to whaydenscana

‎10-12-2011 10:23 AM

Same issue here, on both sensors and modules, though not on EVERY device.

0 Helpful

mulhollandm
Level 1
Level 1

‎10-12-2011 01:00 PM

folks

aip-ssm-20 same problem

if i rest it runs ok for a day or so then stops logging though curiously when i reset the ips module my asa cluster fails over

didn't expect that

i'm running in promiscuous mode but i think i'll upgrade

currently running

Cisco Intrusion Prevention System, Version 7.0(2)E4

Host:                                                 
    Realm Keys          key1.0                               
Signature Definition:                                        
    Signature Update    S601.0                   2011-10-10  

Upgrade History:

* IPS-sig-S600-req-E4       17:37:23 UTC Wed Oct 05 2011  
  IPS-sig-S601-req-E4.pkg   18:37:09 UTC Tue Oct 11 2011  

i'll update when i upgrade and get a signature update

0 Helpful

ABDUL MAJID KHAN
Level 1
Level 1

‎11-12-2011 04:47 AM

Folks,

This is a known BUG with ID (CSCti79423) in the version 7.0 (4) E4 that will be caused once the global correlation is configured. It is fixed in 7.0 (6) E4.

For details find the following URL;

   http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCti79423

Workaround:

Disable global correlation:

conf t

service global-correlation

global-correlation-inspection off

exit and accept changes

Restart the IDSM-2

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents
Review Cisco Networking products for a $25 gift card