AIP-SSM module hung

[sbgcsd]

I have recently confgured my AIP-SSM-20 module in my firewalls (ASA 5540) which are configured in HA(Active/Standby).This implementation i have done on 13th June. It was working fine.

Now, i have observerd that the AIP-SSM-20 module in the primary firewall had gone to unresponsive state.

Below is the status of show module and show failover command.

FW1-5540# sh module

Mod Card Type                                    Model              Serial No.
--- -------------------------------------------- ------------------ -----------
  0 ASA 5540 Adaptive Security Appliance         ASA5540            JMX1234L11F
  1 ASA 5500 Series Security Services Module-20  ASA-SSM-20         JAF1341ADPS

Mod MAC Address Range                 Hw Version   Fw Version   Sw Version
--- --------------------------------- ------------ ------------ ---------------
  0 0021.d871.77ab to 0021.d871.77af  2.0          1.0(11)4     8.0(3)6
  1 0023.ebf6.11ce to 0023.ebf6.11ce  1.0          1.0(11)5     6.2(2)E4

Mod SSM Application Name           Status           SSM Application Version
--- ------------------------------ ---------------- --------------------------
  1 IPS                            Not Applicable   6.2(2)E4

Mod Status             Data Plane Status     Compatibility
--- ------------------ --------------------- -------------
  0 Up Sys             Not Applicable
  1 Unresponsive       Not Applicable

FW1-5540# sh failover
Failover On
Failover unit Primary
Failover LAN Interface: FAILOVER GigabitEthernet0/2 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 3 of 250 maximum
Version: Ours 8.0(3)6, Mate 8.0(3)6
Last Failover at: 09:06:14 UTC Jun 15 2010
        This host:

                This host: Primary - Failed
                Active time: 191436 (sec)
                slot 0: ASA5540 hw/sw rev (2.0/8.0(3)6) status (Up Sys)
                  Interface DMZ_LAN (10.192.153.13): Normal (Waiting)
                  Interface INTRANET (10.192.154.13): Normal (Waiting)
                  Interface management (0.0.0.0): Link Down (Waiting)
                slot 1: ASA-SSM-20 hw/sw rev (1.0/6.2(2)E4) status (Unresponsive/Down)
                  IPS, 6.2(2)E4, Not Applicable
        Other host: Secondary - Active
                Active time: 192692 (sec)
                slot 0: ASA5540 hw/sw rev (2.0/8.0(3)6) status (Up Sys)
                  Interface DMZ_LAN (10.192.153.5): Unknown (Waiting)
                  Interface INTRANET (10.192.154.5): Unknown (Waiting)
                  Interface management (0.0.0.0): Unknown (Waiting)
                slot 1: ASA-SSM-20 hw/sw rev (1.0/7.0(2)E4) status (Up/Up)
                  IPS, 7.0(2)E4, Up

Stateful Failover Logical Update Statistics
        Link : Unconfigured.

I have tried using the

hw-module module 1 reset

to reset the IPS module but the status is always unresponsive.

Its production environment where i cannnot expirement much. Ned help to rectify the problem.

[Scott Fringer]

It may be necessary to power-off the ASA to force a full power-off reset of the AIP-SSM.

Another option is to re-image the AIP-SSM to factory defaults as outlined here:

http://www.cisco.com/en/US/docs/security/ips/6.2/configuration/guide/cli/cli_system_images.html#wp1230355

If those options do not correct the issue, it would be best to open a service request with TAC to allow further troubleshooting to occur.

Scott

[sbgcsd]

7.0.2(E4)

Hi Scott,

Thanks a for yours suggestion. As this device is located in US while i am managing it from India, its not possible for me to power on and power off the device. But i did tried using the reload command on the ASA box.

The state is same as unresponsive.

Then i tried loading re-image to AIP-SSM using TFTP to the IPS Engine version 7.0.2(E4).

I also turned on the debug module-boot command to see the error then i got this error messages on the ASA

Slot-1 181> Received 30562532 bytes
Slot-1 182> Bad magic number (0x-1abac777)
Slot-1 183> Rebooting due to Autoboot error ...

Then i stop the recovery of the AIP-SSM20 and then again it went into unresponsive state.

Any further sugeestion which can be helpful.

[Scott Fringer]

Unfortunately the reload command on the ASA does not power-off the AIP-SSM, so it will not fully reset the module.

Did you use the correct re-image file for the exact model AIP-SSM you have installed (an AIP-SSM-20)?  The filename should be:

IPS-SSM_20-K9-sys-1.1-a-7.0-2-E4.img

The file size you indicate was transferred is 30562532 bytes, but the above file is 29510002 bytes.  It does not appear the correct file was used, and in turn the error you received.

Scott

[zohaib.shabir]

Hello all,

My SSM Module is in unresponsive state now how can i reimage it bcoz its not accepting ip address .

Is there is any to assign ip address to SSM Module while its in unresponsive state ????????

[johflore]

Zohaib,

Re-image process will assign IP address to your unresponsive module. Is some kind of ROMMON for modules.

You can refer following link:

http://www.cisco.com/c/en/us/support/docs/security/intrusion-prevention-system/116155-configure-product-00.html

Johan.

[sawgupta]

You can specify the IP address via ASA using the CLI

hw-module module 1 recover configure

It will prompt for "Port IP Address [0.0.0.0]:"

http://www.cisco.com/en/US/docs/security/ips/7.0/installation/guide/hw_system_images.html#wp1231447

Thanks & Regards,

Sawan Gupta

[leolmelo_jcl]

Hi Scott, 

I have almost same problem of sbgcsd in my customer. I'm deploying two ASA-5512 in failover configuration. One day, after almost 2 months testing project in a lab, when we install in customer's datacenter the systems presented following errors:

  ciscoasa2(config)# failover

        Detected an Active mate

  ciscoasa2# Mate NOT PRESENT card in slot 1 is different from mine IPS5512

I tried to discover what was happened with IPS modulo, then I saw error in IPS status: "Unresponsive".

  ciscoasa2# sh module ips

  Mod  Card Type                                    Model              Serial No.
  ---- -------------------------------------------- ------------------ -----------
   ips Unknown                                      N/A                FCH1712J7UL

  Mod  MAC Address Range                 Hw Version   Fw Version   Sw Version
  ---- --------------------------------- ------------ ------------ ---------------
   ips 7cad.746f.8796 to 7cad.746f.8796  N/A          N/A 

  Mod  SSM Application Name           Status           SSM Application Version
  ---- ------------------------------ ---------------- --------------------------
   ips Unknown                        No Image Present Not Applicable  

  Mod  Status             Data Plane Status     Compatibility
  ---- ------------------ --------------------- -------------
   ips Unresponsive       Not Applicable 

  Mod  License Name   License Status  Time Remaining
  ---- -------------- --------------- ---------------
   ips IPS Module     Disabled        perpetual

According with Cisco Foruns I tried to "Reloading, Shutting Down, Resetting, and Recovering AIP-SSM" (*) using "hw-module module " command. But unfortunatelly ASA didn't accept this command. See below:

  ciscoasa2# hw-module module 1 reload
             ^
  ERROR: % Invalid input detected at '^' marker

What happened with this command (hw-module) ? Maybe is a problem in Software version ? When I entered "sh flash" command I saw that didn't exist any software for AIP-SMM module:

  ciscoasa2# sh flash
  --#--  --length--  -----date/time------  path
   11  4096        Sep 12 2013 13:56:54  log
   21  4096        Sep 12 2013 13:57:10  crypto_archive
  100  0           Sep 12 2013 13:57:10  nat_ident_migrate
   22  4096        Sep 12 2013 13:57:10  coredumpinfo
   23  59          Sep 12 2013 13:57:10  coredumpinfo/coredump.cfg
  101  34523136    Sep 12 2013 14:00:14  asa861-2-smp-k8.bin
  102  17851400    Sep 12 2013 14:04:36  asdm-66114.bin
  103  38191104    Apr 24 2014 12:59:58  asa912-smp-k8.bin
  104  6867        Apr 24 2014 13:01:20  startup-config-jcl.txt
  105  24095116    Jun 17 2014 14:54:14  asdm-721.bi

But another ASA (#1) have image:

ciscoasa1# sh flash
--#--  --length--  -----date/time------  path
   11  4096        Sep 10 2013 06:42:56  log
   21  4096        Apr 17 2014 03:13:12  crypto_archive
  123  5276864     Apr 17 2014 03:13:12  crypto_archive/crypto_eng0_arch_1.bin
  110  0           Sep 10 2013 06:43:12  nat_ident_migrate
   22  4096        Sep 10 2013 06:43:12  coredumpinfo
   23  59          Sep 10 2013 06:43:12  coredumpinfo/coredump.cfg
  111  34523136    Sep 10 2013 06:44:24  asa861-2-smp-k8.bin
  112  42637312    Sep 10 2013 06:45:46  IPS-SSP_5512-K9-sys-1.1-a-7.1-4-E4.aip <===

But I am not sure if this image is really the right image do AIP-SSM in ASA#2. But anyway I copy (through a simple TFTP server) from ASA#1 to ASA#2 , but after this, the same problem ramained ! 

Because I didn't applied the Failover condition to system. 

What can I do now ?

Thank you very much in advance.

Leonardo_Melo.(CCAI-JCL-Brazil).