topic You have an ASA 5520 with the in Network Security

Cisco ASA IPS Detect service card failure Service card in other unit has failed

teatrodelsogno — Tue, 12 Mar 2019 05:35:25 GMT

Hi Guys,

I have a strange problem in a part connected to one cisco asa bug, but is not belonging to my version.

I have one cluster active/standby failover and in random time this is the behavior that is happening:

From State To State Reason
==========================================================================
12:48:10 CEST Mar 4 2015
Just Active Active Drain Service card in other unit has failed

12:48:10 CEST Mar 4 2015
Active Drain Active Applying Config Service card in other unit has failed

12:48:10 CEST Mar 4 2015
Active Applying Config Active Config Applied Service card in other unit has failed

12:48:10 CEST Mar 4 2015
Active Config Applied Active Service card in other unit has failed

12:59:04 CEST Mar 4 2015
Active Standby Ready Other unit wants me Standby

12:59:05 CEST Mar 4 2015
Standby Ready Failed Detect service card failure

12:59:11 CEST Mar 4 2015
Failed Standby Ready My service card is as good as peer

12:59:11 CEST Mar 4 2015
Standby Ready Just Active Service card in other unit has failed

12:59:11 CEST Mar 4 2015
Just Active Active Drain Service card in other unit has failed

12:59:11 CEST Mar 4 2015
Active Drain Active Applying Config Service card in other unit has failed

12:59:11 CEST Mar 4 2015
Active Applying Config Active Config Applied Service card in other unit has failed

12:59:11 CEST Mar 4 2015
Active Config Applied Active Service card in other unit has failed

13:03:07 CEST Mar 4 2015
Active Standby Ready Set by the config command

13:03:55 CEST Mar 4 2015
Standby Ready Failed Detect service card failure

13:06:38 CEST Mar 4 2015
Failed Standby Ready My service card is as good as peer

13:10:15 CEST Mar 4 2015
Standby Ready Just Active Other unit wants me Active

13:10:15 CEST Mar 4 2015
Just Active Active Drain Other unit wants me Active

13:10:15 CEST Mar 4 2015
Active Drain Active Applying Config Other unit wants me Active

13:10:15 CEST Mar 4 2015
Active Applying Config Active Config Applied Other unit wants me Active

13:10:15 CEST Mar 4 2015
Active Config Applied Active Other unit wants me Active

I try to manually reset the IPS module with hw-module module 1 reset but I'm receiving the same messages.

Some other information:

Cisco Adaptive Security Appliance Software Version 8.2(1)

Mod Card Type                                    Model              Serial No.
--- -------------------------------------------- ------------------ -----------
0 ASA 5520 Adaptive Security Appliance         ASA5520            JMX1414L0X3
1 ASA 5500 Series Security Services Module-10 ASA-SSM-10         JAB10070GJP

Mod MAC Address Range                 Hw Version   Fw Version   Sw Version
--- --------------------------------- ------------ ------------ ---------------
0 0026.99c0.c0df to 0026.99c0.c0e3 2.0          1.0(11)2     8.2(1)
1 0015.c6fa.3b31 to 0015.c6fa.3b31 1.0          1.0(10)0     7.1(8)E4

Mod SSM Application Name Status SSM Application Version
--- ------------------------------ ---------------- --------------------------
1 IPS Up 7.1(8)E4

Mod Status             Data Plane Status     Compatibility
--- ------------------ --------------------- -------------
0 Up Sys             Not Applicable
1 Up                 Up

------------------ show memory ------------------

Free memory:      1722778808 bytes (80%)
Used memory:       424704840 bytes (20%)
-------------     ----------------
Total memory:     2147483648 bytes (100%)

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Failover On
Failover unit Primary
Failover LAN Interface: faillink Management0/0 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 2 of 250 maximum
Version: Ours 8.2(1), Mate 8.2(1)
Last Failover at: 13:10:15 CEST Mar 4 2015
   This host: Primary - Active
       Active time: 665238 (sec)
       slot 0: ASA5520 hw/sw rev (2.0/8.2(1)) status (Up Sys)
          Interface outside (x.x.x.x): Normal
          Interface inside (10.254.0.3): Normal
          Interface WIFI_no (10.254.14.1): Normal (Not-Monitored)
          Interface DMZ_dsoi (172.16.1.1): Normal (Not-Monitored)
          Interface Adfafd (x.x.x.x): No Link (Not-Monitored)
       slot 1: ASA-SSM-10 hw/sw rev (1.0/7.1(8)E4) status (Up/Up)
          IPS, 7.1(8)E4, Up
   Other host: Secondary - Standby Ready
       Active time: 70225 (sec)
       slot 0: ASA5520 hw/sw rev (1.1/8.2(1)) status (Up Sys)
          Interface outside (81.208.53.221): Normal
          Interface inside (10.254.0.4): Normal
          Interface WIFI_no (0.0.0.0): Normal (Not-Monitored)
          Interface DMZ_dsoi (172.16.1.3): Normal (Not-Monitored)
          Interface Adfafd (85.20.9.11): Normal (Not-Monitored)
       slot 1: ASA-SSM-10 hw/sw rev (1.0/7.1(8)E4) status (Up/Up)
          IPS, 7.1(8)E4, Up

Stateful Failover Logical Update Statistics
   Link : faillink Management0/0 (up)
   Stateful Obj    xmit       xerr       rcv        rerr
   General       483879055 0          14089818   10606
   sys cmd     97977      0          97977      0
   up time     0          0          0          0
   RPC services     0          0          0          0
   TCP conn    97196877   0          4327486    1514
   UDP conn    386494404 0          9650250    9092
   ARP tbl     45801      0          4968       0
   Xlate_Timeout     0          0          0          0
   VPN IKE upd    17191      0          1784       0
   VPN IPSEC upd    26805      0          7353       0
   VPN CTCP upd    0          0          0          0
   VPN SDI upd    0          0          0          0
   VPN DHCP upd    0          0          0          0
   SIP Session    0          0          0          0

   Logical Update Queue Information
         Cur    Max    Total
   Recv Q:    0    25    16395021
   Xmit Q:    0    111    505815152

Could you please send me some suggestion or share past experience about it?

Many regards

Matteo

There are few know bugs:

Tushar Bangia — Thu, 05 Mar 2015 05:29:38 GMT

There are few know bugs:

https://tools.cisco.com/bugsearch/bug/CSCun81616/?reffering_site=dumpcr

https://tools.cisco.com/bugsearch/bug/CSCun82492/?reffering_site=dumpcr

https://tools.cisco.com/bugsearch/bug/CSCun81616/?reffering_site=dumpcr

Analyzing logs would help to identify the cause of the issue.

Regards,

Tushar Bangia

Note - Please do rate post if you find it helpful!!

Hi Tushar,correct, I found

teatrodelsogno — Thu, 05 Mar 2015 08:19:01 GMT

Hi Tushar,

correct, I found also these bugs but as you can see, are different affected release version than mine.

BTW, shall we consider that also other version could be affected? (like mine --> 8.2(1))

And the workaround proposed by bugs, is little bit "crazy". Cause the workaround is only collect logs from syslog?! In which way the collection of the logs could solve the problem? 🙂

Any council update from 8.2 could be "flagged" like stable with IPS module installed in the ASA?

Regards

Matteo

Hi,Such issues are generally

Tushar Bangia — Thu, 05 Mar 2015 11:14:28 GMT

Hi,

Such issues are generally reported because of AIP SSM failure. Please check the sanity of module via "sh module x details".
The issue can be caused because of oversubscription of the sensor module hence advise you to create an ACL to limit the inspected traffic.

Please try below and see if the module comes up as expected:

hw-module module 1 shutdown and then hw-module module 1 reset.

Alternatively, you can raise a case with Cisco TAC for replacement of module.

Regards,

Tushar Bangia

thanks..yes, this I forgot to

teatrodelsogno — Thu, 05 Mar 2015 11:49:26 GMT

thanks..

yes, this I forgot to attach, but also the module seems in a good health state:

show module 1 details

Getting details from the Service Module, please wait...

ASA 5500 Series Security Services Module-10

Model: ASA-SSM-10

Hardware version: 1.0

Serial Number: JAB10070GJP

Firmware version: 1.0(10)0

Software version: 7.1(8)E4

MAC Address Range: 0015.c6fa.3b31 to 0015.c6fa.3b31

App. name: IPS

App. Status: Up

App. Status Desc: Normal Operation

App. version: 7.1(8)E4

Data plane Status: Up

Status: Up

Mgmt IP addr: xx.x.x.x

Mgmt Network mask: 255.255.255.0

Mgmt Gateway: x.x.x.x

Mgmt Access List: 1x.x.x.x

Mgmt Access List: 10.x.x.x.x

Mgmt Access List: 10x.x.x.x

Mgmt Access List: 10.x.x.x.x

Mgmt Access List: 1x.x.x.x

Mgmt Access List: 10.x.x.x.x

Mgmt web ports: 443

Mgmt TLS enabled: true

I already tried module reset...

yes, the only way maybe is the TAC.

Thanks the same, I will send an update also here or in case in the while, let me know if you have some other ideas.

Regards

While your version 8.2(1)

Marvin Rhoads — Thu, 05 Mar 2015 14:58:53 GMT

While your version 8.2(1) might not be specifically listed in the BugIDs, it is very old and there are many many updates released since then. At a minimum I would upgrade to the last 8.2(x) release - 8.2(5) maintenance release or 8.2(5.13) interim.

Hi Marvin,yes, you are right.

teatrodelsogno — Thu, 05 Mar 2015 14:58:54 GMT

Hi Marvin,

yes, you are right.

I have to follow some other particularly steps to upgrade the firewall with SSM (IPS) module on board?

Or steps to follow are the same like the "normal" upgrading on the ASA appliance?

Many regards in advance.

Matteo

Hi,I think it is clear from

Vibhor Amrodia — Thu, 05 Mar 2015 15:02:41 GMT

Hi,

I think it is clear from the ASA Fail-over history that the ASA IPS module is causing the fail-over events on the ASA HA pair.

That can be due to various reasons on the IPS module.

I would recommend you open a TAC to find the root cause of this issue on the IPS.

Thanks and Regards,

Vibhor Amrodia

Matteo,Upgrading the ASA is

Marvin Rhoads — Thu, 05 Mar 2015 15:10:17 GMT

Matteo,

Upgrading the ASA is the same process with or without the SSM installed. Most of the type of bugs you are seeing are relate to the base ASA software, not the IPS version.

Of course once you get the ASA updated it would not hurt to also have the IPS SSM up to the latest release. You have 7.1(8)E4 from October 2013 and there is a version 7.1(9)E4 from October 2014 available for that platform.

IPS download

IPS Release notes

Hi Marvin,thanks once again

teatrodelsogno — Thu, 05 Mar 2015 15:27:04 GMT

Hi Marvin,

thanks once again for your support.

Ok, I can think to upgrade the ASA...

Regarding the IPS, I can't with the software release by you suggested.
Cause ASA5520 doesn't support this version.

I Vibhor,thanks for your

teatrodelsogno — Thu, 05 Mar 2015 15:28:37 GMT

I Vibhor,

thanks for your answer.

Yes exactly. In the meantime I opened one TAC, but I was curious to receive in any case some feedback from the community about this problem.

Regards

You have an ASA 5520 with the

Marvin Rhoads — Thu, 05 Mar 2015 15:33:35 GMT

You have an ASA 5520 with the IPS on an SSM-10. The release notes list:

"ASA 5500 AIP SSM-10".

5500 means 5500 series - 5510, 5520 etc.

ah yes sorry, you are right.

teatrodelsogno — Thu, 05 Mar 2015 15:40:17 GMT

ah yes sorry, you are right.

regards

Ok,TAC Support answeredAs

teatrodelsogno — Thu, 05 Mar 2015 17:04:53 GMT

Ok,

TAC Support answered

As Marvin said. One upgrade of the asa is high suggested.

BTW in other, in one my troubleshooting, I found these different:

NOTE: Primary is the one with problem

Primary:

5520

ips FW version 1.0(10)

Secondary:

5520-k8

ips FW version 1.0(11)

I think that is not connected to my problem ( k8 is just the different belonging to the license) and FW version could be different on the same model? ...

BTW, do you have some other feedback about this?

Regards

The "ips FW" (Firmware)

Marvin Rhoads — Thu, 05 Mar 2015 21:23:34 GMT

The "ips FW" (Firmware) versions do not need to be identical. That is most likely due to different manufacturing dates.

Let's see if the issue persists after you have upgraded your ASA software.

Hi,If you are planning for an

Vibhor Amrodia — Fri, 06 Mar 2015 01:24:02 GMT

Hi,

If you are planning for an upgrade for the ASA code , I would recommend an upgrade to ASA 8.2.5 latest interim to pick up the fix for this Defect as well:- CSCts98806

Thanks and Regards,

Vibhor Amrodia

Hi all,thanks for the support

teatrodelsogno — Fri, 06 Mar 2015 08:48:04 GMT

Hi all,

thanks for the support once again.

I will proceed with the upgrade.

Anyway I checked the list of interim software and I found also asa825-55-k8.bin version. (55).

There is any particularly reason why you suggested (33)?

Thanks and regards

Matteo

Hi,No , Actually i just

Vibhor Amrodia — Mon, 09 Mar 2015 09:51:18 GMT

Hi,

No , Actually i just wanted to recommend a Code which fixes the defect that i listed earlier.

Thanks and Regards,

Vibhor Amrodia

Hi ALL,guys, confirmed.The

teatrodelsogno — Mon, 16 Mar 2015 14:24:26 GMT

Hi ALL,

guys, confirmed.

The update to 825(55) solve this problem!

In case of similar problem to other devices or version, please try to follow this street first!

Bye!

Rate me if this post has been usefully.