cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
12291
Views
12
Helpful
5
Replies

ASA with IPS has control channel communications failure

cstockwe
Level 1
Level 1

ASA5520 with AIP-SSM-20 IPS module

IPS software: 7.0(1)E3

ASA software: 8.0(3)

We have two of these ASA's set up in a failover pair.

The problem seems to be at random times that the primary ASA/IPS member will failover to secondary with:

%ASA-3-323001: Module in slot 1 experienced a control channel communication failure.

The 'sh module' shows that the 'failed' ASA thinks that its IPS is not responsive:

Mod SSM Application Name Status SSM Application Version

--- ------------------------------ ---------------- --------------------------

1 IPS Not Applicable 7.0(1)E3

Mod Status Data Plane Status Compatibility

--- ------------------ --------------------- -------------

0 Up Sys Not Applicable

1 Unresponsive Not Applicable

The way to recover this is to reload the failed ASA. Interestingly, when a reload is done on the ASA and after it is back online, if you session into the IPS module the uptime is still several days - and it seems to function normally.

We've logged a tac case for this, and it is ongoing. The tac has replaced the IPS module twice but the problem remains - it certainly seems like the issue is with the ASA and not the IPS module.

We are currently running a 'debug cplane 255' on the console of the ASA and we are awaiting another failure - which could happen anytime up to 2 weeks from the last failure. Has anyone seen anything like this before and possibly could offer any advice?

Thanks

5 Replies 5

Hi Cameron,

Unfortunately I don't recall the exact circumstances, but I remember seeing a similar issue. The problem ended up being the backplane connector on the ASA itself. As in your case, the SSM was replaced a couple of times but it turned out that the entire ASA needed to be replaced due to the bad backplane.

Hope that helps.

-Mike

vmoopeung
Level 5
Level 5

1)Console up to the ASA and start capturing all messages and logs etc

2)First try issuing a "hw module 1 reset" which will power cycle on the card

3)If still the same issue, physically reset the module.

4)If the above steps do not work, you need to recover the module.

Step 3 Configure the recovery settings for ASA-SSM:

asa (enable)# hw-module module 1 recover configure

Note If you make an error in the recovery configuration, use the hw-module module 1 recover stop command to stop the system reimaging and then you can correct the configuration. The link below provides additional details.

http://www.cisco.com/en/US/docs/security/ips/5.0/configuration/guide/cli/clissm.html

cstockwe
Level 1
Level 1

Had to wait a while to get another failure but it happened again on the weekend. I'll take this back to the tac for advice (debug cplane 255 output below).

Thanks for your help guys.

=======

Aug 28 2009 08:33:27: %ASA-3-216001: internal error in es_PostEvent: invalid descriptor

Aug 28 2009 08:33:31: %ASA-3-323001: Module in slot 1 experienced a control channel communication failure.

cp_transport_client_disconnect: closing socket 3

cp_transport_connection_handler: client at socket 3 disconnectedAug 28 2009 08:33:31: %ASA-4-411002: Line protocol on Interface Outside, changed state to down

Aug 28 2009 08:33:31: %ASA-4-411002: Line protocol on Interface Inside, changed state to down

Aug 28 2009 08:33:31: %ASA-4-411001: Line protocol on Interface Outside, changed state to up

Switching to Standby

Aug 28 2009 08:33:31: %ASA-4-411001: Line protocol on Interface Inside, changed state to up

Aug 28 2009 08:33:31: %ASA-1-104002: (Primary) Switching to STNDBY - Other unit wants me Standby. Secondary unit switch reason: Service card in other unit has failed.

Aug 28 2009 08:33:31: %ASA-1-105003: (Primary) Monitoring on interface Outside waiting

Aug 28 2009 08:33:31: %ASA-1-105003: (Primary) Monitoring on interface Inside waiting

Switching to Failed state.

Aug 28 2009 08:33:31: %ASA-1-104003: (Primary) Switching to FAILED.

cp_connect: Connecting to card 1, socket 3, port 7000

cp_transport_connection_handler: listening on socket 3

cp_socket_read_looped: Error during socket read

cp_transport_connection_handler: client at socket 3 disconnected

cp_transport_client_connect: Spawned thread c748a048 for new connection

cp_transport_client_connect: Created new connection with id 0

cp_update_connection: Error updating connection_id 0

cp_connect: Connecting to card 1, socket 3, port 7000

cp_transport_connection_handler: listening on socket 3

cp_transport_client_connect: Spawned thread c748bfe4 for new connection

cp_transport_client_connect: Created new connection with id 0

cp_transport_client_disconnect: closing socket 3

cp_transport_connection_handler: client at socket 3 disconnected

cp_connect: Connecting to card 1, socket 3, port 7000

cp_transport_connection_handler: listening on socket 3

cp_transport_client_connect: Spawned thread c7488288 for new connection

cp_transport_client_connect: Created new connection with id 0

Aug 28 2009 08:33:49: %ASA-3-323001: Module in slot 1 experienced a control channel communication failure.

======

I've had this happen to ASA models 5510 and 5520 with either ssm-10 or ssm-20 ips modules. I was told by Cisco the solution is to upgrade to version 7.0(1)E3 which I have done and still get that problem. I was then told it may be an issue of the ips module being oversubscribed and to adjust the acl that diverts traffic to the ips module. I've done that and still I have issues with it failing. I think next time I'll demand they send another firewall w/ ips module and let someone else figure out why it happens.

Just as an update - we've had our ASA replaced and have not had a problem since (we still have the same IPS modules installed in the new chassis).

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: