Should SSH sessions from Inside hosts to DMZ hosts survive ASA statefull failover?

[stevebohrer]

This is an issue I'm currently exploring with TAC, but I'd like a quick reality check. We have a pair of ASA 5510s in Active/Standby stateful failover mode. In some tests failing over from the active to the standby system breaks SSH connections from hosts on our Inside to hosts on our DMZs.

A specific example is our backup server on Inside which is connecting to our mail server in the DMZ2, and running ssh/rsync/scp for the backups. A running backup job fails with network timeout errors when I trigger the failover. Also, sometimes the mail server loses or hangs on its connection to our LDAP server in DMZ1, although sometimes this connection is fine (DMZ2 is more "inside" than DMZ1, and I assume the LDAP look ups are many short connections, vs the rsync backup being one long connection).

TAC has suggested that open SSH sesions will always fail when the ASAs failover. I believe this is true for management connections to the ASA, but I don't see why it should be the case for an SSH session through the ASA to a server in the DMZ. TAC has suggested that I open some connections to servers in the DMZ and test what happens, and I can do so this Wednesday morning during a maintenance window.

But, in general, is this true? That is, given an SSH session from a workstation to a server, should a failover break it? If so, why?

The setup is:

MyWorkStation-INSIDE -> CoreSwitch (vlan 10) -> [ ASA-INSIDE - - (ASA-internal-connection) - - ASA-DMZ ] -> CoreSwitch (vlan 3) -> TargetServer

That is, all our inside VLANs are routed by our core L2/3 switch to a VLAN that connects to the Primary and Secondary ASA's INSIDE ports. There are also seperate VLANS on the core for the ASA's DMZ1 and DMZ2 connections, which go to both ASAs and to any servers in these zones.

The description of the ASA Stateful failover

( http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_configuration_example09186a00807dac5f.shtml )

says:

"The state information passed to the standby unit includes these:

·         The NAT translation table
·         The TCP connection states
·         The UDP connection states
·         The ARP table
·         The Layer 2 bridge table (when it runs in the transparent firewall mode)

·         The HTTP connection states (if HTTP replication is enabled)

·         The ISAKMP and IPSec SA table

·         The GTP PDP connection database

The information that is not passed to the standby unit when stateful failover is enabled includes these:

·         The HTTP connection table (unless HTTP replication is enabled)

·         The user authentication (uauth) table

·         The routing tables

·         State information for security service modules

"

Honestly, I'm not quite sure what the ISAKMP and IPSec SA tables do, but shouldn't an SSH connection through the ASA be just a TCP connection?

Thanks for any insights, even simple ones like, "For us, SSH from Inside to hosts in the DMZ survives failover," or, "Yah, failover breaks all SSH sessions."

Steve Bohrer

[Phillip Strelau · Accepted solution]

Hi Stephen,

     Yup, thru the box ssh sessions are treated as regular tcp sessions. As long as you have stateful failover configured these connections should be replicated between conn tables. ISAKMP and IPSec SAs both have to do with vpn connections that are terminated on the local ASA and nothing to do with your ssh session. What may be happening is brief packet loss from which the tcp stack on your servers can not recover from. This would explain why every time a failover happened, the connection dies. A packet capture during a failover would be able to confirm the breakdown.

--Phil

http://www.cisco.com/en/US/docs/security/asa/asa82/command/reference/c1.html#wp2129312

[stevebohrer]

Thanks, Phil

Steve