ASA Failover Interface Principles

[Gerald Wiltse]

I'm about to implement my first 5520's, and I have questions about the details which aren't really explained in the documentation. My situation is active/standby.

Failover Link:

1.  Etherchannel is now possible on the ASA's, but is it supported on the Failover link?  I sure hope so.

2.  What is the REAL FUNCTIONAL difference between the failover link going down when using a switch versus a crossover cable between ASA's?  Specifically speak to what happens from a detection/reaction perspective in the IOS on both sides.

3. If the failover link is down on either side (or both), do the ASA's still communicate across their inside/outside interfaces to prevent a conflict?  How does it work exactly.

I've read a lot of documentation, and am very familar with interface tracking for HSRP/VRRP, routing protocols, and even IPSLA tracking and logic for failover from my routing and switching experience. Still, I'm having a hard time understanding how the ASA's will behave.

I am looking for a unique and verbose explanation from people who have worked with the ASA's exhaustively (in their own words).  Please do not post links to any cisco documentation, nor post statements that begin with "I think" or "I believe".

Thanks,

Jerry

[andrew.prince]

See the below links

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_configuration_example09186a00807dac5f.shtml

http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a0080834058.shtml

Sent from Cisco Technical Support iPad App

[Gerald Wiltse]

Thanks for taking a shot Andrew, but this did not help. I realize you were working from your IPAD, so I'll cut you some slack on not reading my whole post and providing a helpful answer.

Plus, I appreciate anyone who tries to help.

Jerry

[Jon Marshall]

Gerald

1) can't answer this but i suspect not.

2) Main difference is that if you are using a crossover cable if one end fails then the other end goes down as well. This means when you login to the firewalls you can't necessarily tell which firewall interface went down first.

Obviously the main advantage is that you are not reliant on another device ie. a switch to stay up as well ie. with a crossover cable only a failure on the firewalls is relevant rather than a switch also.

The general recommendation is to use a switch.

3) Yes they will communicate over the other interfaces, provided you have ip addresses on the standby interfaces, which you don't acually have to. If they can communicate over another interface the firewalls will not failover but the failover link will be seen as failed.

Jon

[Gerald Wiltse]

Thanks Jon for the response. 

1) I guess I will add "I suspect" to my list next time. Hopefully someone from Cisco will say for sure whether or not etherchannel is supported.

2) Good answer, I'm hoping that's the only one. I really want to see if anybody else can point out even one more significant difference. In my opinion, the "which interface went down first" is irrelevant and a terrible reason to insert a switch into the equation. In fact, there is still no logic behind this reason at all... it's nonsenical unless we're still missing something else that's huge. Otherwise... the emperor has no clothes people. Switch = BAD IDEA.

3) Ok this is really helpful. So... the OUTSIDE and INSIDE interfaces each get their standard "virtual" IP and mac which are used for normal operation. You're saying that putting optional "secondary" IP's on each of the interfaces on both the active and standby ASA's allows them to use these links to communicate with each other over.  This seems like a really good practice.  In theory, does this eliminate the need for the standby link altogether? 

Thanks,

Jerry

[Jon Marshall]

Jerry

I guess I will add "I suspect" to my list next time

Yes, i thought i might be pushing my luck but then again if you had included it i wouldn't have answered the other questions

2) Like i say the Cisco recommendation is to use a switch and it is mainly to help troubleshooting but it is just as valid to use a cross over if that is what you wan to.

3) No it doesn't eliminate the link because remember you are not just sending keepalives between the firewalls you are also replicating state information, for which you should use a separate cable. So you need failover cables to send this info.

The idea of interface monitoring is so that you can failover even if the active firewall is still up but an important interface on the firewall has failed. For example when we ran the FWSM we had many interfaces in one context with many DMZs. Some of these really weren't that important and we didn't bother monitoring them but we did monitor the outside interface because if that went down all access through the FWSM was lost.

Jon

[Gerald Wiltse]

Jon,

Thanks Again,

2) What I'm saying is that the Cisco recommendation is almost certainly terrible... and I really want to call it out here and have some guys FROM CISCO who know this topic inside and out weight in to confirm or deny my points. Most importantly, if there's a GOOD reason that we've missed, I want it to be revealed here. Thats the goal of making this noise.  It's very unlike Cisco to make a bad recommendation.

3)  Ok, keepalives and monitoring aside... the question is... why can't they sync across the other two interfaces?  The bandwidth of the synchronization is tiny, and maybe there's a latency requirement of "must be less than 10 ms".  Ok... neither of those are problems... so is there another reason?

Any super-technical Cisco guys want to weigh in?

[praprama]

Hi Gerald,

1) Etherchannelling is supported on the failover link. Here's the link to the documentation:

http://www.cisco.com/en/US/docs/security/asa/asa84/configuration/guide/interface_start.html#wp1326437

Quoting from the documentation: "When you use a redundant or EtherChannel interface as a failover link, it must be pre-configured on both units in the failover pair; you cannot configure it on the primary unit and expect it to replicate to the secondary unit because the failover link itself is required for replication".

2) I am not sure about the reommendation of having a switch over a directly connected cable. If you look at the documentation, it only says it is better to use a different switch for the failover interfaces as opposed to the regular data interfaces. At the same time, it does say (and i am quoting from the documentation here) "To make the ASA failover pair resistant to failover LAN interface failure, we recommend that failover LAN interfaces NOT use the same switch as the data interfaces, as shown in the prededing connections. Instead, use a different switch or use a direct cable to connect two adaptive security appliance failover interfaces". I hope this clears the air out about the recommendation of a switch in between. If you are interested, here's the link to that document (it discusses few other scenarios for deplaying failover as well):

http://www.cisco.com/en/US/docs/security/asa/asa82/configuration/guide/ha_overview.html#wp1089655

As to the functional difference betwen the two, what Jon mentioned is correct.

3) There are 2 main types of monitoring performed: unit health monitoring and interface health monitoring. The unit health monitoring is performed over the Failover interface (in addition to the config sync) and interface monitoring is performed over the respective data interfaces. As mentioned by Jon, if the failover interface fails, the ASAs use the data interfaces to figure out if the other ASA is dead or if just the failover interface has failed and accordingly take an action.

Let me know if you have further questions. Will do my best to answer those for you.

Thanks and Regards,

Prapanch

[Gerald Wiltse]

Prapanch,

Thanks so much for your response... GREAT ANSWERS!!

1) Great find and Great News! So i must humbly admit that I had forgotten that the concept of the "Redundant Link" had existed for some time (long before the etherchannel support. The etherchannel is just a new option. It's interesting that the documentation indicates that it will actually function much like a redundant link, with only one link in the bundle passing traffic at a time.  So this is good.

2) Actually multiple Cisco documents feature a direct recommendation for the switch, so I'm surprised that you say you're "not sure" about it. Here's an example:

"Instead of using a crossover Ethernet cable to directly link the units, Cisco recommends that you use a dedicated switch between the primary and secondary units."

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_configuration_example09186a00807dac5f.shtml

This has lead to countless other blogs by third parties echoing the same misguided recommendation. I still can't see the situation where this is a good decision, but I suppose it's a widely accepted standard now, so perhaps my petition here is pointless.

3) This is a REALLY helpful explanation. Thanks again. I did find the table of failover events at the same link above:

Failure Event	Policy	Active Action	Standby Action	Notes
Active unit failed (power or hardware)	Failover	n/a	Become active; mark active as failed	No hello messages are received on any monitored interface or the failover link.
Formerly active unit recovers	No failover	Become standby	No action	None
Standby unit failed (power or hardware)	No failover	Mark standby as failed	n/a	When the standby unit is marked as failed, the active unit does not attempt to failover, even if the interface failure threshold is surpassed.
Failover link failed within operation	No failover	Mark failover interface as failed	Mark failover interface as failed	You must restore the failover link as soon as possible because the unit cannot failover to the standby unit while the failover link is down.
Failover link failed at startup	No failover	Mark failover interface as failed	Become active	If the failover link is down at startup, both units become active.
Stateful failover link failed	No failover	No action	No action	State information becomes out of date, and sessions are terminated if a failover occurs.
Interface failure on active unit above threshold	Failover	Mark active as failed	Become active	None
Interface failure on standby unit above threshold	No failover	No action	Mark standby as failed	When the standby unit is marked as failed, the active unit does not attempt to fail over even if the interface failure threshold is surpassed.

Again, thanks for the input and participation.

Regards,

Jerry

[praprama]

Hi Jerry,

About number 2), like Julio mentioned below that's the only advantage of having a switch in between that i can really think of. It makes it easier to pin-point which side of theinterface ahs really failed.

Anyways, glad to know you got answers to most of your questions.

P.S: Do mark the post as answered if you feel you got the answer you were looking for.

Cheers,

Prapanch

[blue phoenix]

@praprama

Please let me know if I need to trunk the failover interface?  I am making the ASA to function as a layer 3 device.  I have configured the SVI's in them and I would like to know if I need to trunk the failover interface.

Thanks,

[Vibhor Amrodia]

Hi,

Are you using the Switch only for interconnecting the Failover interfaces ? Anyways , you don't have to trunk these interfaces.

You can just connect the ASA interface to the Switch as an Access port.

Thanks and Regards,

Vibhor Amrodia

[blue phoenix]

Hi,

I am using the ASA as a L3 Gateway for all the vlans.  I will migrate all SVI's from the core switch to the ASA.


Thanks,

[Julio Carvajal]

Hello Gerald,

About question number 2, The use of the switch between the failover Link?????

Its so important and helpful to use a switch as Cisco recommends between the 2 Failover units , This because it  will let you know which interface is the one down or  with problems and then will  safe you a bunch of time.

Lets see an example ? If you use a cable between the units and you fell that something is wrong so  you run commands to check the state of the interfaces and without the switch in the middle you are going to see both interfaces ( Active unit /Standby unit) down. So how are you going to determine which ASA is the one with the interface down or with issues?? As I told you there are ways to figure this out but there is time involved .

Now with the switch in the middle you will safe time and it would be so much easier to troubleshoot this problems. With just one comand you will find out wich interface is the one having the problem.

I hope this helps you out

Julio

[radgerjr1]

I have ASA 5505, 7.2 with 2 ISP's - I followed some postings and cannot seem to get it configured.  I do have the Full License.  I thought I would just post my configuration.  I'm at a point of wiping it out and starting over, but don't want to end up in the same spot.  It will show the backup ISP (TWC) IP address, but will not connect to anything.  Any help would be appreciated:

sho running-con

: Saved

:

ASA Version 7.2(4)

!

interface Vlan1

nameif inside

security-level 100

ip address 192.168.1.1 255.255.255.0

ospf cost 10

!

interface Vlan2

backup interface Vlan3

nameif Verizon

security-level 0

ip address 65.208.133.170 255.255.255.248

ospf cost 10

!

interface Vlan3

nameif TWC

security-level 0

ip address 74.62.207.66 255.255.255.240

ospf cost 10

!

interface Ethernet0/0

switchport access vlan 2

!

interface Ethernet0/1

switchport access vlan 3

!

interface Ethernet0/2

!

interface Ethernet0/3

!

interface Ethernet0/4

!

interface Ethernet0/5

!

interface Ethernet0/6

!

interface Ethernet0/7

!

ftp mode passive

dns server-group DefaultDNS

domain-name ????????

object-group service DM_INLINE_TCP_1 tcp

port-object eq 10019

port-object eq 8016

port-object eq 8200

object-group icmp-type DM_INLINE_ICMP_1

icmp-object echo

icmp-object echo-reply

icmp-object time-exceeded

icmp-object unreachable

object-group icmp-type DM_INLINE_ICMP_2

icmp-object echo

icmp-object echo-reply

icmp-object time-exceeded

icmp-object traceroute

icmp-object unreachable

object-group icmp-type DM_INLINE_ICMP_3

icmp-object echo

icmp-object echo-reply

icmp-object time-exceeded

icmp-object traceroute

icmp-object unreachable

object-group icmp-type DM_INLINE_ICMP_4

icmp-object echo

icmp-object echo-reply

icmp-object time-exceeded

icmp-object unreachable

object-group service DM_INLINE_TCP_2 tcp

port-object eq 10019

port-object eq 8016

port-object eq 8200

access-list outside_access_in remark Inbound for Security System

access-list outside_access_in extended permit tcp any host 65.208.133.174 object-group DM_INLINE_TCP_1

access-list outside_access_in extended permit icmp any host 65.208.133.174 object-group DM_INLINE_ICMP_1 inactive

access-list outside_access_in extended permit icmp any 65.208.133.168 255.255.255.248 object-group DM_INLINE_ICMP_2

access-list TWC_access_in extended permit icmp any 74.62.207.64 255.255.255.240 object-group DM_INLINE_ICMP_3

access-list TWC_access_in extended permit icmp any host 74.62.207.78 object-group DM_INLINE_ICMP_4 inactive

access-list TWC_access_in remark Inbound For Security System

access-list TWC_access_in extended permit tcp any host 74.62.207.78 object-group DM_INLINE_TCP_2

pager lines 24

logging enable

logging asdm informational

mtu inside 1500

mtu Verizon 1500

mtu TWC 1500

no failover

icmp unreachable rate-limit 1 burst-size 1

asdm image disk0:/asdm-524.bin

no asdm history enable

arp timeout 14400

global (Verizon) 1 interface

nat (inside) 1 192.168.1.0 255.255.255.0

nat (Verizon) 1 0.0.0.0 0.0.0.0

static (inside,Verizon) 65.208.133.174 192.168.1.99 netmask 255.255.255.255 dns

static (inside,TWC) 74.62.207.78 192.168.1.99 netmask 255.255.255.255 dns

access-group outside_access_in in interface Verizon

access-group TWC_access_in in interface TWC

route Verizon 0.0.0.0 0.0.0.0 65.208.133.169 1 track 3

route TWC 0.0.0.0 0.0.0.0 74.62.207.65 2

timeout xlate 3:00:00

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute

aaa authentication ssh console LOCAL

http server enable

http 0.0.0.0 0.0.0.0 TWC

http 0.0.0.0 0.0.0.0 Verizon

http 0.0.0.0 0.0.0.0 inside

no snmp-server location

no snmp-server contact

snmp-server enable traps snmp authentication linkup linkdown coldstart

sla monitor 123

type echo protocol ipIcmpEcho 65.208.133.169 interface Verizon

num-packets 3

frequency 10

sla monitor schedule 123 life forever start-time now

!

track 3 rtr 123 reachability

telnet 68.195.244.98 255.255.255.255 Verizon

telnet timeout 5

ssh 0.0.0.0 0.0.0.0 Verizon

ssh 0.0.0.0 0.0.0.0 TWC

ssh timeout 5

ssh version 2

console timeout 0

dhcpd dns 68.18.136.8 69.18.136.9

dhcpd auto_config Verizon

!

dhcpd address 192.168.1.100-192.168.1.200 inside

dhcpd dns 8.8.8.8 208.184.36.10 interface inside

dhcpd enable inside

!

username ????? password ???????

!

class-map inspection_default

match default-inspection-traffic

!

!

policy-map type inspect dns preset_dns_map

parameters

message-length maximum 512

policy-map global_policy

class inspection_default

inspect dns preset_dns_map

inspect ftp

inspect h323 h225

inspect h323 ras

inspect rsh

inspect rtsp

inspect esmtp

inspect sqlnet

inspect skinny

inspect sunrpc

inspect xdmcp

inspect sip

inspect netbios

inspect tftp

!

service-policy global_policy global

prompt hostname context

Cryptochecksum:??????????

: end