topic Re: ASA Stateful Failover problem in Network Security

ASA Stateful Failover problem

Bela Mareczky — Mon, 11 Mar 2019 17:39:10 GMT

Dear Forum Community!

We have recently implemented ASA stateful failover between two ASA 5540 operating at two different location. Unfortunately, because of a temporary switch installation, the standby peer has one physical interface at speed 100-duplex full, while the primary device has all interface at speed 1000-duplex full.

Please refer to the output of the "show failover" command executed in the standby device below: the receive error counters shows that something is wrong with stateful HA.

Could anyone help me to find out, if the asymmetric interface speed could cause this symptom?

Thanks and BR

Belabacsi

Budapest, Hungary

Stateful Failover Logical Update Statistics
        Link : ***** (up)
        Stateful Obj    xmit       xerr       rcv        rerr
        General         555244     0          696813995 65685015
        sys cmd         555244     0          555244     0
        up time         0          0          0          0
        RPC services    0          0          0          0
        TCP conn        0          0          199265403 56098066
        UDP conn        0          0          487869778 9492795
        ARP tbl         0          0          9121627    94154
        Xlate_Timeout   0          0          0          0
        VPN IKE upd     0          0          556        0
        VPN IPSEC upd   0          0          1132       0
        VPN CTCP upd    0          0          0          0
        VPN SDI upd     0          0          0          0
        VPN DHCP upd    0          0          0          0
        SIP Session     0          0          255        0

        Logical Update Queue Information
                        Cur     Max     Total
        Recv Q:         0       138     713534140
        Xmit Q:         0       1       555244

Re: ASA Stateful Failover problem

Jennifer Halim — Thu, 29 Apr 2010 21:38:13 GMT

You are right. Interface speed for the stateful failover link needs to be the same on both firewalls. It also needs to be the highest speed on your ASA, so the ASA that has the stateful interface down to 100, you would need to fix the interface so it's 1000, the same as the other ASA stateful interface speed.

Otherwise, you will be seeing what you are currently seeing, ie: receive error (rerr). The standby ASA can't receive the failover state information fast enough through the stateful link, hence you saw the received error.

Hope that answers your question.

Re: ASA Stateful Failover problem

Bela Mareczky — Fri, 30 Apr 2010 09:07:02 GMT

Dear halijenn!

Thanks for Your reply, I think it helps to resolve the problem.

I have just double-checked the configuration: the outside interface of the primary ASA has a speed 100-duplex full state, because it is connected to a temporary device which is C2960 10/100 switch 😮 Every other ports connect to gigabit switchport and have speed 1000-duplex full state, including gigabit 0/3 which serve as state and failover VLAN trunk.

Primary ASA:

###########

Outside: speed 100/duplex full

Inside: speed 1000/duplex full

DMZ: speed 1000/duplex full

HA: speed 1000/duplex full

Secondary ASA:

#############

Outside: speed 1000/duplex full
Inside: speed 1000/duplex full
DMZ: speed 1000/duplex full
HA: speed 1000/duplex full

Do You think, the speed 100 state of the outside interface could also cause the errors?

Thanks in advance !

Regards, Belabacsi

Re: ASA Stateful Failover problem

Jennifer Halim — Fri, 30 Apr 2010 13:41:09 GMT

Speed 100 on the outside interface is OK. However, I am concern about all the rerr that you are getting on the stateful failover link.

You might want to double check if the rerr errors are increasing. Also what version of ASA are you running?

Re: ASA Stateful Failover problem

Bela Mareczky — Mon, 03 May 2010 07:18:35 GMT

Dear halijenn !

Thanks for Your reply, unfortunately the err counters are increasing... 😞

The ASA version information:

Cisco Adaptive Security Appliance Software Version 8.2(1)
Device Manager Version 6.2(5)

Hardware: ASA5540, 1024 MB RAM, CPU Pentium 4 2000 MHz

Internal ATA Compact Flash, 256MB

Do You think, it can be a software bug?

Belabacsi

Re: ASA Stateful Failover problem

Jennifer Halim — Mon, 03 May 2010 10:50:20 GMT

Please check the "show interface" output for the stateful failover link/interface on both ASA firewall. You might also want to check the corresponding switch interfaces/ports. Possibly it could be faulty cable.

Don't think it's software bug at this stage. It's more looking like an interface issue.

Re: ASA Stateful Failover problem

Bela Mareczky — Thu, 06 May 2010 09:59:26 GMT

Dear halijenn!

Thanks for the tips! Unfortunataly, the switch interfaces connecting to the ASA seem to be OK, I hava fount no CRC / errors counting.

We have a scheduled maintenance window on Saturday, when we plan to force-switchover the ASA HA and reboot the device...we expect some posotive results 🙂 I 'll inform You about the err counter status.

Thanks and BR

Belabacsi

Re: ASA Stateful Failover problem

Jennifer Halim — Thu, 06 May 2010 11:11:31 GMT

Thanks for the update. Let us know how it goes after the reload.

Re: ASA Stateful Failover problem

Bela Mareczky — Thu, 13 May 2010 14:16:04 GMT

Dear halijenn!

After ASA HA switchover and reload both devices, the err counters stop counting The software version and HA configuration are the same as before, however we successfully migrated all ASA interfaces to gigabit speed, so all ASA interfaces (of both devices) operating at 1000 / full duplex.

It is an interesting story after the first reboot, everything seemed to be OK...suddenly the ASA ASDM service crashed, the "show asdm session" command output stated that, we reached the permitted concurrent ASDM session limit. I had one active connection from 172.16.129.221 IP address. Trying to disconnect the "stucked" sessions, but no luck...

firewall# show asdm sessions
0 mbela_172.16.129.221
1 mbela_172.16.129.221
2 mbela_172.16.129.221
3 mbela_172.16.129.221
4 mbela_172.16.129.221

firewall#

firewall# asdm disconnect 0

firewall# asdm disconnect 1

firewall# asdm disconnect 2

firewall# asdm disconnect 3

firewall# asdm disconnect 4

firewall# show asdm sessions
0 mbela_172.16.129.221
1 mbela_172.16.129.221
2 mbela_172.16.129.221
3 mbela_172.16.129.221
4 mbela_172.16.129.221

firewall#

Suddenly, I lost the SSH connection and the device rebooted. Finally,this reboot solved the issue.

It is annoying, because we don't know what was the real cause of the problem...

Thanks for Your help!

Regards,

Belabacsi

Re: ASA Stateful Failover problem

Jennifer Halim — Fri, 14 May 2010 09:06:00 GMT

Good to hear that upgrade resolves the issue. Please kindly mark the question as answered. Thank you.

Re: ASA Stateful Failover problem

Bela Mareczky — Tue, 18 May 2010 12:54:54 GMT

Dear halijenn!

Thanks for Your help!

We have not upgraded the ASA software, the HA configuration and software version are the same as before...only the 2nd reboot solves the error counter issue

Unfortunately, we don't know the cause of the problem...

Regards,

Belabacsi