ASA failover failed....

[alastair27]

Hi there people in my phone, 

We have a pair of ASAs in a failover pair, and the failover has failed with an interface error.  Interfaces on the FW and connecting switch are both up/up, no configuration has been changed, and it's been fine for the last couple of years.  Overrun errors on the FW interface are racking up at about 5 a second, and I know overrun errors are usually caused by the box not being able to cope with the traffic hitting it, but this is the standby box (now) so has no traffic hitting it.....

Am I looking at a hardware error and the box should be RMAd, or is there something I can try?

Failed interfaces are all sub-interfaces of 0/1.  It is a multi context FW, and both contexts have failed as they both use sub interfaces on 0/1

Thanks.

xxxxxxxxx/stby/pri# sh fail
Failover On
Failover unit Primary
Failover LAN Interface: fover GigabitEthernet0/2 (up)
Reconnect timeout 0:00:00
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 9 of 216 maximum
MAC Address Move Notification Interval not set
failover replication http
Version: Ours 9.8(2)28, Mate 9.8(2)28
Serial Number: Ours xxxxxxx, Mate xxxxxx
Last Failover at: 13:56:23 WET Jan 8 2021
This host: Primary - Failed
Active time: 66525106 (sec)
slot 0: ASA5525 hw/sw rev (1.0/9.8(2)28) status (Up Sys)
admin Interface inside (xxx.159.53): Normal (Monitored)
admin Interface xxxxdmz (xxx.159.6): Failed (Waiting)
admin Interface xxxxdmz (xxx.60.2): Failed (Waiting)
admin Interface xxxxdmz (xxx.244.130): Failed (Waiting)
admin Interface xxxxdmz (xxx.134.194): Failed (Waiting)
admin Interface management (0.0.0.0): No Link (Waiting)
xxx Interface inside (xxx.159.69): Normal (Monitored)
xxx Interface outside-xxx (xxx.159.77): Normal (Monitored)
xxx Interface xxxdmz (xxx.159.109): Failed (Waiting)
Other host: Secondary - Active
Active time: 3956437 (sec)
slot 0: ASA5525 hw/sw rev (1.0/9.8(2)28) status (Up Sys)
admin Interface inside (xxx.159.52): Normal (Monitored)
admin Interface xxxdmz (xxx.159.5): Normal (Monitored)
admin Interface xxxdmz (xxx.60.1): Normal (Monitored)
admin Interface xxxdmz (xxx.244.129): Normal (Monitored)
admin Interface xxxdmz (xxx.134.193): Normal (Monitored)
admin Interface management (0.0.0.0): No Link (Waiting)
xxx Interface inside (xxx.159.68): Normal (Monitored)
xxx Interface outside-xxx (xxx.159.76): Normal (Monitored)
xxx Interface xxxdmz (xxx.159.108): Normal (Monitored)

xxxxxxx/stby/pri# sh fail state

State Last Failure Reason Date/Time
This host - Primary
Failed Ifc Failure 13:56:23 WET Jan 8 2021
xxxdmz: Failed
xxxdmz: Failed
xxxdmz: Failed
xxxdmz: Failed
admin management: No Link
xxxdmz: Failed

xxxxxxx/stby/pri# sh int g 0/1
Interface GigabitEthernet0/1 "", is up, line protocol is up
Hardware is i82574L rev00, BW 1000 Mbps, DLY 10 usec
Full-Duplex(Full-duplex), 1000 Mbps(1000 Mbps)
Input flow control is unsupported, output flow control is off
Description: link-to-xxx-g9/44_vlxxx
Available for allocation to a context
MAC address xxx, MTU not set
IP address unassigned
201853492912 packets input, 133492696908091 bytes, 0 no buffer
Received 23689448 broadcasts, 0 runts, 0 giants
7476547 input errors, 0 CRC, 0 frame, 7476547 overrun, 0 ignored, 0 abort
0 pause input, 0 resume input
0 L2 decode drops
251266867632 packets output, 171685628643737 bytes, 487 underruns
0 pause output, 0 resume output
0 output errors, 0 collisions, 2 interface resets
0 late collisions, 0 deferred
0 input reset drops, 0 output reset drops
input queue (blocks free curr/low): hardware (504/362)
output queue (blocks free curr/low): hardware (511/0)

Other host - Secondary
Active Comm Failure 22:35:52 WEST Aug 25 2020

[balaji.bandi]

check some of the interface are waiting they need to be physically test and make sure those VLAN and other side reachable.

[alastair27]

So, had to wait till the weekend, but thought as a first step we'd swop the cable, and if that didn't work, reboot the box, and then finally RMA it.  And, very boringly, swopping the cable brought the failover back up, and I've now failed it back onto the primary.  Switch interface is clean, and all is happy in Toytown. 

I'm putting it down to cable fault, but whether just cycling the port would have done anything....

Thanks, Al.

[Wassim Aouadi]

In my opinion, resetting the port would have not solved the problem. You clearly showed in your first post that there are many input errors. So the patch cable should have been the problem.

[balaji.bandi]

post latest information show failover and what is the switch logs.

[alastair27]

What logs are you after?  My next plan was to have a beer and get ready for the Monday morning password resets.....

xxxxxx/admin/act/pri# sh fail
Failover On
Last Failover at: 11:06:18 WET Feb 28 2021
        This context: Active
                Active time: 34849 (sec)
                  Interface inside (xx.159.52): Normal (Monitored)
                  Interface xxx-dmz (xx.159.5): Normal (Monitored)
                  Interface xx-dmz (xx.60.1): Normal (Monitored)
                  Interface xxxx-dmz (xx.244.129): Normal (Monitored)
                  Interface xxx-dmz (xx.134.193): Normal (Monitored)
                  Interface management (0.0.0.0): No Link (Waiting)
        Peer context: Standby Ready
                Active time: 4396232 (sec)
                  Interface inside (xx.159.53): Normal (Monitored)
                  Interface xxx-dmz (xx.159.6): Normal (Monitored)
                  Interface xx-dmz (xx.60.2): Normal (Monitored)
                  Interface xxx-dmz (xx.244.130): Normal (Monitored)
                  Interface xxx-gmol-dmz (xx.134.194): Normal (Monitored)
                  Interface management (0.0.0.0): No Link (Waiting)

[balaji.bandi]

 Interface management (0.0.0.0): No Link (Waiting)   - check this link where physically connected or any issue on switch side ?