C3850 HA sync fails

[sftechsupport]

I have a pair of Cisco C3850-48T switches  which i have stacked together.

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48T 03.06.04.E cat3k_caa-universalk9 INSTALL
 2 56 WS-C3850-48T 03.06.04.E cat3k_caa-universalk9 INSTALL

Switch #2 (the Standby switch) appears to boot normally, and joins the stack as a "member".

Switch/Stack Mac Address : 006c.bc57.9a00 - Local Mac Address
Mac persistency wait time: Indefinite
 H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 006c.bc57.9a00 15 V06 Ready
 2 Member 042a.e2a7.9500 10 V06 Ready

After a few minutes, it is elected as the "Standby" and goes into "HA sync in progress" mode.

Switch/Stack Mac Address : 006c.bc57.9a00 - Local Mac Address
Mac persistency wait time: Indefinite
 H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active 006c.bc57.9a00 15 V06 Ready
 2 Standby 042a.e2a7.9500 10 V06 HA sync in progress

However, the sync never completes, and after 30 minutes, the switch is removed from the stack, and reboots.

#show switch stack-ports

Switch# Port1 Port2
----------------------------
1 OK OK
2 OK OK

As you see the stacking cables are connected OK.

Sw#/Port# Port Status Neighbor Cable Length Link OK Link Active Sync OK #Changes to LinkOK In Loopback
-------------------------------------------------------------------------------------------------------------------
1/1 OK 2 50cm Yes Yes Yes 59 No
1/2 OK 2 50cm Yes Yes Yes 179 No
2/1 OK 1 50cm Yes Yes Yes 6 No
2/2 OK 1 50cm Yes Yes Yes 1 No

I am puzzled as to how to proceed to fix this HA sync failure   :( 

(The switches are at a remote location, so i do not have console access to them)

Has anyone seen this behaviour before ?

[acampbell]

Hi,

See this link :-

https://clnv.s3.amazonaws.com/2015/usa/pdf/BRKCRS-3146.pdf

look at slide 61 - you need to gather some information and raise a TAC.

Regards

Alex

[rojesara.prashant]

We are having same issue. Did you find any solution ? Our IOS-XE version is 03.07.04E and Model is WS-C3850-24XS.

[David Nolasco]

Hi Rojesara,

Please take a look at "show sw stack-port summary", check the number of changes to link. From the output provided above you can see +100 changes for one of the stack-ports. This might be due to a loose cable or faulty stack-cable. Also, it is possible to have a faulty stack-port.

Regards

DN

[bsecrest]

Has similar issue with a switch stuck in HA sync in progress. There is a watchdog timeout after about 35 minutes that would cause the switch to reload. After opening a ticket with Cisco it was determined that there was an internal hardware issue with the switch and ended up having to RMA.

[ALINA ANDIRAOS]

Hi!
I have had exactly the same issue  two days ago, I end up having RMA.
WS-C3850-24S

Regards,
Alina Andiraos

[ahmadzubair654]

I am still having the same issue with 3850-24T. 

Is there any solution other than RMA? If someone has figured out please update this post. 

Also, show redundancy shows standby as "STANDBY COLD". no way to change it.

DC6-ECSSOC-3850s#show switch stack-mode
Switch# Role Mac Address Version Mode Configured State
--------------------------------------------------------------------------------------------------------------------
1 Standby 00cc.fc24.2222 V06 N+1 None HA sync in progress
*2 Active c4b3.6a24.2223 V07 N+1 None Ready

[ahmadzubair654]

I can reload the member switch from master switch, yet HA sync in progress never ends.

Only way to make config change is to 'reload member switch' make quick changes to master switch.  

When both switches are up, process stuck at HA Sync in progress.

"conf t
Config mode cannot be entered during Standby initialization or when switch is in recovery mode"

#show switch stack-ports summary
Sw#/Port# Port Status Neighbor Cable Length Link OK Link Active Sync OK #Changes to LinkOK In Loopback
-------------------------------------------------------------------------------------------------------------------
1/1 OK 2 50cm Yes Yes Yes 1 No
1/2 OK 2 50cm Yes Yes Yes 1 No
2/1 OK 1 50cm Yes Yes Yes 7 No
2/2 OK 1 50cm Yes Yes Yes 7 No

Under #changes to LinkOK, There is 7.  thats mainly because I have reloaded member switch about 7 times.

[RedOne]

Same problem here

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48P 16.9.4 CAT3K_CAA-UNIVERSALK9 INSTALL
   2 32 WS-C3850-24P 16.9.4 CAT3K_CAA-UNIVERSALK9 INSTALL

License Usage:
License Entitlement tag Count Status
-----------------------------------------------------------------------------
C3850_48_Ipbase (C3850-48 IP Base) 1 AUTHORIZED
C3850_24_Ipbase (C3850-24 IP Base) 1 AUTHORIZED

Sw#/Port# Port Status Neighbor Cable Length Link OK Link Active Sync OK #Changes to LinkOK In Loopback
-------------------------------------------------------------------------------------------------------------------
1/1 OK 2 50cm Yes Yes Yes 1 No
1/2 OK 2 50cm Yes Yes Yes 1 No
2/1 OK 1 50cm Yes Yes Yes 1 No
2/2 OK 1 50cm Yes Yes Yes 1 No

Switch# Role Mac Address Version Mode Configured State
*1 Active MAC_addr V05 N+1 None Ready
  2 Standby MAC_adr V06 N+1 None HA sync in progress

Switch# Port1 Port2
----------------------------
1 OK OK
2 OK OK

Stack Ring Speed : 480G
Stack Ring Configuration: Full
Stack Ring Protocol : StackWise

#sh redundancy states
my state = 13 -ACTIVE
peer state = 4 -STANDBY COLD
Mode = Duplex
Unit = Primary
Unit ID = 1

Redundancy Mode (Operational) = sso
Redundancy Mode (Configured) = sso
Redundancy State = sso
Maintenance Mode = Disabled
Manual Swact = disabled (peer unit not yet in terminal standby state)
Communications = Up

client count = 90
client_notification_TMR = 30000 milliseconds
RF debug mask = 0x0