Re: NTP synchronization lost frequently - IOS XR

simone.c · ‎09-25-2020

Hi all,

on all our new ASR9k we are observing a loss of NTP synchronization. We have many IOS devices which do not show this issue, so obviously it is not a network problem as the NTP servers are the same for all devices (internal server).

I see from past discussions here in the community that some IOS versions were affected by sudden loss of synchronizations (buggy behaviour). I've also seen bugs CSCsx73385 and CSCsz22456, where something similar happens due to RP failover (which is not our case). Does anybody have similar experience with IOS XR? We have IOS XR 6.6 and use NTP authentication.

RP/0/RSP1/CPU0:Sep 10 11:46:55.727 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.212796(s)
RP/0/RSP1/CPU0:Sep 10 11:46:55.727 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 10 11:53:31.830 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.059150(s)
RP/0/RSP1/CPU0:Sep 10 12:01:26.946 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.065387(s)
RP/0/RSP1/CPU0:Sep 11 13:39:16.765 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.227962(s)
RP/0/RSP1/CPU0:Sep 11 13:39:16.765 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 11 13:45:57.843 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.036743(s)
RP/0/RSP1/CPU0:Sep 11 13:53:55.968 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.070478(s)
RP/0/RSP1/CPU0:Sep 13 06:35:30.592 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.173937(s)
RP/0/RSP1/CPU0:Sep 13 06:35:30.592 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 13 06:42:04.725 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.093361(s)
RP/0/RSP1/CPU0:Sep 14 04:51:42.770 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.227679(s)
RP/0/RSP1/CPU0:Sep 14 04:51:42.770 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 14 04:54:06.812 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.023710(s)
RP/0/RSP1/CPU0:Sep 14 04:59:06.872 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.026426(s)
RP/0/RSP1/CPU0:Sep 14 05:26:37.171 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.141477(s)
RP/0/RSP1/CPU0:Sep 14 05:26:37.171 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 14 05:28:00.196 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.013212(s)
RP/0/RSP1/CPU0:Sep 14 09:29:18.808 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.269006(s)
RP/0/RSP1/CPU0:Sep 14 09:29:18.808 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 14 09:31:45.909 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.072639(s)
RP/0/RSP1/CPU0:Sep 14 09:41:59.029 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.054322(s)
....
RP/0/RSP1/CPU0:Sep 21 11:46:54.706 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.207255(s)
RP/0/RSP1/CPU0:Sep 21 11:46:54.706 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 21 11:49:19.793 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.070564(s)
RP/0/RSP1/CPU0:Sep 21 11:58:24.893 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.042290(s)
RP/0/RSP1/CPU0:Sep 23 01:36:15.634 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.224755(s)
RP/0/RSP1/CPU0:Sep 23 01:36:15.634 CEST: ntpd[360]: %IP-IP_NTP-5-SYNC_LOSS : Synchronization lost : DLRSC node : The clock was stepped and needs to be resynced
RP/0/RSP1/CPU0:Sep 23 01:40:41.684 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.023506(s)
RP/0/RSP1/CPU0:Sep 23 01:47:29.747 CEST: ntpd[360]: %IP-IP_NTP-5-CLOCK_UPDATE : Clock changed by source DLRSC node by an offset 0.019481(s)

mjwan0001 · ‎06-23-2021

Hello, I also have this problem, may I ask if yours has been resolved

simone.c · ‎06-23-2021

Hi mjwan0001,

having no answer here I had opened a case with Cisco TAC, I'm pasting below their answer.

Meanwhile we changed SW release and the problem is not visible anymore.

Based upon the logs provided and research on my end, this is a harmless condition in which no impact to traffic or services is expected. Enhancements for this are made through DDTS CSCvq18900, integrated in releases 6.6.3 onwards.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvq18900/?rfs=iqvred

There is no impact from this behavior and thus there is no immediate need to upgrade. The bug is simply introducing enhancements with regards to the NTP internal communication between RSP and LC.

-------------------------------------------------------------------------

Case Summary:

The NTP synchronization messages are due to the NTP protocol keeping the clock in sync by small adjustments called ‘slew’. Slew will correct the internal clock with higher and higher error tolerances till it exceeds a max tolerance of 128ms at which time, NTP gives up and ‘steps’ the clock – essentially an abrupt jump.