NTP / Nexus 7K

sdavids5670 · ‎11-29-2012

I'm having issues with time synchronization on some 7Ks. It looks like the default VDC is setup for NTP:

clock protocol ntp vdc 1

!

ntp distribute

ntp server 192.5.41.209 use-vrf default

ntp peer 192.168.19.2

ntp peer 192.168.19.3

ntp server 208.66.175.36 prefer use-vrf default

ntp source-interface Vlan17

ntp master 15

ntp commit

!

However, on the non-default VDC, the time is not synchronizing with the default VDC. Here's the ntp configuration on the non-default VDC:

ntp distribute

ntp server 192.168.17.2

ntp server 192.168.17.3

ntp source-interface Ethernet8/4

ntp master 10

ntp commit

"show ntp status" on the non-default VDC is:

Distribution : Enabled

Last operational state: Fabric Locked

I didn't set this up to begin with but in looking through the documentation on Cisco website (specifically "Cisco Nexus 7000 Series NX-OS System Management Configuration Guide, Release 6.x") it seems like there's too much configuration on the non-default VDC. From what I can tell, based on this out of the guide:

Configuring NTP on a Secondary (Non-Default) VDC

You can configure a non-default VDC to get a timing update from the default VDC and its clients in order

to synchronize with it.

BEFORE YOU BEGIN

Use the

switchto vdc

command to switch to the desired non-default VDC.

SUMMARY STEPS

1.

config t

2.

feature ntp

3.

ntp master

4.

(Optional) ntp source-interface interface
5.
(Optional) ntp source ip-address
6.
(Optional) copy running-config startup-config

I shouldn't need the "ntp server" commands. I'm guessing that the clock is synchronized with the default VDC, via NTP, using the special IP address of 127.127.1.0. Here's the show ntp peer-status output on the non-default VDC:

Total peers : 1
* - selected for sync, + - peer mode(active),
- - peer mode(passive), = - polled in client mode
remote local st poll reach delay vrf
-------------------------------------------------------------------------------
*127.127.1.0 172.22.1.22 10 16 377 0.00000

I tried turning on debugging and logging for NTP but I didn't get any messages that I thought were useful or revealing. Does the "Fabric Locked" have anything to do with this?

Regards,

Steven

sdavids5670 · ‎11-29-2012

Did a little more digging. Maybe somebody else can chime in on this but I think that "ntp master" should only be used if you want the switch to be an authorative time source on your network and the switch is NOT going to synch with a NTP time source on the Internets. If you put "ntp master" in your configuration then I think the switch is going to want to select itself as the synch source (regardless of whether or not you have configured reachable NTP servers using the "ntp server..." command). At least that's the behavior I observed when my default VDC was configured with the

clock protocol ntp vdc 1

!

ntp master

!

commands at the same time. Once I removed the ntp master command my time started synchronizing with an external source.

Leo Laohoo · ‎11-29-2012

Try to avoid using "ntp master". I have no idea why this command is still available for NX-OS.

chuckwilson · ‎05-08-2013

According to the documentation, "ntp master" causes the nexus to continue to answer ntp requestes from clients if the nexus's time sources (servers) are unreachable.

Of course this would depend on nexus ntp working correctly, whic as far as I can tell, it does not.

I'm on 6.1 and still cannot get the thing to act as a reliable ntp server.

retwaru · ‎08-16-2013

Gentlemen - I found out NTP is not supported on non-default VDC prior to Code 5.2.1.

Sorry to say, I have been unable to locate any clear documentation that define how NTP 'supposed to work' in a Non-Default VDC.

Here is what I have done to get around the issue.

We are running 5.2.(4) on Nexus7010.

We needed the 7K to provide NTP to our DC, and other client downstream, so we configure NTP on the Default VDC as follow:

ntp server 1.1.1.2 prefer use-vrf default

ntp source-interface loopback1

ntp authentication-key 1 md5 PleaseWorkNTP

ntp trusted-key 1

ntp logging

Then we configure non-default VDC as follow:

ntp source-interface loopback1

ntp master 8

I'm happy to say, this solution work for us. My TAC case was escalated to Developers and they also suggest using this as the solution. It worked for me. I have over 1000 Linux and Windows servers working correctly.

I confirmed pointing to GLBP and HSRP address on the 7K work under the following condition:

====================================================================

*On the Linux side, we do not use the -u option in NTPDATE for NTP configuration.

*On the Windows side, Release R2 is needed to make it work

*Just want to point out; my solution is not the solution for all setup. Each infrastructure is different, and solution should be customized and tested and verified to your satisfaction before you go ahead and implement this solution in your production environment.

Best of luck with your testing.