NTP Design Questions

[grochowskir]

Hi,

I have a question about the proper NTP design for my organization. We are using a collapsed core design here. We have two Core switches present with one being the primary HSRP gateway and the primary STP root for all vlans. What I would like to do is to connect a GPS time source directly to each Core switch in order to provide stratum 1 time reference. Then I would set up NTP on each core and access layer devices as follows:

The IP addresses are obviously arbitrary.

CORE1: 

ntp server 192.168.100.40 --> (this is the IP address of the directly connected GPS device 1)

ntp peer 192.168.100.3 --> (this is the IP address of the CORE2 SVI)

ntp orphan 3

CORE2:

ntp server 192.168.100.41 -------> (this is the IP address of the directly connected GPS device 2)

ntp peer 192.168.100.2 --------> (this is the IP address of CORE1 SVI)

ntp orphan 3

Access Layer Switches Configuration:

ntp server 192.168.100.1 ----------> (this is the IP address of the HSRP instance for vlan 100. This vlan is also our management vlan as well as the default gateway address for each access layer switch.)

-The idea is to for the Core switches to obtain time from their respective GPS devices. Then all access layer switches as well as other hosts such as ESXi servers and domain controllers obtain time from the Cores by being pointed to the virtual IP provided by the HSRP 192.168.100.1. In case one of the Core switches loses connectivity to their respective GPS device, they can sync their time from the second Core switch. 

Notice that I am not using loopback interfaces on the Core switches and referring to the SVI. I am attaching a portion of the network diagram. 

-Is this a correct approach? Have missed or misconfigured here? What would you change/recommend?

Thank you.

[Leo Laohoo]

Make it simple.  

CORE 1. 

ntp server <GPS 1 IP address> primary

ntp server <GPS 2 IP address>

CORE 2

ntp server <GPS 2 IP address> primary

ntp server <GPS 1 IP address>

Everyone else use CORE 1 or CORE 2 as their primary NTP server.

[grochowskir]

Thank for your reply --> one more question. How about the DMZ zone? there is nothing mission critical there now. How would you provide time to the stack of switches inside the DMZ zone? Should i just point them to the pool.ntp.org? I am including a more complete diagram. Thanks again for your opinion.

[Leo Laohoo · Accepted solution]

Your DMZ switches can talk to your core switches to get NTP synch.  You just make sure of a firewall rule allowing ONLY the two DMZ switches to talk to the two core switches. 

As what Reza says, if your core switch can support loopback interfaces, then use this as an NTP interface.

[grochowskir]

Thank you all.

[Reza Sharifi]

If the outside interface of the DMZ and the firewalls have public IPs, you can point them to a public site and for everything else use the GPS. Or as Leo said, open the firewall to access the internal GPS for timing.

HTH

[Reza Sharifi]

Hi,

Your design should work fine, but I would use a loopback interface instead of an SVI. A loopback interface never goes down unless the whole switch is down. With the current design, if you lose the SVI your access switches and all the servers behind them will lose time.

HTH

[joe19366 · Accepted solution]

A Cisco device is not a very reliable/accurate NTP server IMHO. we have seen drift of up to 30 SECONDS at times depending on host startup  / shutdown and resume - especially in places with vmware and windows server "polling" ntp servers.

The Cisco processor is subject to interrupt switching and the ntp daemon in Cisco is not as accurate as a dedicated appliance such as symmetricom or sonoma.

Another flaw of NTP is the client is allowed to poll and step/slew time which has all sorts of accuracy issues. think of NTP as a cheap manual watch.

We use the Sonoma N12 CDMA edition in places we cant run a GPS antenna to the roof with Precision time protocol. We have found only PTP is good at keeping an accurate clock across all systems.

Remember - these are the accuracy "budgets" of each type of antennae -

CDMA - accurate to within 10usec (micro seconds)

GPS - accurate to 1-2usec

As you can see GPS is designed for places where accuracy is key.

I would not use NTP at all - I would use PTPv2 if you are going the route of accurate time.

Here is what we do in these scenarios -

you can send ptpv2 over regular nics on the "main vlan" contrary to popular rumor it doesnt require its own "time vlan". If you have 10gb ethernet and solarflare/chelsio/intel x540T2 nic cards your time will vary. PTPv2 however setup correctly will ALWAYS be much much much more accurate than any NTP implementation by a factor of 100 to 1000.

install "domain time" client on all windows hosts - greyware.com its great for windows. they even have a walk through how to configure.

install the yum ptp client for linux servers -

[root@trade01 ~]# rpm -qa | grep ptp
ptpd-2.3.0-1.el6.x86_64

(attached is our ptp config file)

You cant get these results with ntp. note we tested many PTP options - the libpcap mode where ntp packets are analyzed off the wire ended up being most accurate.

E2E is the style of PTP you probably want to use when you setup your PTP grandmaster. the Sonoma N12 is like $5000. the Symmetricom is probably overkill and is $40,000+

Note: ptp grand master and clients must be in the SAME DOMAIN (a number) otherwise they will not update time

[root@trade01 ~]# more /var/run/ptpd2.status
Interface          :  bond0
Preset             :  slaveonly
Transport          :  ipv4 + libpcap
IP mode            :  multicast
Delay mechanism    :  E2E
Sync mode          :  TWO_STEP
PTP domain         :  0
Port state         :  PTP_SLAVE
Local port ID      :  000f53fffe213f90(unknown)/01
Best master ID     :  000efefffe0101f6(unknown)/01
Best master IP     :  162.251.204.100
GM info            :  Priority1 127, Priority2 128, clockClass 6
Time properties    :  PTP timescale, traceable: time Y, freq Y
UTC properties     :  UTC valid: Y, UTC offset: 35
Offset from Master :  0.000000972 s, mean  0.000000010 s, dev  0.000000244 s
One-way delay      :  0.000000000 s, mean  0.000000000 s, dev  0.000000000 s
Clock status       :  calibrated
Drift correction   : -23.456 ppm, mean -23.456 ppm, dev  0.000 ppm
Message rates      :  8/s sync, 1/s delay, 1/4s announce

Announce received  :  1289018
Sync received      :  41248543
Follow-up received :  41248539
DelayReq sent      :  5309895
DelayResp received :  0
State transitions  :  19
PTP Engine resets  :  2

Finally, in a local linux crond shell script or windows powershell script as a task - you can use a shell script wrapped around some variant of sed/awk to read the status file /var/run/ptpd2.status with the "Offset from Master :  0.000000972 s" result pattern matched for accuracy. In our world, all results must BEGIN with the pattern "0.00000" meaning, we have LESS than 10us of offset or we get the alert. as you can see above - we are accurate with PTP to

0.000000972 s

UNDER 1 MICROSECOND :)

[grochowskir]

Thank you so much!