cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3388
Views
30
Helpful
15
Replies

CUCMBE as NTP server

joe.bennett
Level 3
Level 3

Hi,

My CUPS server is complaining that the CM publisher's NTP isn't working. The publisher is a CUCM-BE and a show ip preferences shows that NTP is blocked by the firewall. It's synced fine to our internal NTP servers but won't let anything else sync to it which is tricky because CUPS insists on syncing to it. Any ideas how to enable NTP on the server?

Thanks,

Joe

15 Replies 15

Fernando Rivas Martin
Cisco Employee
Cisco Employee

Hello Joe

Can you check the status of the ntp with this command in the CUCMBE:

utils ntp status

Also check the output on the CUCMBE of the command:

utils firewall list

It should include a line saying:

ACCEPT     udp  --  anywhere             anywhere            udp dpt:ntp

Please send the output of those commands to see how it looks like.

Try also if you have available the command: utils ntp start

( i dont have a CUCMBE that i can test, i only have CUCM servers)


Thanks
Fernando

Well, my first question would be the following:

Did you configure your CUCMBE to sync to an external NTP server - if so, is that server truly EXTERNAL (i.e., on the internet) and something that may well be blocked by your firewall?  Or, rather did you sync to a server on a different subnet that may be inadvertently blocked by a firewall?

The first is an issue that has come up many times and is not a recommended configuration IMHO.  For many reasons...which I won't go into.  The second is something that can be easily rectified.  What I would do is configure a local gateway (for example 2811) as an NTP server and set that as the NTP source for the CUCMBE.  See if you can sync to that as a source - internally, and ideally within a trusted network.

Hailey

Please rate helpful posts!

admin:utils ntp status

ntpd (pid 30559) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 LOCAL(0) 10 l 20 64 377 0.000 0.000 0.004

*10.222.69.248 203.12.160.2 3 u 125 128 377 1.077 8.685 1.726

10.222.67.1 .INIT. 16 u - 1024 0 0.000 0.000 4000.00

synchronised to NTP server (10.222.69.248) at stratum 4

time correct to within 119 ms

polling server every 128 s

Current time in UTC is : Tue Nov 23 23:35:55 UTC 2010

Current time in Australia/Queensland is : Wed Nov 24 09:35:55 EST 2010

admin:utils firewall ipv4 list

Table: mangle

Not happy publishing my firewall rules although they're default.

Anyway - standard IPTables and no entries for port 123 or NTP in the config.

I'd figured that the firewall rules were the cause - the question is can we tweak them? My guess was CUCMBE is standalone so doesn't want to talk NTP but CUPS insists on learning NTP from the CM.

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

Joe,

Could you check/post the output of "utils ntp status" from the CUPS server console? In regards to the firewall list on the CUCM, there should be a line for udp port 123. If you have a Linux or Mac host, you could check to see if you can reach NTP on the CUCM server by using the ntpq command.

For example:

sh-3.2# ntpq -c peers 10.3.4.20
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*10.3.2.1        204.9.54.119     2 u   45   64  377    2.341  -81.569  29.076

In the example above, 10.3.4.20 is my CUCM publisher node.

If you only have a Windows PC/server then you can probably find a NTPQ windows binary on the internet. The older CallManager servers had a NTP package called XNTP, which I used on my Windows boxes in the lab.

If you can successfully run a NTP query (ntpq) on the CUCM from a remote station, then the CUCM firewall is not blocking you.

You could also try to troubleshoot this using the network capture facility on the CUCM and CUPS servers. For example:

admin:utils network capture port 123 count 1000 size 128 host ip 192.168.1.4
Executing command with options:
size=128                count=1000              interface=eth0
src=                    dest=                   port=123
ip=192.168.1.4

In this example, I am running the capture on my CUCM and the 192.168.1.4 address is my macbook running the ntpq command. However, you could set this up on both the CUCM-BE and CUPS server.

On CUCM:

admin:utils network capture file ntpcheckFromCUCM count 100000 size all port 123

On CUPS:

admin:utils network capture file ntpcheckFromCUPS count 100000 size all port 123

You could also filter by IP addresses if you wanted to see other information (like an ICMP port unreachable?).  In this latter case, I would recommend filtering by IP address on the CUPS server  (host ip ) and by port on the CUCM server.  Mmm, well maybe if you did an ip filter in both it wouldn't be too bad. You may need to check to see if there is too much info in the trace.

Anyway, after you execute the command on both servers. You may want to open up a NEW SSH session to the CUPS server and execute a NTP restart (utils ntp restart) or start (utils ntp start). After a minute, you will want to enter Ctrl-C on the CUCM-BE and CUPS SSH sessions where you initiatied the capture.

You can then download the capture files:

On CUCM:

admin: file get activelog platform/cli/ntpcheckFromCUCM.cap

On CUPS:

admin: file get activelog platform/cli/ntpcheckFromCUPS.cap

You can then use Wireshark or some other protocol analyzer to review the sniffer traces.

Now, you don't necessarily have to capture the output for processing off box, if you go back to the example where I was capturing traffic on port 123 to/from host 192.168.1.4 (above) then you could at least determine packets were being exchanged. In our example, I ran a ntpq -c peers command from 192.168.1.4 right after setting up the capture on the CUCM host. Here is the output:

admin:utils network capture port 123 count 1000 size 128 host ip 192.168.1.4
Executing command with options:
size=128                count=1000              interface=eth0
src=                    dest=                   port=123
ip=192.168.1.4
19:44:52.953623 IP 192.168.1.4.61591 > iecucm01.cnclab.com.ntp: NTPv2, Reserved, length 12
19:44:52.954080 IP iecucm01.cnclab.com.ntp > 192.168.1.4.61591: NTPv2, Reserved, length 16
19:44:52.957733 IP 192.168.1.4.61591 > iecucm01.cnclab.com.ntp: NTPv2, Reserved, length 12
19:44:52.957910 IP iecucm01.cnclab.com.ntp > 192.168.1.4.61591: NTPv2, Reserved, length 480
19:44:52.957975 IP iecucm01.cnclab.com.ntp > 192.168.1.4.61591: NTPv2, Reserved, length 144

You can tell two things from this output. First, the NTP packets from 192.168.1.4 successfully reached the CUCM server. This definitively tells us the network isn't blocking. Second, you can see that the CUCM server (iecucm01) responds on the NTP port. This definitevely tells us that the CUCM firewall isn't blocking anything. Of course, I suspect that if the host firewall was blocking NTP we may even find that the first NTP packet is not seen. I would have to test that but the result is not important.

At this point, you should have enough data to determine where the communication is breaking.

Note, that if you needed to test the host based firewall (i.e. remove it from the equation), then you could try to disable the firewall (utils firewall disable). I haven't done this myself, so read up on this command before you start your testing.

HTH.


Regards,
Bill

Please rate helpful posts.

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

Presence server

admin:utils ntp status

ntpd (pid 8458) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

*127.127.1.0 LOCAL(0) 10 l 52 64 377 0.000 0.000 0.001

10.222.66.234 .INIT. 16 u - 1024 0 0.000 0.000 4000.00

synchronised to local net at stratum 11

time correct to within 12 ms

polling server every 64 s

Current time in UTC is : Wed Nov 24 01:53:17 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 11:53:17 EST 2010

======================================================================

joebennett@toad:~$ ntpq -c peers cucmbe

cucmbe: timed out, nothing received

***Request timed out

============================================

Ssh to CUCMBE;

admin:utils firewall ipv4 disable

Warning: Disabling the internal firewall can cause disruption in network

services. In particular redirected traffic such as HTTP and TFTP will be

disrupted which can affect phone registrations.

Do you want to continue?

Enter (yes/no)? yes

Stopping atd:

Starting atd:

firewall (iptables) is disabled

firewall (iptables) will be enabled at Wed Nov 24, 2010 12:07:30

admin:exit

================================================

SSH to presence

admin:utils ntp restart

Restarting NTP

admin:utils ntp status

ntpd (pid 29716) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 LOCAL(0) 10 l 2 64 1 0.000 0.000 0.001

10.222.66.234 10.222.69.248 4 u 1 64 1 1.187 0.822 0.001

unsynchronised

time server re-starting

polling server every 64 s

Current time in UTC is : Wed Nov 24 02:02:59 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 12:02:59 EST 2010

========================================================

Works fine with no firewall - how do I add a rule in IPTables with no shell access?

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

joe.bennett wrote:


================================================

SSH to presence

admin:utils ntp restart

Restarting NTP

admin:utils ntp status

ntpd (pid 29716) is running...

     remote           refid      st t when poll reach   delay   offset  jitter

==============================================================================

127.127.1.0     LOCAL(0)        10 l    2   64    1    0.000    0.000   0.001

10.222.66.234   10.222.69.248    4 u    1   64    1    1.187    0.822   0.001

unsynchronised

  time server re-starting

   polling server every 64 s

Current time in UTC is : Wed Nov 24 02:02:59 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 12:02:59 EST 2010

========================================================

Works fine with no firewall - how do I add a rule in IPTables with no shell access?

Joe,

Your output still shows unsynchronized. Did the NTP eventually establish sync?  If it did, then maybe you are running into this bug:

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCti88192&from=summary

NTP port is not opened on Publisher firewall after changing server role
Symptom:

Installation of secondary node fails with following error message  
"NTP server inaccessible, system will halt"

Conditions:

It  is observed on VOS based products ( like Communication Manager, Unity  Connection) where the installation of the Publisher server was started  of as secondary node and later changed as primary ( first node in the  cluster )  during the installation procedure.

Workaround:

Following commands can be used to verify whether Publisher ( primary node ) is accepting the NTP traffic or not.

show network ipprefs all
show firewall ipv4 list

If  NTP is disabled and udp/123 port is blocked in the firewall, we can  execute the following command on Publisher to update the firewall  configuration

/usr/local/bin/base_scripts/ipprefs -T ntp --enable

Which means something got fowled up during the install. You would need to contact Tac to execute the workaround.

HTH.


Regards,

Bill

Please rate helpful posts.

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

admin:utils ntp status

ntpd (pid 29716) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 LOCAL(0) 10 l 2 64 377 0.000 0.000 0.001

*10.222.66.234 10.222.69.248 4 u 42 64 377 0.204 20.483 0.009

synchronised to NTP server (10.222.66.234) at stratum 5

time correct to within 198 ms

polling server every 64 s

Current time in UTC is : Wed Nov 24 02:39:22 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 12:39:22 EST 2010

The bug looks interesting except that the cause is different (can't start a CUCMBE install as a sub). It looks like shell access is the answer either way.

Thanks for your help. I'd never poked around with the firewall and NTP from the CLI so those commands were useful.

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

OK.  Glad to be of some nominal help.

Regards,
Bill

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

Hello

In CUCM BE there is no concept of Publisher or Subcribers. It is a standalone server.

And that might be the cause of the problem.

I checked some of the standalone servers in my lab (CUCM 5.1.3, CUCM 6.1.4)

All of them have the firewall locked for the NTP traffic. (As there is no other server configured for clustering).

I will try with CUCM 7.X and CUCM 8.X now.

Which version of CUCMBE do you have installed?

Do yo have your CUPS configured in the CUCM under Server -> Application Server?

Thanks
Fernando

Fernando,

Good feedback, thanks (+5).

I agree with the lack of pub/sub in CUCM-BE. I thought about the FW filter as it relates to NTP and CUCM-BE but I haven't worked with CUCM-BE in well over a year and I wasn't sure. I dismissed it because I believe CUCM-BE and CUPS are supposed to be compatible and in CUPS 7.0(4) (??) and later, I am pretty sure that CUPS will assume it recovers NTP from the CUCM cluster it is associated with. IOW, you no longer assign a NTP server via the CUPS GUI (maybe that has changed in 8.x).  Anyway, if what you say is true (and I don't doubt you) then this could still be logged as a bug and the mitigation would likely be the same.

Regards,
Bill

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

Hi Will

I totally agree with you.

CUPS should be able to work with the CUCM-BE.

Basically CUPS should be added in the Application Server list in the CUCM.

Then CUPS should try to authenticate with the CUCM (Cluster Manager service is responsible for that).

If the CUCM authenticate the CUPS (meaning that CUCM has CUPS listed in Application Server config and that the security password matches) then it will open the firewall.

At this moment i would expect the NTP line to be added in the firewall rules.

(btw the CUCM stand-alone servers version 7.1.3 and 8.0.3 i tested had the rule added by default, contrary to CUCM 5.x and 6.x)

So i believe that Joe's problem could be that his CUPS server is not authenticated by the Cluster Manager and hence FW still is closed.

Otherwise as you said it is a big bug, but it should be affecting more customers (or i would expect it at least) and i havent seen any SR like that or any defect mention it.

@Joe:

Please get the output of the command: show network cluster from your CUCM

It should list the CUCM and the CUPS.

What versions of CUCM and CUPS are you using?

Also another problem is that in my lab i dont have any CUCM-BE, i only have CUCM single-node and multinode clusters so i cannot really test same scenario as Joe.

Tomorrow i will try to add a CUPS to one of my CUCM 6.x or 5.x that had the FW closed for NTP and test if the ClusterManager service opens the FW once the CUPS is validated. (I would expect that).


Thanks
Fernando