cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

2594
Views
30
Helpful
15
Replies
joe.bennett
Participant

CUCMBE as NTP server

Hi,

My CUPS server is complaining that the CM publisher's NTP isn't working. The publisher is a CUCM-BE and a show ip preferences shows that NTP is blocked by the firewall. It's synced fine to our internal NTP servers but won't let anything else sync to it which is tricky because CUPS insists on syncing to it. Any ideas how to enable NTP on the server?

Thanks,

Joe

15 REPLIES 15
Fernando Rivas Martin
Cisco Employee

Hello Joe

Can you check the status of the ntp with this command in the CUCMBE:

utils ntp status

Also check the output on the CUCMBE of the command:

utils firewall list

It should include a line saying:

ACCEPT     udp  --  anywhere             anywhere            udp dpt:ntp

Please send the output of those commands to see how it looks like.

Try also if you have available the command: utils ntp start

( i dont have a CUCMBE that i can test, i only have CUCM servers)


Thanks
Fernando

Well, my first question would be the following:

Did you configure your CUCMBE to sync to an external NTP server - if so, is that server truly EXTERNAL (i.e., on the internet) and something that may well be blocked by your firewall?  Or, rather did you sync to a server on a different subnet that may be inadvertently blocked by a firewall?

The first is an issue that has come up many times and is not a recommended configuration IMHO.  For many reasons...which I won't go into.  The second is something that can be easily rectified.  What I would do is configure a local gateway (for example 2811) as an NTP server and set that as the NTP source for the CUCMBE.  See if you can sync to that as a source - internally, and ideally within a trusted network.

Hailey

Please rate helpful posts!

admin:utils ntp status

ntpd (pid 30559) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 LOCAL(0) 10 l 20 64 377 0.000 0.000 0.004

*10.222.69.248 203.12.160.2 3 u 125 128 377 1.077 8.685 1.726

10.222.67.1 .INIT. 16 u - 1024 0 0.000 0.000 4000.00

synchronised to NTP server (10.222.69.248) at stratum 4

time correct to within 119 ms

polling server every 128 s

Current time in UTC is : Tue Nov 23 23:35:55 UTC 2010

Current time in Australia/Queensland is : Wed Nov 24 09:35:55 EST 2010

admin:utils firewall ipv4 list

Table: mangle

Not happy publishing my firewall rules although they're default.

Anyway - standard IPTables and no entries for port 123 or NTP in the config.

I'd figured that the firewall rules were the cause - the question is can we tweak them? My guess was CUCMBE is standalone so doesn't want to talk NTP but CUPS insists on learning NTP from the CM.

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

Joe,

Could you check/post the output of "utils ntp status" from the CUPS server console? In regards to the firewall list on the CUCM, there should be a line for udp port 123. If you have a Linux or Mac host, you could check to see if you can reach NTP on the CUCM server by using the ntpq command.

For example:

sh-3.2# ntpq -c peers 10.3.4.20
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*10.3.2.1        204.9.54.119     2 u   45   64  377    2.341  -81.569  29.076

In the example above, 10.3.4.20 is my CUCM publisher node.

If you only have a Windows PC/server then you can probably find a NTPQ windows binary on the internet. The older CallManager servers had a NTP package called XNTP, which I used on my Windows boxes in the lab.

If you can successfully run a NTP query (ntpq) on the CUCM from a remote station, then the CUCM firewall is not blocking you.

You could also try to troubleshoot this using the network capture facility on the CUCM and CUPS servers. For example:

admin:utils network capture port 123 count 1000 size 128 host ip 192.168.1.4
Executing command with options:
size=128                count=1000              interface=eth0
src=                    dest=                   port=123
ip=192.168.1.4

In this example, I am running the capture on my CUCM and the 192.168.1.4 address is my macbook running the ntpq command. However, you could set this up on both the CUCM-BE and CUPS server.

On CUCM:

admin:utils network capture file ntpcheckFromCUCM count 100000 size all port 123

On CUPS:

admin:utils network capture file ntpcheckFromCUPS count 100000 size all port 123

You could also filter by IP addresses if you wanted to see other information (like an ICMP port unreachable?).  In this latter case, I would recommend filtering by IP address on the CUPS server  (host ip ) and by port on the CUCM server.  Mmm, well maybe if you did an ip filter in both it wouldn't be too bad. You may need to check to see if there is too much info in the trace.

Anyway, after you execute the command on both servers. You may want to open up a NEW SSH session to the CUPS server and execute a NTP restart (utils ntp restart) or start (utils ntp start). After a minute, you will want to enter Ctrl-C on the CUCM-BE and CUPS SSH sessions where you initiatied the capture.

You can then download the capture files:

On CUCM:

admin: file get activelog platform/cli/ntpcheckFromCUCM.cap

On CUPS:

admin: file get activelog platform/cli/ntpcheckFromCUPS.cap

You can then use Wireshark or some other protocol analyzer to review the sniffer traces.

Now, you don't necessarily have to capture the output for processing off box, if you go back to the example where I was capturing traffic on port 123 to/from host 192.168.1.4 (above) then you could at least determine packets were being exchanged. In our example, I ran a ntpq -c peers command from 192.168.1.4 right after setting up the capture on the CUCM host. Here is the output:

admin:utils network capture port 123 count 1000 size 128 host ip 192.168.1.4
Executing command with options:
size=128                count=1000              interface=eth0
src=                    dest=                   port=123
ip=192.168.1.4
19:44:52.953623 IP 192.168.1.4.61591 > iecucm01.cnclab.com.ntp: NTPv2, Reserved, length 12
19:44:52.954080 IP iecucm01.cnclab.com.ntp > 192.168.1.4.61591: NTPv2, Reserved, length 16
19:44:52.957733 IP 192.168.1.4.61591 > iecucm01.cnclab.com.ntp: NTPv2, Reserved, length 12
19:44:52.957910 IP iecucm01.cnclab.com.ntp > 192.168.1.4.61591: NTPv2, Reserved, length 480
19:44:52.957975 IP iecucm01.cnclab.com.ntp > 192.168.1.4.61591: NTPv2, Reserved, length 144

You can tell two things from this output. First, the NTP packets from 192.168.1.4 successfully reached the CUCM server. This definitively tells us the network isn't blocking. Second, you can see that the CUCM server (iecucm01) responds on the NTP port. This definitevely tells us that the CUCM firewall isn't blocking anything. Of course, I suspect that if the host firewall was blocking NTP we may even find that the first NTP packet is not seen. I would have to test that but the result is not important.

At this point, you should have enough data to determine where the communication is breaking.

Note, that if you needed to test the host based firewall (i.e. remove it from the equation), then you could try to disable the firewall (utils firewall disable). I haven't done this myself, so read up on this command before you start your testing.

HTH.


Regards,
Bill

Please rate helpful posts.

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

Presence server

admin:utils ntp status

ntpd (pid 8458) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

*127.127.1.0 LOCAL(0) 10 l 52 64 377 0.000 0.000 0.001

10.222.66.234 .INIT. 16 u - 1024 0 0.000 0.000 4000.00

synchronised to local net at stratum 11

time correct to within 12 ms

polling server every 64 s

Current time in UTC is : Wed Nov 24 01:53:17 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 11:53:17 EST 2010

======================================================================

joebennett@toad:~$ ntpq -c peers cucmbe

cucmbe: timed out, nothing received

***Request timed out

============================================

Ssh to CUCMBE;

admin:utils firewall ipv4 disable

Warning: Disabling the internal firewall can cause disruption in network

services. In particular redirected traffic such as HTTP and TFTP will be

disrupted which can affect phone registrations.

Do you want to continue?

Enter (yes/no)? yes

Stopping atd:

Starting atd:

firewall (iptables) is disabled

firewall (iptables) will be enabled at Wed Nov 24, 2010 12:07:30

admin:exit

================================================

SSH to presence

admin:utils ntp restart

Restarting NTP

admin:utils ntp status

ntpd (pid 29716) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 LOCAL(0) 10 l 2 64 1 0.000 0.000 0.001

10.222.66.234 10.222.69.248 4 u 1 64 1 1.187 0.822 0.001

unsynchronised

time server re-starting

polling server every 64 s

Current time in UTC is : Wed Nov 24 02:02:59 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 12:02:59 EST 2010

========================================================

Works fine with no firewall - how do I add a rule in IPTables with no shell access?

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

joe.bennett wrote:


================================================

SSH to presence

admin:utils ntp restart

Restarting NTP

admin:utils ntp status

ntpd (pid 29716) is running...

     remote           refid      st t when poll reach   delay   offset  jitter

==============================================================================

127.127.1.0     LOCAL(0)        10 l    2   64    1    0.000    0.000   0.001

10.222.66.234   10.222.69.248    4 u    1   64    1    1.187    0.822   0.001

unsynchronised

  time server re-starting

   polling server every 64 s

Current time in UTC is : Wed Nov 24 02:02:59 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 12:02:59 EST 2010

========================================================

Works fine with no firewall - how do I add a rule in IPTables with no shell access?

Joe,

Your output still shows unsynchronized. Did the NTP eventually establish sync?  If it did, then maybe you are running into this bug:

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCti88192&from=summary

NTP port is not opened on Publisher firewall after changing server role
Symptom:

Installation of secondary node fails with following error message  
"NTP server inaccessible, system will halt"

Conditions:

It  is observed on VOS based products ( like Communication Manager, Unity  Connection) where the installation of the Publisher server was started  of as secondary node and later changed as primary ( first node in the  cluster )  during the installation procedure.

Workaround:

Following commands can be used to verify whether Publisher ( primary node ) is accepting the NTP traffic or not.

show network ipprefs all
show firewall ipv4 list

If  NTP is disabled and udp/123 port is blocked in the firewall, we can  execute the following command on Publisher to update the firewall  configuration

/usr/local/bin/base_scripts/ipprefs -T ntp --enable

Which means something got fowled up during the install. You would need to contact Tac to execute the workaround.

HTH.


Regards,

Bill

Please rate helpful posts.

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

admin:utils ntp status

ntpd (pid 29716) is running...

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 LOCAL(0) 10 l 2 64 377 0.000 0.000 0.001

*10.222.66.234 10.222.69.248 4 u 42 64 377 0.204 20.483 0.009

synchronised to NTP server (10.222.66.234) at stratum 5

time correct to within 198 ms

polling server every 64 s

Current time in UTC is : Wed Nov 24 02:39:22 UTC 2010

Current time in Australia/Brisbane is : Wed Nov 24 12:39:22 EST 2010

The bug looks interesting except that the cause is different (can't start a CUCMBE install as a sub). It looks like shell access is the answer either way.

Thanks for your help. I'd never poked around with the firewall and NTP from the CLI so those commands were useful.

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

OK.  Glad to be of some nominal help.

Regards,
Bill

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

Hello

In CUCM BE there is no concept of Publisher or Subcribers. It is a standalone server.

And that might be the cause of the problem.

I checked some of the standalone servers in my lab (CUCM 5.1.3, CUCM 6.1.4)

All of them have the firewall locked for the NTP traffic. (As there is no other server configured for clustering).

I will try with CUCM 7.X and CUCM 8.X now.

Which version of CUCMBE do you have installed?

Do yo have your CUPS configured in the CUCM under Server -> Application Server?

Thanks
Fernando

Fernando,

Good feedback, thanks (+5).

I agree with the lack of pub/sub in CUCM-BE. I thought about the FW filter as it relates to NTP and CUCM-BE but I haven't worked with CUCM-BE in well over a year and I wasn't sure. I dismissed it because I believe CUCM-BE and CUPS are supposed to be compatible and in CUPS 7.0(4) (??) and later, I am pretty sure that CUPS will assume it recovers NTP from the CUCM cluster it is associated with. IOW, you no longer assign a NTP server via the CUPS GUI (maybe that has changed in 8.x).  Anyway, if what you say is true (and I don't doubt you) then this could still be logged as a bug and the mitigation would likely be the same.

Regards,
Bill

HTH -Bill (b) http://ucguerrilla.com (t) @ucguerrilla

Please remember to rate helpful responses and identify

Hi Will

I totally agree with you.

CUPS should be able to work with the CUCM-BE.

Basically CUPS should be added in the Application Server list in the CUCM.

Then CUPS should try to authenticate with the CUCM (Cluster Manager service is responsible for that).

If the CUCM authenticate the CUPS (meaning that CUCM has CUPS listed in Application Server config and that the security password matches) then it will open the firewall.

At this moment i would expect the NTP line to be added in the firewall rules.

(btw the CUCM stand-alone servers version 7.1.3 and 8.0.3 i tested had the rule added by default, contrary to CUCM 5.x and 6.x)

So i believe that Joe's problem could be that his CUPS server is not authenticated by the Cluster Manager and hence FW still is closed.

Otherwise as you said it is a big bug, but it should be affecting more customers (or i would expect it at least) and i havent seen any SR like that or any defect mention it.

@Joe:

Please get the output of the command: show network cluster from your CUCM

It should list the CUCM and the CUPS.

What versions of CUCM and CUPS are you using?

Also another problem is that in my lab i dont have any CUCM-BE, i only have CUCM single-node and multinode clusters so i cannot really test same scenario as Joe.

Tomorrow i will try to add a CUPS to one of my CUCM 6.x or 5.x that had the FW closed for NTP and test if the ClusterManager service opens the FW once the CUPS is validated. (I would expect that).


Thanks
Fernando

It's definitely in the application server list and the firewall has rules for it, jut not NTP.

Table: filter

Chain INPUT (policy DROP)

target prot opt source destination

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:4040 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:1500 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:1515 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:8001 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:1090 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:1099 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:2555 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:2556 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpts:2551:2552 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:1502 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:1503 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpts:4900:4904 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:19003 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:20055 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:22000 flags:0x02/0x02

ACCEPT udp -- 10.222.66.235 0.0.0.0/0 udp dpt:22000

ACCEPT udp -- 10.222.66.235 0.0.0.0/0 udp dpt:22001

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpt:22001 flags:0x02/0x02

ACCEPT tcp -- 10.222.66.235 0.0.0.0/0 tcp dpts:20500:20502 flags:0x02/0x02

admin:show network cluster

10.222.66.234 cucmbe.bpc.local cucmbe Publisher authenticated

10.222.66.235 presence.bpc.local presence Subscriber authenticated using UDP since Wed Nov 24 12:03:18 2010

CUCM-BE is

Active Master Version: 8.5.0.96091-8

Presence is

Active Master Version: 8.0.2.10000-30

Regards,

Joe Bennett

Network Consultant

Bridge Point Communications

Hi Joe

I could not reproduce the issue using normal CUCM standalone servers (All have the NTP rule in the FW).

I am trying to install the CUCM-BE in one of my VMs and will try to reproduce the issue.

I will keep you posted.

In any case, if it is urgent you can open a TAC SR and we will handle it.

(If you open it during european hours, send me a private message with the SR number and i will try to grab it).

Thanks
Fernando

Hi Joe

I managed to install a CUCM-BE 8.5 and a CUPS 8.0.1

I found that the CUCM-BE as you said has the ntpd disabled and the firewall is closed.

During the installation of CUPS i was prompted to enter an NTP  source, which could be any server. I didnt point it to the CUCM-BE i  pointed to a real NTP.

Post install when i introduced the CUCM in the CUPS is when the NTP source changed to CUCM.

Apparently this is enforced automatically post-install.

Also at the same it is normal that the CUCM-BE doesnt have the NTP active, as it is not expecting anybody

There is one defect:

CSCtf06250    Can not specify NTP server in OS administration

That it is supposed to be fixed in CUPS 8.0.2 which is the one you have, so you should be able to change the NTP source.

But i believe this is a defect on the CUCM side, firewall is not opening the NTP ports. Although the CUPS is CLM validated.

At this point quickest way would be open a TAC SR so they can get root and open the NTP ports.

Regards
Fernando

Content for Community-Ad