When IPT first came to planet earth, the gods told us that we must ensure that everything is deployed using IP. They told us that DNS is unreliable, it doesn’t work well with IPT solution etc.
Fast forward a few years and now the gods are forced to eat their words, because for some reasons and due to the demands of aliens on our planet, IPT now needs DNS.
So what if you listened to the gods a few years back and due to the demands of your IPT inhabitants, you need to migrate your CUCM servers from IP based to FQDN. How do you go about achieving this goal without disrupting your services.
I found myself in this dilemma recently and due to the lack of documentation on the exact approach to use for this, I have decided to document the process and highlights the pitfalls you are likely to face along the way and how to mitigate against such risks.
Enterprise collaboration is no longer phone/device centric, it has evolved into a user centric solution and hence application centric.
The challenge with this is that applications such as Cisco Jabber now rely heavily on network infrastructure such as PKI (certficates) and obviously DNS.
Today you will find your self in one of two positions. Your current infrastructure needs to be migrated to a DNS reliant one or you are deploying a new solution and must be DNS enabled.
Before you embark on a project like this, you need to consider under lying elements that will be impacted. You need to ask yourself what the dependencies are. In my environment there were two major ones
Your IP Phones have been using IP address to communicate with CUCM. Now you are going to change this behavior, hence DNS is absolutely crucial. You need to ensure all your phones have DNS. For this to be successful you need to know the DHCP source for all your devices.
Once you have identified the DHCP source, you need to verify that the DHCP scope has DNS configured. This is critical. If you change your CUCM host name to FQDN without DNS in your IP phone's DHCP scope, then you run the risk of loosing so many phones during this change.
Some devices may have their DHCP scope on gateways while others have DHCP on windows DNS server. This is critical.
Dont assume your IP phones all have common DHCP source. Do your due diligence and find out what the DHCP source is especially in a multi site deployment.
If you need to add DNS servers to your DHCP scope on Cisco IOS router here is the command to do it
ip dhcp pool POOLNAME dns-server 192.168.1.100 192.168.1.101
MGCP gateways: This was a hidden one and could cause a lot of grief. Check your mgcp gateways and ensure that they are enabled for DNS.
The reason for this is that some mgcp gateways are configured with ccm-manager config, in which case these devices will automatically download their config from cucm. Once the system>server name has been changed, the mgcp gateway will try and connect to the FQDN of the CUCM and without a valid DNS and domain lookup configured it will fail.
To configure DNS on your mgcp gateway do the following:
WRS1#sh ccm-manager MGCP Domain Name: WRSW1 Priority Status Host ============================================================ Primary Domain Name Unresolvable UK-HM-SCMS04.net.live.net First Backup Domain Name Unresolvable UK-HA-SCMS04.net.live.net Second Backup None
Once DNS is enabled you should see the following: Priority Status Host ============================================================ Primary Registered UK-HM-SCMS04.net.live.net (192.168.20.14) First Backup Down UK-HA-SCMS04.net.live.net (192.168.20.44)
You are not out of the woods yet. The fact that you have done all your due diligence and everything looks good doesn't mean nothing van go wrong. A lot can still go wrong. So don't just run off and make the change. Read on....
CUCM configuration>system > server
Ensure that you enter the correct FQDN of the cucm servers as defined in DNS. This is crucial. This could make a long night longer for you. So pay attention here. Its critical that you enter the correct FQDN in the CUCM configuration page.
In my change, CUCM host name was changed from IP to only the DNS name and not the FQDN as showed in the image below
The implication of this was huge
NB: the FQDN of the servers were already added through the CUCM cli platform page, so the TVS , tftp certs contain the FQDN.
The most critical issue was that the phones could not connect to their TVS server to verify their new configuration file
Here is the process in detail:
Once the phones downloaded the new configuration, in essence the phones download a new ITL file, Once ITL file is successfully received, phone then request a signed config file.
Next phone must verify ITL before it can trust the configuration file.
If the phone already has ITL it must verify the downloaded ITL files match the signature in the ITL /CTL or connect to the TVS server if the ITL signature doesn’t match.
The trace below details the process.
866: NOT 15:17:58.550372 TFTP: :Requesting ITLSEP0CD9969170BB.tlv from 10.105.40.15 867: NOT 15:17:58.776273 TFTP: :Finished --> rcvd 13384 bytes ####
Next thing is the phone must verify the ITL file since it already has ITL file installed..Here since the phone is unable to verify the signer credentials from the existing ITL, It will try to contact its TVS server
#### 872: ERR 15:17:58.789752 SECD: CTL_GetFileSigner: Error: Failed to find signer credentials 873: ERR 15:17:58.790558 SECD: TL_ValidateSignedTL: Error: Unable to retrieve signer info from TL ####
Since the phone cant verify the new ITL from the existing it tries to connect to the TVS server but it goes via the TVS proxy to know which TVS server it should connect to..the TVS proxy is the tftp server
874: NOT 15:17:58.792914 SECD: setupSocketToTvsProxy: TVS client sock fd 9 bound to </tmp/secClntTvs_92_5878> 875: NOT 15:17:58.794048 SECD: setupSocketToTvsProxy: Connected to TVS proxy server -- 875: NOT 15:17:58.794048 SECD: setupSocketToTvsProxy: Connected to TVS proxy server 876: NOT 15:17:58.795033 SECD: tvsReqQueryCertificate: Sent Request to TVS proxy, len: 2684 877: NOT 15:17:58.795648 SECD: tvsReqQueryCertificate: Waiting for response from TVS Proxy 878: NOT 15:17:58.797387 SECD: clpTvsInit: Client message received on TVS proxy socket 879: NOT 15:17:58.798363 SECD: processTvsClntReq: Success reading the client TVS request, len : 2684 880: NOT 15:17:58.799056 SECD: processTvsClntReq: TVS Query Certificate request 881: NOT 15:17:58.799670 SECD: processTvsClntReq: signer : CN=LELHM-SCMT01.net.live.net;OU=I.T;O=LAB;L=labham;ST=London;C=GB 882: NOT 15:17:58.801202 SECD: processTvsClntReq: issuer : CN=LELHM-SCMT01.net.live.net;OU=I.T;O=LAB;L=labham;ST=London;C=GB
Next TVS proxy informs the phone which TVS it should connect to..it should be first server in its cucm group.
88: NOT 15:17:58.808350 SECD: getTvsSrvrSock: Expecting ip address, got hostname: SCMS04
#### Here we see that the phone is unable to resolve the TVS server hostname###
896: ERR 15:17:58.910323 SECD: EROR:getTvsSrvrSock: Failed to resolve hostname: SCMS04
There phone says I cant verify my configuration file..and hence no registration..
981: ERR 15:18:00.179985 SECD: EROR:getTvsSrvrSock: secReq_getTvsServer succeeded but got an empty ip address for index : 2 982: NOT 15:18:00.181489 SECD: sendErrRespToClient: Sending the failed response to all TVS client and cleaning up 983: NOT 15:18:00.182991 SECD: tvsReqQueryCertificate: Received the response from TVS proxy, status: 1 984: NOT 15:18:00.183579 SECD: tvsReqQueryCertificate: The cert len received is 0 985: NOT 15:18:00.184958 SECD: Failed validation using TVS, 0============> 986: ERR 15:18:00.185784 SECD: EROR:verifyFile: sgn verify file failed </usr/ram/SEP0CD9969170BB.cnf.xml>, errclass 8, errcode 19 (signer not in CTL) 987: ERR 15:18:00.186425 SECD: EROR:verifyFile: verify FAILED, </usr/ram/SEP0CD9969170BB.cnf.xml
It is important to note that this step is really crucial. Several phones could be affected especially if they are generation two phones that have SBD enabled. In some cases you might need to delete the ITL files on the phones to get them back, so be careful here
This change will impact database replication. In some cases you could end up with corrupt cdr server list and as such you may need to rebuild the cdr server lists for the whole cluster.
If you are not very confident on resolving database replication issue, then I suggest you open a pro active TAC case before you begin this. This was the most challenging part of the whole job. Depending on the size of your cluster, this could take a long time. I was working on a huge cluster with 13 servers, so it took me more than 7 hours to get dbreplication going again. Be prepared to battle this out!!!
Do not just attempt to reset dbreplication if things are broken, this could just prolong your pain. If you have a corrupt cdr server list, then reset dbreplication wont help you. So make sure you know what the issue is and then you can use the right dbreplication command to address it.
Finally there is no need to reboot any server after this change.
Good Afternoon, Situation, want to be able to take office phones from home and office and be able to use them with untagged traffic at home. Office is using VLAN 10 for Data network. We want the phones to work at home with untagged traffic, then brin...
Is it possible to change presence status such as available, Away, DND on Jabber client using API Query and use that as a gadget in Finesse.
idea is if agent mark himself as not ready from finesse then he will be still available in Ja...
I was trying to locate IM and presense Developer guide and cookbook for IMP version 12.5.1 But I find that last developer Guide or cookbook for IMP was for version 10.5.2. Is there any new document available for IMP version 12.5 or does the old ...
I ask me to explain the following:I have CUCM 12.5 Pub and Sub. There is CUCM 9.1 Pub only (for tests). Made SIP Trunk from 9.1 to 12.5 (only on Sub !!!) and from 12.5 to 9.1 (All Nodes is checked). Calls from 9.1 to 12.5 pass, from 12.5 to 9.1 no (503 er...