Check the latest back up for all the below server and as mentioned by the customers, the latest back up is avaialable the reachability of the new NTP server Made the changes of NTP server reference in the following servers as Primary NTP server with IP address: 1. Cisco unified communication manager Publisher server 2. Cisco unified communication manager Subscriber server 3. Cisco IM & Prescence server 4. Cisco Unity connection server Restarted the NTP Services in all the above mentioned servers Current NTP stratum value as 4 with new server Test Performed Taken one Ip phone as reference and Restarted the IP phones, Check the NTP refrence
... View more
Pre-change tasks and system health checks DNS change Before the configuration it is highly recommended to ensure that the prerequisites are met. Step 1. Check DNS configuration. Run these commands from CUCM CLI to ensure that DNS service is configured and FQDN entries for node names can be resolved both locally and externally. admin:show network eth0 admin:utils network host cucm105pub.mydomain.com Local Resolution: External Resolution: Step 2. Network diagnostic test. Ensure that network diagnostic test is passed by running this CLI command. admin:utils diagnose module validate_network Step 3. DHCP configuration for endpoints. Ensure that necessary Dynamic Host Configuration Protocol (DHCP) configuration is added for the registered phones to be able to do DNS resolution. Step 4. Database replication. Ensure that CUCM database replication is working. Cluster replication state must be 2 for all nodes. admin:utils dbreplication runtimestate Step 5. Backup. Run Cisco Disaster Recovery System (DRS) backup of the current setup. Hostname can be obtained from show status and the domain can be obtained from show network eth0 command output. Change IP address (or hostname) from IP address to FQDN format for all CUCM servers listed. In order to update configuration files, restart Cisco TFTP service on all CUCM nodes. In order to push updated configuration files to the registered devides, restart Cisco Callmanager service on all CUCM nodes. The main catch is that your MGCP gateways and all your phones will need to be able to resolve those names via DNS to register. You'll want to make sure your DNS is redundant and distributed geographically on all the phone DHCP pools. Certificate back up Step 1 On the Destination Cluster Publisher, navigate to Cisco Unified Operating System Administration and choose Security > Bulk Certificate Management. Step 2 Define the Central SFTP server IP address, port, user, password, and directory. Step 3 Use the Export button to export all TFTP certificates from the destination cluster to the central SFTP server. Change the TFTP Ip address on the DHCP (Voice gateway or Switch) and reset the phones from current CUCM and hoping it will register to the new named CUCM. If your cluster is using CA-signed certificates, you will need to have them re-signed. Run the CTL Client to update the CTL file if you used that process to put your cluster into mixed mode. If you used the tokenless CTL feature, then run the CLI command utils ctl update CTLFile UAT: 1. Ensure that all the endpoints successfully registered back with CUCM nodes. This can be achieved with Real-Time Monitoring Tool (RTMT) In case there is an integration with other servers via SIP, SCCP, MGCP protocols - some configuration might be required on the 3rd party servers. 2. Ensure that the change is propagated successfully to all the nodes in the CUCM cluster and the output is the same across all nodes. Execute this command on all the nodes:- Admin: run sql select name,nodeid from processnode
... View more
Pay attention to certificates. If your CallManager certificates currently have IP addresses in them, they will need to be regenerated with FQDNs. You need to do this one server at a time and reset all phones after each server, then verify that they are getting the updated certificates. If you change all of your servers at once and the phones don't have at least one trusted CallManager certificate, they will cease to accept configuration changes from CUCM. You can avoid this hassle if you are using CTLs, or if you put the cluster into rollback mode which blanks the ITL files. If you plan on getting multi-SAN certificates, take this time to lowercase all of your server hostnames and domains. Check your cluster health by running a Database Status report in Unified Reporting before you make any changes. Run a DRS backup. If you opted for rollback mode to avoid certificate issues, do this now. Make sure you have DNS A and PTR records configured before you do this. If the DNS records are not present then you will break replication. Make sure you have the DNS client enabled on the servers and the domain set correctly. Run "show network eth0" from CLI to verify that the domain name and both DNS servers are set. If you do not have this set in OS/CLI then you will break replication. Make sure that all of your Unified CM groups have 2 or more servers assigned. Start with the publisher, change the IP address to the FQDN. If you did NOT do rollback mode: Regenerate the CallManager certificate then reset a phone and check that it gets the new ITL with the FQDN instead of the IP address. If it does then reset all of your phones. Repeat the last 2 steps for each subscriber. Disable rollback mode then reset phones. Update all of your MGCP gateways with the FQDNs.
... View more
After the Tomcat restart following error received : : test - tomcat_memory : Failed - Tomcat's memory usage is unusually high.
i hope this can be neglected.
seems to be a bug: CSCtj50884
Found the following from the logs
JavaThread "localhost-startStop-2" daemon [_thread_in_native, id=14307, stack(0x87b7b000,0x87bcc000)]
A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x0045eed5, pid=1298, tid=2275924848 # # JRE version: Java(TM) SE Runtime Environment (7.0_85-b15) (build 1.7.0_85-b15) # Java VM: Java HotSpot(TM) Server VM (24.85-b06 mixed mode linux-x86 ) # Problematic frame: # C [libc.so.6+0x60ed5] fgets+0x35 # # Core dump written. Default location: //core or core.1298 # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # --------------- T H R E A D --------------- Current thread (0x0b2dd400): JavaThread "localhost-startStop-2" daemon [_thread_in_native, id=5526, stack(0x87a2d000,0x87a7e000)]
jvm_args: -Djava.library.path=/lib:/opt/cisco/connection/lib:/usr/local/lib:/usr/local/thirdparty/java/j2sdk/jre/lib/i386:/usr/local/thirdparty/java/j2sdk/jre/lib/i386/server:/usr/lib/pgsql:/usr/lib:/usr/local/cm/lib:/opt/cisco/connection/lib:/usr/local/Nuance/Recognizer_Service/amd64/lib:/usr/local/Nuance/OAM/x86/lib:/usr/local/Nuance/Common/x86/lib:/usr/local/Nuance/Common/amd64/lib:/usr/local/platform/lib:/usr/local/Nuance/Recognizer/lib -XX:+UseParallelGC -XX:GCTimeRatio=10 -XX:ErrorFile=/usr/local/thirdparty/jakarta-tomcat/logs/diagnostic-info.jvm-crash.%p.tomcat.txt -Dsun.zip.disableMemoryMapping=true -XX:OnOutOfMemoryError=/home/tomcat/tomcat_diagnostics.sh -XX:OnError=/home/tomcat/tomcat_diagnostics.sh -Dnet.sf.ehcache.skipUpdateCheck=true -XX:-UseSplitVerifier -Djavax.net.ssl.trustStore=/usr/local/platform/.security/tomcat/trust-certs/tomcat-trust.keystore -Djavax.net.ssl.trustStorePassword=OqlcSyEJfpeGF1EF -Djavax.net.ssl.trustStoreType=PKCS12 -Djavax.net.ssl.keyStore=/usr/local/platform/.security/tomcat/certs/tomcat.keystore -Djavax.net.ssl.keyStorePassword=UYvv3cUY782qoNbD -Djavax.net.ssl.keyStoreType=PKCS12 -DamKeyGenDescriptor.provider=JsafeJCE -DamCryptoDescriptor.provider=JsafeJCE -DamRandomGenProvider=JsafeJCE -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/usr/local/thirdparty/jakarta-tomcat/conf/logging.properties -Xmx1216m -Xms256m -XX:MaxPermSize=448m -Djava.endorsed.dirs=/usr/local/thirdparty/jakarta-tomcat/endorsed -Djava.security.policy==/usr/local/thirdparty/jakarta-tomcat/conf/catalina.policy -Dcatalina.base=/usr/local/thirdparty/jakarta-tomcat -Dcatalina.home=/usr/local/thirdparty/jakarta-tomcat -Djava.io.tmpdir=/usr/local/thirdparty/jakarta-tomcat/temp -Dcommons.daemon.process.id=7258 -Dcommons.daemon.process.parent=7257 -Dcommons.daemon.version=1.0.7 abort
... View more
Is that possible to restart the "Connection Conversation Manager" service via CUC CLI ?
Cisco Unity connection : Active version: 126.96.36.19900-11
There were an issue that the CUC pub GUI cannot be accessed its giving 'HTTPS 404 error'
while checking the services via CLI it found Tomcat service down, it start running after reboot the tomcat service via CLI. But still PUB (GUI / RTMT) cannot be accessible
And there were no issue with Sub(one PUB & one SUB) sub is accessible
1. CUC Pub cannot be accessible
2. Voice mail cannot be retrieved
Early response would be appreciated!
Note: attached the Tomcat log from Pub
... View more
Thanks for your quick response.
i have already taken the outputs as you mentioned. And also couldnt find the unusual messages in Event log.
Is there any method to find the Memory usage from the GW (the same way as we find CPU utilization)
... View more
VG 248 firmware 1.2.1
analysis 1. Verified the following, found that the status of the log level is in Trace level 2. Although it is peak usage time and almost all the resource are utilised, but CPU utilisation is 40% and memory utilisation is 83% which is in constant level on the business hours. 3. And there were no memory spike as per the analysis on the monitoring tool (solar wind) update. 4. on the event log on the VG 248 voice gateway, could not find any alarms/ error message regarding the memory spike or the high CPU utilization
... View more