CUCM Bad Performance, dropping calls and ICMP from a simple machine.

[Sergio Baulde]

Hi,

We have a CUCM cluster (Pub-Sub running each one in a different UCS). The UCS got no drops on his Admin IP, and no drops too on the others appliances installed on them. The Sub is worst than Pub, I got constant ICMP drops from a normal machine (I'm aware of the drop rate from an ios), and got drop calls from different phones running SIP and SCCP, being on the same LAN and from different WAN's.

Is there something that should I check ?, the network perfomance is fine, not satured, the switch port dont count errors of any kind, and Im running out of ideas to looking for. The RTMT alarms think are normal.

Best Rgs

Sergio

[Aaron Harrison]

Hi 

Have you checked  CPU utilisation on 'show status' and 'show process using-most cpu'?

Aaron

[ndehmer]

Do add on what Aaron said, have you looked at other performance metrics with in vCenter (cpu, disk, etc).  Is the UCS blade/server over subscribed.  Is there other processes going on that could affect performance like backups?  Is there QoS on the network to ensure that voice and voice signaling traffic is protected?

[Sergio Baulde]

Thks for the quick reply, I was doing a couple of test, I already discard network issues, today I restarted both boxes, and off hours the Sub doesn't drops ICMP. But today again with same issue.

admin:show status

Host Name    : BAISUCM2
Date         : Fri Oct 2, 2015 09:31:47
Time Zone    : Argentine Time (America/Argentina/Buenos_Aires)
Locale       : en_US.UTF-8
Product Ver  : 8.6.2.22900-9
OS Ver       : 5.0.0.0-2
License MAC  : 

Uptime:
 09:31:48 up  1:49,  1 user,  load average: 6.74, 5.59, 5.53

CPU Idle:   25.13%  System:   06.53%    User:   56.28%
  IOWAIT:   11.06%     IRQ:   00.00%    Soft:   01.01%   Intr/sec: 1114.00

Memory Total:        4016920K
        Free:         695632K
        Used:        3321288K
      Cached:        1359388K
      Shared:              0K
     Buffers:          87560K

                        Total            Free            Used
Disk/active         14643120K        2519260K       11975096K (83%)
Disk/inactive       14410496K       12204552K        1462116K (11%)
Disk/logging        50179664K       32991440K       14598048K (31%)

Result Test:

admin:utils diagnose test

Log file: platform/log/diag1.log

Starting diagnostic test(s)
===========================
test - disk_space          : Passed (available: 2461 MB, used: 11695 MB)
skip - disk_files          : This module must be run directly and off hours
test - service_manager     : Passed
test - tomcat              : Passed
test - tomcat_deadlocks    : Passed
test - tomcat_keystore     : Passed
test - tomcat_connectors   : Passed
test - tomcat_threads      : Passed
test - tomcat_memory       : Passed
test - tomcat_sessions     : Failed - The following web applications have an unusually large number of active sessions: ccmcip.  Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
test - validate_network    : Reverse DNS lookup failed
test - raid                : Passed
test - system_info         : Passed (Collected system information in diagnostic log)
test - ntp_reachability    : Passed
test - ntp_clock_drift     : Passed
test - ntp_stratum         : Passed
skip - sdl_fragmentation   : This module must be run directly and off hours
skip - sdi_fragmentation   : This module must be run directly and off hours
test - ipv6_networking     : Passed

The strange issue that I can detect is at the screen, I don't know what that failure is, cause the disk led is not alarmed.

[Aaron Harrison]

CPU usage looks high - did you try the command I suggested above?

You'll want to have the failing disk replaced as soon as possible. That may or may not be contributing to the issues..

Aaron