We have a two node cluster (one pub., one sub) running v10.5.2. They are VMs (using medium OVA) running on similar Cisco Business Edition 600 hardware. Small environment (100 devices) with all phones registered to the sub.
When I go to install a firmware or device pack to both nodes, it takes considerably longer on the publisher. For example with cmterm-devicepack10.5.2.12014-1.cop.sgn it installed on the sub in 4 minutes. It took 38 minutes on the pub. Is this normal just because it's the pub? Or is this an indication that something's wrong?
Show hardware from pub:
And from Sub:
Can you check the CPU and memory utilization on Pub and Sub while doing such firmware installs or any upgrades. Also, try with a SFTP server which is in the same subnet as Pub, the difference is definitely too much between Pub and Sub so something is definitely wrong. Is there a reason for the CPU specs being diff for Pub and Sub?
Thanks for the reply. Yes, the TFTP server is on the same subnet.
I assume the difference in the CPU is because the underlying server hardware (Cisco Business Edition 6000) was purchased at different times, so may contain slightly different CPUs?
Yes, CPU does spike on pub. Alert shown below
Processor load over configured threshold for configured duration of time . Configured high threshold is 90 % watchdog/1 (17 percent) uses most of the CPU. Processor_Info: For processor instance 1: %CPU= 90; %User= 5; %System= 4; %Nice= 0; %Idle= 9; %IOWait= 80; %softirq= 1; %irq= 0. For processor instance _Total: %CPU= 91; %User= 5; %System= 4; %Nice= 0; %Idle= 9; %IOWait= 80; %softirq= 1; %irq= 0. For processor instance 0: %CPU= 92; %User= 5; %System= 5; %Nice= 0; %Idle= 8; %IOWait= 80; %softirq= 1; %irq= 0. The alert is generated on Mon May 11 08:32:48 EDT 2015 on node CUCMP. Memory_Info: %Mem Used= 80; %VM Used= 70. Partition_Info: Swap: %Disk Used=50. Active: %Disk Used=88. Common: %Disk Used=93. Process_Info: processes with D-State: flush-8:0 cdragent md5sum jbd2/sda2-8 postmaster#7 dbcfs#1 jbd2/sda6-8
Thanks for the details. Since this is the Publisher I would suggest opening a TAC case to find the root cause for CPU spike. You may check the output of the following to get some more details about the health of your Publisher
utils diagnose test
utils core active list
A install duration difference between pub and sub is normal. COP installs on the publisher must update the database while installs on subscribers simply copy files into the TFTP directory. Since a device pack has to do this for every phone model I wouldn't be at all concerned about the 30 minute time delta.
TFTP operations do not have a CPU quota. You should be installing these during a maintenance window as it could impact system performance.
Lastly, the CPU spec difference is most likely just the difference between a BE7000H (M4) UCS C240 M4SX TRC#1 and a BE6000H (M3) UCS C220 M3S TRC#3. This is not a cause for alarm.
Thanks guys! I opened a TAC case just incase. TAC confirmed that this is normal (passed all diagnostics).
Jonathan -- thanks for the advice re: maintenance windows.
One question about that. You mentioned the TFTP operation is not subject to a CPU quota. The actual long delay and CPU spike is during the package install (after the SFTP transfer). Is that expected as well?