cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
814
Views
5
Helpful
4
Replies

core dump 10.5.2.14071-1 .cucm imp

balaji.babu1
Level 1
Level 1

Helo all,

Am getting lot of core dump alert from cucm and IMP.Is there any bug reported for this ?

regards

balaji

4 Replies 4

Mohammed Khan
Cisco Employee
Cisco Employee

Core can occur because of various reason.

provide the output of below from all the server

++ utils core active list

++ utils core active analyze <core filename>

++ utils diagnose test

++ utils ntp status

++ show status

++ show hardware

Refer below link to TS coredump issues

https://supportforums.cisco.com/document/56631/troubleshooting-core-dumps

http://www.cisco.com/c/en/us/support/docs/unified-communications/unified-communications-manager-callmanager/118702-technote-cucm-00.html

Hi mohammed,

Please find the attached

regards

balaji 

Hi Balaji,

Collect the event viewer application logs and check for events prior to the generation of core dump. You may collect the RIS DC perfmon logs to verify the CPU usage.

http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cucm/service/8_0_1/rtmt/RTMT/rtlog.html

Look for alerts in event viewer like low virtual memory/Low swap partition/CPU pegging or transient connect attempts.

Aseem

Here is the summary of coredump analysis.

 core.9857.6.ccm.1474190133
 backtrace - CUCM
 ===================================
 #0  0xf7797430 in __kernel_vsyscall ()
#1  0xf6982871 in raise () from /lib/libc.so.6
#2  0xf698414a in abort () from /lib/libc.so.6
#3  0x0838911e in IntentionalAbort () at ProcessCMProcMon.cpp:88
#4  CMProcMon::verifySdlRouterServices () at ProcessCMProcMon.cpp:748
#5  0x0838934a in CMProcMon::callManagerMonitorThread (cmProcMon=0xd8b6e530) at ProcessCMProcMon.cpp:429
#6  0xf6ca5398 in ACE_OS_Thread_Adapter::invoke (this=0xc9611f78) at OS_Thread_Adapter.cpp:103
#7  0xf6c65491 in ace_thread_adapter (args=0xc9611f78) at Base_Thread_Adapter.cpp:126
#8  0xf6939bc9 in start_thread () from /lib/libpthread.so.0
#9  0xf6a3ac9e in clone () from /lib/libc.so.6

Above coredump occured because  CPU/Memory was probably spiking which can also cause SDL links to go OOS.

\\
 core.23089.6.ccm.1474295950
 backtrace - CUCM
 ===================================
 #0  0xf771d430 in __kernel_vsyscall ()
#1  0xf6908871 in raise () from /lib/libc.so.6
#2  0xf690a14a in abort () from /lib/libc.so.6
#3  0xf6b2afe7 in __gnu_cxx::__verbose_terminate_handler() () from /usr/local/cm/lib/libstlport.so.5.2
#4  0xf6b28e46 in __cxxabiv1::__terminate(void (*)()) () from /usr/local/cm/lib/libstlport.so.5.2
#5  0xf6b28e83 in std::terminate() () from /usr/local/cm/lib/libstlport.so.5.2
#6  0xf6b28fcb in __cxa_throw () from /usr/local/cm/lib/libstlport.so.5.2
#7  0xf6b29557 in operator new(unsigned int) () from /usr/local/cm/lib/libstlport.so.5.2
#8  0x09821598 in DMRemoteDeviceRegisterUnRegister::createCopy (this=0xd1e4f3e0) at ../Include/TDCLdevicesigs.hpp:2880
#9  0x09e4e953 in SdlProcessBase::prepareSignal (this=0xd48c2e28, _sdlSignal=..., _signalPriority=@0xd1e4d92c, _destPID=...) at SdlProcessBase.cpp:161
#10 0x09e4dc05 in SdlProcessBase::output (this=0xd48c2e28, rSignal=..., rProcessId=..., _signalPriority=kNormalPriority) at SdlProcessBase.cpp:236
#11 0x098118c2 in SMDMSharedData::propagateRegisterInfoToNewNode (callerP=0xd48c2e28, aRemoteDMPropagationPid=...) at SMDMSharedData.cpp:2576
#12 0x0868f20d in DMPropagation::initialized_SdlLinkISV (this=0xd48c2e28, s=...) at ProcessDMPropagation.cpp:167
#13 0x09e4f9cf in SdlProcessBase::inputSignal (this=0xd48c2e28, rSignal=0xc8fe6aa8, traceType=SdlSystemLog::SignalThreadedNoPriorities, highPriority=0, normalPriority=387, lowPriority=0, veryLowPriority=0, lazyPriority=0, dbUpdatePriority=0) at SdlProcessBase.cpp:406
#14 0x09e7a714 in SdlThreadedProcess::threadQueueReader (this=0xd48c2e28) at SdlThreadedProcess.cpp:110
#15 0x09e7a93f in SdlThreadedProcess::threadQueueReaderInit (sdlThreadedProcess=0xd48c2e28) at SdlThreadedProcess.cpp:75
#16 0xf68bfbc9 in start_thread () from /lib/libpthread.so.0
#17 0xf69c0c9e in clone () from /lib/libc.so.6

>This coredump requires additional logs, IMO Open a TAC case for TS

\\

core.28154.6.ccm.1474416770
 backtrace - CUCM
 ===================================
 #0  0xf76e0430 in __kernel_vsyscall ()
#1  0xf68cb871 in raise () from /lib/libc.so.6
#2  0xf68cd14a in abort () from /lib/libc.so.6
#3  0x09e956ae in SdlMsgQueueCirc::enqueueSignal (this=0xc878070, msg=0xc3078a08, priority=4) at SdlMsgQueueCirc.cpp:140
#4  0x09e42887 in enqueueSignal (this=0xc877f90, sdlSignal=0xc3078a08) at /view/BLD-cm_10_5_2-cct-ccm-d/vob/ccm/Common/Include/Sdl/SdlMsgQueue.hpp:66
#5  SdlNetworkService::enqueueSignal (this=0xc877f90, sdlSignal=0xc3078a08) at SdlNetworkService.cpp:559
#6  0x09e53eb1 in SdlRouter::enqueueSignal (this=0xc877370, sdlSignal=0xc3078a08) at SdlRouter.cpp:289
#7  0x09e4e93d in SdlProcessBase::prepareSignal (this=0xc875cb0, _sdlSignal=..., _signalPriority=@0xd5fca60c, _destPID=...) at SdlProcessBase.cpp:194
#8  0x09e4dc05 in SdlProcessBase::output (this=0xc875cb0, rSignal=..., rProcessId=..., _signalPriority=kNormalPriority) at SdlProcessBase.cpp:236
#9  0x094a1946 in SIPTcp::wait_SdlSPISignal (this=0xc875cb0, s=...) at ProcessSIPTcp.cpp:1710
#10 0x09e4f9cf in SdlProcessBase::inputSignal (this=0xc875cb0, rSignal=0xddb1fd8, traceType=SdlSystemLog::SignalRouterThread, highPriority=0, normalPriority=0, lowPriority=0, veryLowPriority=0, lazyPriority=0, dbUpdatePriority=0) at SdlProcessBase.cpp:406
#11 0x09e541c6 in SdlRouter::callProcess (this=0xc877370, _sdlSignal=0xddb1fd8, _deleteSignal=@0xd5fcc2ff, _traceType=SdlSystemLog::SignalRouterThread, _hp=0, _np=0, _lp=0, _vlp=0, _lzp=0, _dbp=0) at SdlRouter.cpp:257
#12 0x09e54470 in SdlRouter::scheduler (this=0xc877370) at SdlRouter.cpp:164
#13 0x09e5496f in SdlRouter::schedulerInit (sdlRouter=0xc877370) at SdlRouter.cpp:107
#14 0xf6882bc9 in start_thread () from /lib/libpthread.so.0
#15 0xf6983c9e in clone () from /lib/libc.so.6
 ====================================

 >This coredump requires additional logs, IMO Open a TAC case for TS

>  This might turn into new issue.

> Pro-active suggestions as follows

- Upgrade the system to last version

- Ensure VM specs matches OVA template.

- Check if there is latency on storage.

- Check ESXI Host performace