cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1343
Views
0
Helpful
3
Replies

Subcribers not sending CDR files to Publisher in the cluster.

baskaranm
Level 1
Level 1

HI,

 

We have call manager cluster 8.6 version and there are 5 nodes in the cluster.

 

I could see CDR files generated in the subscriber nodes but it is not uploaded publisher node CDR repository .

 

I am not seeing any CDR files in Publisher preserve folder.

 

 

2014-05-06 19:43:41,517 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:213) - After failure to send files in the previous attempt. Creating new ftpClient object
2014-05-06 19:43:41,517 DEBUG [main] sftpapi.ftpClient (ftpClient.java:148) - ftpClient constructor
2014-05-06 19:43:41,517 DEBUG [main] sftpapi.ftpClient (ftpClient.java:150) - host = 166.98.48.155
2014-05-06 19:43:41,517 DEBUG [main] sftpapi.ftpClient (ftpClient.java:151) - user = ccmservice
2014-05-06 19:43:41,517 DEBUG [main] sftpapi.ftpClient (ftpClient.java:153) - localDir = null
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.ftpClient (ftpClient.java:154) - remoteDir = null
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.ftpClient (ftpClient.java:155) - secureFTP = true
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.ftpClient (ftpClient.java:156) - permKey = true
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.ftpClient (ftpClient.java:157) - Component Name : cdragent
2014-05-06 19:43:41,518 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:216) - After creating the ftpClient object
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.ftpClient (ftpClient.java:179) - connect()-->
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.ftpClient (ftpClient.java:182) - connect(): connect for SFTP
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:94) - Inside the SFTPConnection Constructor
2014-05-06 19:43:41,518 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:364) - checkConnection for Publisher using Key Authentication-->
2014-05-06 19:43:41,519 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:365) - checkConnection: UserName : ccmservice
2014-05-06 19:43:41,519 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:366) - checkConnection: hostname : 166.98.48.155
2014-05-06 19:43:41,519 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:215) - getSshClient-->
2014-05-06 19:43:41,519 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:272) - getSshConnector-->
2014-05-06 19:43:41,520 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:227) - SSH Connection object obtained
2014-05-06 19:43:41,566 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:405) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa
2014-05-06 19:43:41,576 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:428) - The User is Authenticated. Need to get the SftpClient
2014-05-06 19:43:41,616 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:432) - SftpClient is obtained Successfully
2014-05-06 19:43:41,617 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:453) - <--checkConnection for Publisher using Key Authentication
2014-05-06 19:43:41,617 DEBUG [main] sftpapi.ftpClient (ftpClient.java:194) - The status of checkConnection : true
2014-05-06 19:43:41,617 DEBUG [main] sftpapi.ftpClient (ftpClient.java:217) - connect():Successfully connect to 166.98.48.155
2014-05-06 19:43:41,617 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:224) - After attempt to get ftpClient connection
2014-05-06 19:43:41,617 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:237) - Sleeping for 60 seconds
2014-05-06 19:44:41,628 INFO  [main] cdragent.cdrAgent (cdrAgent.java:199) - waking up
2014-05-06 19:44:41,628 INFO  [main] cdragent.cdrAgent (cdrAgent.java:384) - handleCDRFiles -->
2014-05-06 19:44:41,663 INFO  [main] cdragent.cdrAgent (cdrAgent.java:285) - At the moment, # of files ready to be transfered to Publisher is 14011
2014-05-06 19:44:41,763 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:426) - sendFile(): Try bldSendFileCmds
2014-05-06 19:44:41,763 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:430) - FtpManager - sendFile() : The Parent directory of the remote directory send the files : /var/log/active/cm/cdr_repository/
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:849) - bldSendFileCmds(): -->
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:862) - bldSendFileCmds(): - no need to build command file
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:863) - local directory: /var/log/active/cm/cdr/
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:864) - remote directory: /var/log/active/cm/cdr_repository/trans/
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:865) - final remote directory: /var/log/active/cm/cdr_repository/tmp/
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:867) - file: cdr_Dublin_03_201404250000_140996
2014-05-06 19:44:41,764 DEBUG [main] sftpapi.ftpClient (ftpClient.java:869) - bldSendFileCmds(): return <--
2014-05-06 19:44:41,764 DEBUG [main] cdrrep.Util (Util.java:102) - checkJVMFreeMemory(): -->
2014-05-06 19:44:41,764 DEBUG [main] cdrrep.Util (Util.java:158) - checkJVMFreeMemory(): <--
2014-05-06 19:44:41,764 INFO  [main] cdragent.cdrAgent (cdrAgent.java:433) - CDRAgent before sending the file: JVM status: [Max = 227.56MB, Total = 58.44MB, Free = 48.29MB (82.64%), Used = 10.15MB (17.36%)]
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:499) - sendFiles(): -->
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:510) - sendFiles(): file to send = /var/log/active/cm/cdr/cdr_Dublin_03_201404250000_140996
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:511) - sendFiles(): remoteDirectory = /var/log/active/cm/cdr_repository/trans/
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:518) - sendFiles():remoteDirectoryFinal(NOT NULL)=/var/log/active/cm/cdr_repository/tmp/
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:519) - sendFiles():remoteFilePathFinal=/var/log/active/cm/cdr_repository/tmp/cdr_Dublin_03_201404250000_140996
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:521) - sendFiles(): localFilePath = /var/log/active/cm/cdr/cdr_Dublin_03_201404250000_140996
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:522) - sendFiles(): remoteFilePath = /var/log/active/cm/cdr_repository/trans/cdr_Dublin_03_201404250000_140996
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.ftpClient (ftpClient.java:526) - sendFiles(): uploadFile
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:163) - sendFileThroughSFTP -->
2014-05-06 19:44:41,765 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:174) - Running put command to copy the file
2014-05-06 19:44:41,767 ERROR [main] sftpapi.SFTPConnection (SFTPConnection.java:194) - Sftp Status Exception : com.maverick.sftp.SftpStatusException: Permission denied.: Permission denied
2014-05-06 19:44:41,768 DEBUG [main] sftpapi.ftpClient (ftpClient.java:538) - Sftp Status Exception
2014-05-06 19:44:41,768 ERROR [main] sftpapi.ftpClient (ftpClient.java:539) - SFTP error in transferring file to another machine.
2014-05-06 19:44:41,768 ERROR [main] sftpapi.ftpClient (ftpClient.java:540) - com.maverick.sftp.SftpStatusException: Permission denied.: Permission denied
2014-05-06 19:44:41,768 ERROR [main] sftpapi.ftpClient (ftpClient.java:559) - Exception while sending files: com.maverick.sftp.SftpStatusException: Permission denied.: Permission denied
2014-05-06 19:44:41,768 DEBUG [main] cdrrep.Util (Util.java:102) - checkJVMFreeMemory(): -->
2014-05-06 19:44:41,768 DEBUG [main] cdrrep.Util (Util.java:158) - checkJVMFreeMemory(): <--
2014-05-06 19:44:41,768 INFO  [main] cdragent.cdrAgent (cdrAgent.java:443) - CDRAgent after sending the file: JVM status: [Max = 227.56MB, Total = 58.44MB, Free = 47.77MB (81.74%), Used = 10.67MB (18.26%)]
2014-05-06 19:44:41,768 ERROR [main] cdragent.cdrAgent (cdrAgent.java:509) - sftp failed for file cdr_Dublin_03_201404250000_140996
2014-05-06 19:44:41,768 ERROR [main] cdragent.cdrAgent (cdrAgent.java:519) - Alarm: Unable to successfully send file to repository node.
2014-05-06 19:44:41,768 INFO  [main] cdragent.cdrAgent (cdrAgent.java:548) - raising alarm CDRAgentSendFileFailureContinues
2014-05-06 19:44:41,769 INFO  [main] alarmInf.AlarmSender (AlarmSender.java:107) - alarmName: CDRAgentSendFileFailureContinues
2014-05-06 19:44:41,769 DEBUG [main] alarmInf.AlarmSender (AlarmSender.java:113) - paramName[0] = CDRRepositoryNodeAddress -> paramValue[0] = 166.98.48.155
2014-05-06 19:44:41,769 DEBUG [main] alarmInf.AlarmSender (AlarmSender.java:113) - paramName[1] = CDRAgentNodeAddress -> paramValue[1] = DUB2IPTCCMPIN22/166.98.48.157
2014-05-06 19:44:41,769 DEBUG [main] alarmInf.AlarmSender (AlarmSender.java:119) - sendAlarm ...
2014-05-06 19:44:41,769 INFO  [main] cdragent.cdrAgent (cdrAgent.java:575) - <-- handleCDRFiles
2014-05-06 19:44:41,769 DEBUG [main] sftpapi.ftpClient (ftpClient.java:283) - disconnect(): <-ftpClient->
2014-05-06 19:44:41,769 DEBUG [main] sftpapi.ftpClient (ftpClient.java:287) - disconnect(): SFTP Connection -->
2014-05-06 19:44:41,769 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:492) - disconnect(): SFTP Connection -->

 

.

3 Replies 3

Could you please check with TAC whether your CUCM version is also affected by the bug CSCsz99841 as I see "ccmservice" user in your logs too? Description Symptom: Cluster Manager will update ccmservice user SSH security keys whenever a new node is added or removed from the cluster Conditions: CDR Agent cannot send CDR files from sub to pub using sftp. This is because SFTP is initiating a connection using the user 'ccmservice' Workaround: i) Stop CDR Agent service on Call Processing Node ii) Gain root access via remote access support account. And then Copy over the authorized keys in the specific user's SSH directory: /home/sftpuser/.ssh/authorized_keys file to /home/ccmservice/.ssh/authorized_keys Root CLI command: cp /home/sftpuser/.ssh/authorized_keys /home/ccmservice/.ssh/authorized_keys iii) Start CDR Agent service on Call Processing Node. Further Problem Description: This was caused by changes that were made to tighten up file permissions around the CUCM file structure
//Suresh Please rate all the useful posts.

Hi suresh,

 

thanks for the update..

 

any other suggestions before reaching cisco TAC

manpreetsingh46
Level 1
Level 1

From logs seeing that sftp is initiating a connection with username "ccmservice". This is a known bug caused by incorrect permissions and you will need to open a TAC case as the authorized keys in the specific user's SSH directory need to be copied manually from root access.

CSCsz99841