Delay in email notification of Cisco Unity Connection Unified messaging

[voipeee]

Hi Experts,

I am facing the below issue and your help will be highly appreciated:

We have 2 Call Manager Cluster(A & B) and 1 Unity Cluster A. Call Manager Cluster A and Unity Cluster A are at the same location. Cluster B user's are using Voicemail of Cluster A. Now the issue is whenever anyone leaves voicemail for the Cluster B user's. There is a delay of almost 2 hours. So far Cluster A user's have not reported any issue and I assume everything is fine there. We are using office 365 as Unified messaging.

I have checked/Tested and everything seems fine to me.

Here are some of the Mailbox logs.

11:38:43.599 |29935,outlook.office365.com,{e8556395-dc17-4826-a787-9660a5164c97},CsMbxSync,11,CCsMsMbxLink::Kill() Beginning Kill For Mailbox Srvc Entry Data: (Cnx Mbx Id: Cnx Mbx Id: (Mbx Uid: {597ebb02-46a4-4954-a0b7-5ce5bf16e55d}, Inbox Folder Uid: {288765fb-1ce4-49bc-bab3-85ef41861d83}, Mail Store: UnityMbxDb1, Inbox Folder Name: inbox), Srvc Data: External Srvc Data: (Ext Srvc Oid: {dfc6cc37-f72b-442e-bd4e-20286ce05a5f}, Display Name: Office 365, Auth Scheme: 1, Is Enabled: 1, Srvc Supports Sync: 1 , Exch Do Auto Discover: 1, Exch Do Auto Discover 2003: 0, Group Info: CO2PR04, Security Transport Type: 1, Server: outlook.office365.com, Service Account: unityaccount@example.onmicrosoft.com, Service Password: XXXXXXXXX, Service Type: 5, Exch Service Type: 1, Trust Cert Dir: /usr/local/platform/.security/tomcat/trust-certs/, Ldap Security Transport Type: 0, Ldap Validate Server Certificate: 0, Validate Server Certificate: 0, Notification Type: 2, Is Impersontaion Enabled: 1, Proxy Ip Address: 10.249.209.6:8080), Mbx Data: Mbx Data: (Email Addr: testuser@example.com, Subscriber Oid: {e8556395-dc17-4826-a787-9660a5164c97}, Sync Enabled: 1, SESM Oid: {43162b62-f38a-4af0-81fe-694859a04536}, DTMFAccess ID: 86073109))
11:38:43.600 |29935,outlook.office365.com,{e8556395-dc17-4826-a787-9660a5164c97},CsMbxSync,14,CCsMsEwsAggregateNotificationMgr::Remove() 0xe7f5886c removing subscription e793e1a8-b6fd-47b9-b091-285d73b656df as found in monitored map
11:38:43.600 |29935,outlook.office365.com,{e8556395-dc17-4826-a787-9660a5164c97},CsMbxSync,15,CCsMsCnxMailEventProvider::UnRegister() Unregistering subscription for mbx id {597ebb02-46a4-4954-a0b7-5ce5bf16e55d}
11:38:43.600 |29935,outlook.office365.com,{e8556395-dc17-4826-a787-9660a5164c97},CsMbxSync,11,CCsMsMbxLink::Kill() Finished Kill

[Jim duPuis]

I had a similar issue with O365 and unified messaging. We found that there were messages delivered, but the queue length was so long that it took from 15 minutes to a couple hours for messages to show in email. I think we narrowed it down by turning on the following traces:

CsMBXSync: Level 10 to 23

CSWebDav: Level 10, 11, 12, 13, 14

CsEWS: All Levels

EWSNotify: All Levels

MTA: Level 10 to 30

Cuca: All Levels

CsEsMbxLocator: All levels

Send a test voicemail to a user, and then collect these traces from RTMT for: Connection Mailbox Sync, Connection Jetty, Message Transfer Agent. I think Mailbox Sync logs contained the data needed.

We had a high number of outstanding requests. We resolved it by increasing the number of connections. Cisco has some formulas for determining connection count here https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/connection/11x/design/guide/b_11xcucdg/b_11xcucdg_chapter_0110.html#ID-2337-000000da

We just ended up increasing it a little and watching those logs until there were no outstanding requests.