cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5885
Views
16
Helpful
0
Comments

Introduction

This article describes some steps to follow in order to troubleshoot Push Notifications not working from the Server and Client perspective.

 

Prerequisites

Requirements

 

Cisco recommends that you have knowledge of these topics:

  • Cisco Unified Communications Manager (CUCM)
  • Cisco Unified Instant Messaging and Presence (IM&P)
  • Cisco Jabber
  • Cisco Webex
 

Components Used

The information in this document is based on these software and hardware versions:

  • CUCM 11.5 SU8 and later
  • IM&P 11.5 SU8 and later

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

 

Background Information

Refer to the official Push Notification Deployment guideline for implementations of the deployment.

 

Important! Keep in mind that there are limitations on which versions do Push Notifications are supported.

 
The following table highlights minimum releases for basic Push Notifications support.The following table highlights minimum releases for basic Push Notifications support.
 

Problem: Network connectivity

Solution

 

Remember that the CUCM and IM&P (if used) required access to the public network to reach the Apple/Google Cloud to be able to send and receive push notifications. Thus, keep in mind that the following ports are required to be opened:

- TCP 443 between the CUCM and IM&P with the Cisco Cloud

- TCP 5223 from Jabber to Apple Cloud (Fallback port: TCP 443)

- TCP 9966 from CUCM Publisher to the rest of the CUCM Subscriber nodes.

Run these commands on all the CUCM and IM&P nodes via the Command Line Interface (CLI) to confirm that connectivity to the Cisco Cloud is successful.

- utils network connectivity fos-a.wbx2.com 443
- utils network connectivity idbroker.webex.com 443
- utils network connectivity push.webexconnect.com 443
 
The output should like this.
admin:utils network connectivity fos-a.wbx2.com 443
Connection to fos-a.wbx2.com 443 port [tcp/https] succeeded!
Service accessible
admin:utils network connectivity idbroker.webex.com 443
Connection to idbroker.webex.com 443 port [tcp/https] succeeded!
Service accessible
admin:utils network connectivity push.webexconnect.com 443
Connection to push.webexconnect.com 443 port [tcp/https] succeeded!
Service accessible

 

If you are using a proxy server, read the Proxy Support for Cloud Collaboration section in the deployment guide.

The username and password are not mandatory for Proxy usage, although it is highly recommended to avoid any security breaches.

 

Problem: Certificates

Common issues in the APNS flow are usually caused by missing or expired certificates.

Refer to the Certificates for Cloud Connection section of the deployment guide for more information.

In some instances you might encounter an error saying: The resource lock platform.api.network.address is currently locked by process 29775@sgp-cm1(NetMessageDispatch-80). Please try again later. When trying to upload the Tomcat-trust certificate. See the image below.

MiguelCastillomiguecas_1-1666034763004.png

A restart of the DRF Master service requires to be performed.

 

Cloud Onboarding fails due to bad certificates.

"Push Notification/Activation Code Onboarding Settings cannot be configured as a valid certificate is not present in trust store. Either upload the certificates manually or check the check box to have Cisco manage the Cisco Cloud Service CA Certificates. For HTTPS proxy make sure the valid certificates are present for tomcat and tomcat trust store"

 

MiguelCastillomiguecas_2-1666035845801.png
 

Check the following:

- WebEx certificate [wbx] is not missing nor expired.

- GoDaddy certificate is not missing or expired.

- Hydrant certificate is not missing or expired.

- If your goal is to make Cisco manage the Cisco Cloud Service CA Certificates, review your network, firewall and proxy settings, as those might be blocking the download of the certificates.

 

In the Push Notification Logs you might see an error like this in case there is a network issue:

2020-10-02 00:25:13,668 ERROR [http-bio-443-exec-23] utilities.CloudOnboarding - Service validation Error:
org.apache.http.conn.ConnectTimeoutException: Connect to fos-a.wbx2.com:443 [fos-a.wbx2.com/13.59.223.243] failed: connect timed out
 

Once the certificates have been uploaded correctly, keep in mind that the Cisco Tomcat and Cisco Push Notification Services require to be restarted in all the CUCM nodes, and the Cisco Tomcat service in all the IM&P nodes. These actions do not disrupt the workflow of the system, you will just face issues with the web GUI and possible log in errors from the Jabber while the Tomcat service is restarting.

Afterwards, proceed with the Generate Voucher action and continue with the process to enable Push Notifications.

Problem: Onboarding webpage

Many problems start at the beginning of the implementation, specifically in the Cisco Cloud Onboarding webpage.

MiguelCastillomiguecas_1-1665714994458.png

Solution

 
The CUCM must be licensed prior to the onboarding:
- 11.5(x) with Prime License Manager (PLM)
- 12.x with Smart Licensing
 
Push Notification Settings cannot be configured because the required voucher has not been generated in License Manager. Please click on the "Generate Voucher" button to generate a voucher from License Manager.
This might happen if the CUCM is not licensed, CUCM is not able to reach Apple's Cloud, or the Generate Voucher was simply not hit.
 
UnknownHostException Check if your DNS configuration is correct.
Confirm that the DNS is configured correctly, and confirm that the FQDN resolution is working.
Check the Network Connectivity.
If the Cisco Cloud is not reachable, firewall ports might need to be opened or a Proxy should be used to reach the External Addresses.
 
The Cisco Cloud Onboarding webpage does not loads
This might be an issue with the Push Notification service and with some specific certificates related to the Push Notification service.
 
Review the XCP Config Manager logs and make sure that the connections do not have a 400 bad request, like the example below:
2022-09-28 07:24:26,596 ERROR [Timer-4132] xmlframework.XCPConfigMgr - 400 Bad Request: invalid_request, unsupported_grant_type, invalid_client, invalid_refresh_token, tokenlimit_reached
2022-09-28 07:24:26,603 ERROR [Timer-4132] xmlframework.XCPConfigMgr - PNS : FetchAndStoreAccessToken : Did not get a valid 200 response value for accessToken.
2022-09-28 07:34:26,607 INFO  [Timer-4133] utilities.CloudOnboarding - TRACKING ID:::::::FOS_0a2e0098-1582-4f82-91b1-2cb3cfb97251
2022-09-28 07:34:26,857 ERROR [Timer-4133] utilities.TomcatTrustManager - checkServerTrusted:entered 
2022-09-28 07:34:26,864 ERROR [Timer-4133] utilities.TomcatTrustManager - checkServerTrusted:entered 2
 
If you are seeing that, you are hitting the defect: Bad Request while fetching the Access Token - CSCvi01660
 

Problem: Calls not reaching the Jabber device when the Jabber is unregistered

Solution

 
When the Jabber device is not in the foreground, it will get unregistered from CUCM, and when someone calls the number associated with the Jabber, it will receive a Push Notification.
 
Confirm that:
- The onboarding webpage does not present any errors.
- There are no network connectivity issues.
- The associated WebEx certificates are not expired.
- The CUCM and Jabber can reach the Cisco Cloud and Apple/Google Cloud respectively.
- The iOS and Android devices must allow notifications from the Cisco Jabber application.
 

It is common to track the call from the CallManager Traces before looking into the Jabber logs, to confirm that the CUCM sent the call invite to the cloud.

Look for the CI in the CUCM traces like in the excerpt below.

30695118.001 |15:03:46.684 |AppInfo |PushNotify::wait_SsPushNotifyReq() Sending PNReq for trackingID = CUCMCallP_a9869065-b4e8-4eb5-a9aa-d239a2ea7b0b_deploy:onprem_clusterinfo:PIMCO-CUCM-NewportBeach-2, pushId = 2-221133330

 

In the Jabber.log
Generation of the Push Notification
PushNotify::wait_SsPushNotifyReq() Sending PNReq for trackingID = CUCMCallP_e7486baa-0127-43a3- 8711-cb2ab4ffeb2f_deploy:onprem_clusterinfo:StandAloneCluster-1 , CI = 28845312 
 
Always look for the CUCMCallP line, as that is the unique identifier to track the Push request.
 
Notification received from the CallManager Service
queueMessage() message:{"messagetype":"REQUEST_PUSH_NOTIFICATION","TYPE":"incomingcall","REF":","28845312,"DEVICETOKEN":" 7da51a8aee1f469267e53398c9d7cc1187a9999e9dc7a2140c3d195acddf79b5 ","KEY":"-21DDpT0RwyA4G5cApJleqE6L2pmQaVXTThLQCtjpY","ALGO":"A256GCM","APNS","TRACKINGID":"CUCMCallP_e7486baa-0127-43a3-8711
cb2ab4ffeb2f_deploy:onprem_clusterinfo:StandAloneCluster-1}
 
The Response from Cisco Cloud sent to CallManager Service
{"MESSAGETYPE":"RESPONSE_PUSH_NOTIFICATION","TRACKINGID":"CUCMCallP_e7486baa 0127 43a3 8711-
cb2ab4ffeb2f_deploy:onprem_clusterinfo:StandAloneCluster1","STATUS":"200","TEXT":""}
 
Check for VoipSocket configuration
[ConfigService-ConfigStoreManager] [getValue] - key : [enablevoipsocket ] skipLocal : [0] value: [false] success: [true] configStoreName: [TftpConfigStore]
 
iOS Push Launches Jabber
[UI.Action.System] [-[JabberIOSAppDelegate application:didFinishLaunchingWithOptions:]] iOS system(not user) launch jabber at background
 
Push Notification Received
[UI.Action.System] [-[YLCVoipPushManagerpushRegistry:didReceiveIncomingPushWithPayload:forType:]] - jabber receive push message. payload.type: PKPushTypeVoIP , for type: PKPushTypeVoIP, Cisco Tracking ID: CUCMCallP_e7486baa 0127-43a3-8711-cb2ab4ffeb2f_deploy:onprem_clusterinfo:StandAloneCluster-1, teams trackingId:(null)
 

Problem: Instant Messaging not reaching the Jabber device when the Jabber is unregistered

Solution

When the Jabber device is not on the foreground, it will get unregistered from CUCM, and when someone calls the number associated to the Jabber, it will receive a Push Notification.
 
Confirm that:
- The onboarding webpage does not present any errors.
- There are no network connectivity issues.
- The associated WebEx certificates are not expired.
- The CUCM/IM&P and Jabber can reach the Cisco Cloud and Apple/Google Cloud respectively.
- Multiple Device Messaging (MDM) and Stream Management must be enabled in IM&P
- The iOS and Android devices must allow notifications from the Cisco Jabber application.
 
Important questions to keep in mind:
- Is the user logged on to the Cisco Jabber for mobile?
- Does the Mobile user already have an active conversation (IM) with the other user in another Jabber client?
- How long has the user kept the mobile in sleep mode or the Jabber in the background?
 
XCP Config Manager Service
 
Notification received from XCP Router Service 
xcpconfig: Packet from Router<message xmlns="jabber:client" from="iOSuser@domain.com/jabber_1 1 1 1" to="xcpconfigmgr.imp-domain-com" id="a6efdc94-1486-489e-a76b-b9e021"><publish xmlns="http://protocols.cisco.com/push:?><subscriber service= "APNS">7da51a8aee1f469267e53398c9d7cc1 187a9999e9dc7a2140c3d195acddf79b5</subscriber><session>124e4c86-dbaO-4f06-86dd-328b27bb04b5</session><notify type="chat" from=lIuser@domain.com/jaber_2222" to="iOSuser@domain.com" id="257eee24:46f7:4168:9350:940cad203fa3" ... body="APNS Test Message" /><encrypt alg="A256GCM">...
 
PushXMPP packet to Cloud
xmlframework.PushXMPP - <message xmlns="jabber:client" from="iOSuser@domain.com/jabber_1111" to="xcpconfigmgr.imp12papns-domain-com" id="a6efdc94-1486-489e-a76b-b9e021ffc0d6"><publish xmlns="http:// protocols.cisco.com/push:2"><subscriber service="APNS">7da51a8aee1f469267e53398c9d7cc1187a9999e9dc7a2140c3d195acddf79b5</subscriber><session>124e4c86-dba0-4f06-86dd-328b27bb04b5</session><notify type="chat" from=" user@domain.com/jaber_2222" to="iOSuser@domain.com" id="257eee24:46f7:4168:9350:940cad203fa3" ... body="APNS Test message " /><encrypt alg="A256GCM">NUkF98dgr3_K8PlGqaY0pbXnCe1aCc5QpUcKbJhy6s</encrypt></publish></message>
 
Sending packet to Cloud
xmlframework.PushXMPP - PNS: onPacket: Sending packet to: https://push.webexconnect.com/jabber/apns/prod
 
PushNotification Sent with TrackingID
xmlframework.PushPacketHandler - PNS: pushCall: Sending Push Notification for packet with Cisco Tracking ID: IMPXCPConfigMgr_a6efdc94-1486-489e-a76b-b9e021ffc0d6 ... fa3_deploy:onprem
 
Push response received from the cloud
xmlframework.PushPacketHandler - PNS: Push Successful, received successful response code from REST . Cisco Tracking ID: IMPXCPConfigMgr_a6efdc94-1486-489e-a76b-b9e021ffc0d6 _... fa3_deploy:onprem
 
Jabber logs
 
Push Enabled
[IMPServices InstantMessageConversationServiceImpl] [IsPushEnabled ] - <PUSH> IS_PUSH_ENABLED: 1

Pushpacket received with TrackingID
-[-[YLCVoipPushManagerpushRegistry:didReceiveIncomingPushWithPayload:forType:]] - jabber receive push message.payload.type:PKPushTypeVoIP, for type: PKPushTypeVoIP, Cisco-Tracking ID: IMPXCPConfigMgr_a6efdc94 1486-489e-a76b-b9e021ffc0d6_ ... fa3_deploy:onprem, teams trackingId:(null)
 
IM Message received
Recv :<message iOSuser@domain.com" notify="1" subtype="mdm:rcvd" to="iOSuser@domain.com/jabber_1111" type=" chat" uuid ="9f2cf74c-9848-481b-90e7-4f8fd27809ab"><delay from="domain.com" stamp="2019-06-6T17:02:34.962258Z" xmlns=" urn:xmpp:delay message from="user@domain.com/jaber_2222" id="257eee24:46f7:4168:9350:940cad203fa3" to="iOSuser@domain.com" type="chat"
xml:lang en "><body>******</body></ thread>connect30079</thread> ...

 

How to test

In order to verify that APNS is working, perform the next steps.

Step 1. Use an iPhone or Android device with Jabber running in the foreground.

Step 2. Confirm in the CUCM Administration web page > Devices > Phones > TCT or BOT that the device is registered.

Step 3. Close the Jabber application in the cellphone. Wait some time and then check in the Phones menu that the device is unregistered.

Step 4. Make a call and wait a few seconds. The phone should get registered and the call should start ringing in the cellphone.


Known Defects

Version 14

  • CSCwc62283Push notifications stop working with the implementation of Bouncy Castle. Restarting the Push Notification service will allow pushes to work again.

Version 12.5 (and possibly later)

  • CSCwa88279Incoming Calls to Cisco Jabber and WebEx (Android and iOS) Will Fail while in background mode.
  • CSCvy27184: Cisco Cloud Onboarding failed on 12.0.1 SU5
  • CSCvq21687Not all push-enabled devices initialised if unregistered at the time of ccm restart

Version 11.5 (and possibly later)

  • CSCvn18745: Push notification is not working with INFO log level for XCP Config Manger logs.
    • Fixed release:11.5.1Su6
  • CSCvi01660: APNS: Bad Request while fetching the Access Token.
    • Document defect corrected in the current documentation
  • CSCvi58557: APNS Onboarding doesn't implement Certs when checking "I want Cisco To manage my CA Certificates”.
    • Fixed release: 11.5.1Su5 or above.
  • CSCvc51134: CUCM Cloud onboarding cannot be completed because orgname is not populated in the onboardingdetails .
    • Fixed release: 11.5.1Su3 or above. Condition: This happens from an upgrade, so a workaround might be required based on this upgrade path.
  • CSCvs51505Memory leaks in Cisco Push Notification Service
    • Fixed release: 11.5su6 or later, 12.5su3 or later and 14

No version dependant

  • CSCvm67800: Jabber sometimes doesn't receive messages or calls after iOS12 has been installed.
    • Fix: Enable APNS .
  • CSCvm80565: Voice Call Apple Push notifications are not consistent because of iOS limitation .
    • Document defect corrected in current versions of the documentation.
  • CSCvp31510: CSSM 6.2 Intermediate certificate (Satellite CA/Sub CA) missing in TPL CUCM/APNS. Fixed release: 6.2.1 or above.
  • CSCwc37537: Jabber for iOS 14.1.2 not receiving IM when the application is running in the background.
    • Status: Open
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: