topic Re: Jabber MRA cannot make external calls in Collaboration Applications

Jabber MRA cannot make external calls

cehr_itsupport — Mon, 18 Mar 2019 02:10:55 GMT

Hi,

Hoping someone can help with a Jabber MRA problem. This seems to be affecting the later Jabber release 1.8.x and I on IOS (TCT) and Android (BOT) plus Windows and Mac devices and I reckon it's codec related. The problem is we cannot dial any external landline or cell \ mobile numbers through MRA; this happens when the MRA device is connected to WiFi and 4G\cell. The Called party device rings once and then the call automatically disconnects. The call logs report "501 Not Implemented" and "Reason: Q.850;cause=65".

Call route is Jabber MRA device > Expressway-E > Expressway-C > CUCM > SIP Trunk to SIP provider (BT)

Windows CSF devices work fine through MRA using earlier Jabber v11.7.0.
Calls to internal numbers from any MRA device are OK (on Wifi and 4G).

CUCM is v11.5.1.13900-52
Expressway Edge and Core are vX8.10.1
Jabber on IOS and Android is latest v11.9.1

I've had a look at the call logs and I can see that the INVITE from the IOS \ Android devices includes an additional audio codec (number "111") compared to the CSF:

INVITE from Expressway to CUCM
==========================

v=0
o=tandberg 0 1 IN IP4 xxxxx
s=-
c=IN IP4 xxxxx
b=AS:1024
t=0 0
a=cisco-mari:v1
a=cisco-mari-rate
m=audio 55360 RTP/AVP 114 9 104 105 0 8 18 111 101
a=rtpmap:114 opus/48000/2
a=rtpmap:9 G722/8000
a=rtpmap:104 G7221/16000
a=fmtp:104 bitrate=32000
a=rtpmap:105 G7221/16000
a=fmtp:105 bitrate=24000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:111 x-ulpfecuc/8000
a=fmtp:111 max_esel=1420;m=8;max_n=32;FEC_ORDER=FEC_SRTP
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=extmap:14/sendrecv http://protocols.cisco.com/timestamp#100us
a=sendrecv
a=rtcp:55361 IN IP4 10.x.x.x

INVITE from CUCM to SIP Provider - includes the same 111 codec:
=================================================

v=0
o=CiscoSystemsCCM-SIP 4388142 1 IN IP4 xxxxx
s=SIP Call
c=IN IP4 xxxxx
b=TIAS:8000
b=AS:8
t=0 0
a=cisco-mari:v1
a=cisco-mari-rate
m=audio 55360 RTP/AVP 18 114 111 101
a=extmap:14/sendrecv http://protocols.cisco.com/timestamp#100us
a=rtpmap:114 opus/48000/2
a=rtpmap:111 X-ULPFECUC/8000
a=fmtp:111 max_esel=1420;m=8;max_n=32;FEC_ORDER=FEC_SRTP
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=rtcp:55361 IN IP4 10.x.x.x

OFFER from SIP Provider back to CUCM
=================================

We see the SIP Provider offering back the following. The "m=8;max_n=32;FEC_ORDER=FEC_SRTP 0" attribute seems completely out of place.

v=0
o=genband 322521088 1508711079 IN IP4 xxxxx
s=-
c=IN IP4 xxxxx
t=0 0
m=audio 47230 RTP/AVP 18
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=ptime:20
m=8;max_n=32;FEC_ORDER=FEC_SRTP 0
a=rtpmap:18 0 annexb=no
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15

Question then is is this an issue with the Jabber app, CUCM, Expressway or the SIP provider, or something else completely?

Thanks

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Mon, 06 Nov 2017 22:46:09 GMT

I think that it has something to do with the codec G729. Why is your service provider working with G729? Why not G711U or G711A? I think if they'll negotiate G711 U/A or it'll work fine.
In the document "Planning Guide for Cisco Jabber 11.8" there's some note about the supported codecs, you can see it here:
https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/jabber/11_8/cjab_b_planning-guide-jabber-118/cjab_b_planning-guide-jabber-118_chapter_01.html#CJAB_RF_S7E65AD2_00

Not 100% sure about what they're stating about G729A, about the low-bandwidth mode... but maybe it is something that is getting in you way.
Can't you ask your service provider to work on G711 A/U? Or you're working with G729 on intention?

Edit:

Oh, I now understand what is the "low bandwidth mode" they are referring to. In your Cisco Jabber settings in the mobile phone application, you can choose in the Audio & Video options (as I recall) if you want to enable "Low Bandwidth mode". Just out of curiosity, try to enable it and make a call. If it works, so the solution is to change the negotiated codec from ITSP side to be G711.

Re: Jabber MRA cannot make external calls

cehr_itsupport — Tue, 07 Nov 2017 12:26:05 GMT

Thanks Slavik,

By low bandwidth mode we mean the setting within the Jabber app which forces G729a when the device is connected to low bandwidth. We are having the same problems whether this setting is enabled or not. The SIP Traces show that Expressway is offering codec "111" in the INVITE to CUCM and CUCM is then offering this to the SIP provider:

m=audio 55360 RTP/AVP 114 9 104 105 0 8 18 111 101
a=rtpmap:114 opus/48000/2
a=rtpmap:9 G722/8000
a=rtpmap:104 G7221/16000
a=fmtp:104 bitrate=32000
a=rtpmap:105 G7221/16000
a=fmtp:105 bitrate=24000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:111 x-ulpfecuc/8000
a=fmtp:111 max_esel=1420;m=8;max_n=32;FEC_ORDER=FEC_SRTP
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=extmap:14/sendrecv http://protocols.cisco.com/timestamp#100us
a=sendrecv
a=rtcp:55361 IN IP4 [EXP-C IP]

I've no idea what the "111" codec is and it is not listed as a supported codec in the Jabber 11.9 Deployment Guide. So either there's something odd in the Jabber MRA apps, or Expressway is doing something odd.

The codec offer received by CUCM from the SIP provider includes only G.729, but includes a strange m=8 line (in bold) which seems to be related to the original offer sent by CUCM:

m=audio 47230 RTP/AVP 18
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=ptime:20
m=8;max_n=32;FEC_ORDER=FEC_SRTP 0
a=rtpmap:18 0 annexb=no
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15

The SIP provider supports G.722, AMR-WB(G.722.2), G.711 A-Law, G.711 µ-Law, G.729/G.729A,
AMR, GSM-EFR, T.38 for fax and RFC2833/4733 telephone-events for DTMF.

Any suggestions greatly appreciated!

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Tue, 07 Nov 2017 12:41:09 GMT

Hi,
The "111" you're seeing is a codec named OPUS, which introduced in CUCM from version 11.x. This codec probably won't do you any harm here, but you can try to disable it. Just go to CUCM -> Service Parameters -> CallManager (select it). Find "Opus Codec Enabled", and check it as Disabled. It should do the trick.
Anyway, it is good the ITSP is supporting all the codecs you said they support, but as you can see the prefer to negotiate SDP with you in G729. So what I purpose is actually contact your ITSP and ask them to prefer G711A/U in the negotiation. Because on your end is fine, you're suggesting all the relevant codes, it's them that are only giving you G729 in the negotiation.

Re: Jabber MRA cannot make external calls

cehr_itsupport — Tue, 07 Nov 2017 13:03:43 GMT

Hi Slavik,

Disabling Opus didn't work unfortunately... With OPUS disabled in CUCM we see that INVITE from CUCM to SIP provider now does not include codec 114 (Opus), but 111 is still there and it's 111 that appears to be causing the problems. Could 111 be either the iLBC or iSAC codecs?

As I say this has only happened with the introduction of Jabber 11.9 and latest version of the Expressway Edge-Core solution. An MRA device running Jabber 11.7 works perfectly and will successfully negotiate G729/G711 as we prefer.

Thanks

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Tue, 07 Nov 2017 13:06:09 GMT

Yeah, you can try to disable iSAC and iLBC. I'm always doing it when I install a new CUCM.

Re: Jabber MRA cannot make external calls

cehr_itsupport — Tue, 07 Nov 2017 13:23:39 GMT

Thanks but still having the same problem. Have tried disabling OPUS, iLBC, iSAC and G722 but none of these remove the 111 codec being sent from UCM to the SIP provider.

I saw reference here to 111 potentially being the "SIREN" codec and linked to Microsoft? Have Cisco included this for future compatibility?

Does Expressway have any say in the codecs offered to UCM, or is the offer always initiated by the calling device, i.e. the MRA device?

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Tue, 07 Nov 2017 13:59:16 GMT

I'm trying to look for a way to remove it, not sure yet if it's possible, but from what I could find is that it's something introduced in 11.5. From what I understand, it's a new feature and it should be fully supported in the Expressway and CUCM.

Uneven Level Protection Forward Error Correction (ULPFEC) Support for Audio Stream

Previous releases of Cisco Unified Communications Manager supported forward error correction (FEC) for video stream only. With this release, Cisco Unified Communications Manager also supports X-ULPFECUC for audio stream. With this support, the endpoints and infrastructure applications are more resilient to media packet loss and provide higher audio quality to the users. This feature enhances the audio quality during conferences that traverse the public Internet, business-to-business (B2B), mobile and remote access (MRA) solutions.

Re: Jabber MRA cannot make external calls

cehr_itsupport — Tue, 07 Nov 2017 22:27:47 GMT

Hi Slavik,

That's starting to make sense now - X-ULPFECUC definitely seems to be the cause. Unfortunately I cannot find any reference to how we could disable this, either on CUCM, Expressway or Jabber app..

I think the next step is to raise with our CUCM support who may then log with TAC.

As I mentioned before this only seems to be happening on the later versions of the Jabber app which suggests that it is Jabber that is offering this codec and Expressway and CUCM are simply passing that through to the SIP provider who cannot handle the request cleanly.

This isn't looking good!

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Tue, 07 Nov 2017 22:39:13 GMT

By the way. Are you using CUBE in the middle? If so, aren't you using "voice class codec" towards the ITSP in order to restrict the codec list? Maybe if you'll use it, this codec won't be passed to the service provider, and they won't return it back to you. Although, I have a feeling that you won't see this codec on the external interface negotiation with the service provider, but on the internal interface it might still show. Check it out.

Edit:

Actually I'm looking on a trace from my system, I also see this offered in the SDP when it reaches my Voice Gateway (running CUBE):

a=rtpmap:111 X-ULPFECUC/90000

But when the CUBE sends the INVITE to the ITSP, it won't send it with this codec, and I'm not sure it is because the "voice class codec" configurations or maybe this gateway still doesn't support and familiar with this codec. Anyway, when ITSP is offering me back the negotiated codec, and this 200 OK SIP messages is going in through the internal interface, and I don't see the "111" anymore.

Re: Jabber MRA cannot make external calls

cehr_itsupport — Tue, 07 Nov 2017 22:42:59 GMT

Thanks Slavik,

Unfortunately we're not using CUBE; the CUCM is directly connected to the SIP Provider via SIP Trunks configured in CUCM. Perhaps we could apply a SIP Normalization script or something on the SIP Trunk to the provider?

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Tue, 07 Nov 2017 23:11:55 GMT

Oh, in that's case I think that the only way is using normalization script becasuse CUCM barely let you decide the codecs.

Use this guide to create a simple script that can affect the SDP, there's a remove line function from sdp message described over there:
https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cucm/sip_tn/8_5_1/sip_t_n/4-sdp_api.html

Re: Jabber MRA cannot make external calls

cehr_itsupport — Tue, 07 Nov 2017 23:36:25 GMT

Thank you Slavik! You beat me to it - I’ve just found the same resource. I’ll have a go

in the morning and let you know how it goes!

Thanks again

Lee

Re: Jabber MRA cannot make external calls

cehr_itsupport — Wed, 08 Nov 2017 16:59:17 GMT

So the SIP Normalization Script isn't working! Here's what I've got and it's applied to the SIP Trunk between CUCM and Expressway. My theory being this script should act on all outgoing INVITES and remove the two lines in question. Problem is it appears to be doing nothing. I've tried resetting the trunk and that make no difference.. thoughts anyone?

Thanks
Lee

M={}
trace.enable()
function M.outbound_INVITE(msg)
local sdp = msg:getSdp()
if sdp
then
sdp = sdp:removeLine("a=rtpmap", "x-ulpfecuc")
sdp = sdp:removeLine("a=fmtp", "m=8;max_n=32;FEC_ORDER=FEC_SRTP")
trace.format("SDP-UPDATED")
msg:setSdp(sdp)
end
end
return M

Re: Jabber MRA cannot make external calls

Slavik Bialik — Wed, 08 Nov 2017 17:57:04 GMT

OK, I also tried on my end and the same result, no matter if I do it on the inbound or the outbound the "ulpfecuc" isn't being stripped from SDP.
Sorry mate, I have no idea what else can be done.
In my case I have a CUBE in the middle, and CUBE isn't passing this codec to ITSP. By the way, I would never connect myself directly to the ITSP via SIP, it is not secure.

Re: Jabber MRA cannot make external calls

cehr_itsupport — Wed, 08 Nov 2017 21:05:39 GMT

Thanks Slavik, really appreciate all your help. I’m going to try a few more things (e.g. see if the addline parameter will work) as it is odd that no changes are being made to the SDP content. If that doesn’t work then might have to get a TAC case opened. Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Wed, 08 Nov 2017 21:09:09 GMT

If you solve it, please share the solution. It got me very interested as I work a lot with MRA and Jabber deployments.

Good luck mate!

Re: Jabber MRA cannot make external calls

cehr_itsupport — Wed, 08 Nov 2017 23:51:15 GMT

SUCCESS! Just got this working (need to do a bit more testing), but the script below seems to do what I need and this is the first time in weeks I've managed to call an external number from Jabber MRA. The script below has been applied on the SIP Trunk to our SIP provider, so not directly on the Expressway Trunk as I was doing.

Whilst the script works, I cannot for the life of me find the trace output in the SDL files. Any thoughts? Pretty sure all the right trace settings are on in Serviceability.

Thanks for the pointers - would never have resolved this without your help 🙂

Lee

Re: Jabber MRA cannot make external calls

Slavik Bialik — Thu, 09 Nov 2017 19:42:40 GMT

Nice, glad to hear you've managed to get the script working!

But... I cannot see a big difference between this script and the previous one? What was the mistake?

And besides the script itself, can you now make MRA calls? Is it fixed because you've managed to remove this cursed codec?

Re: Jabber MRA cannot make external calls

cehr_itsupport — Thu, 09 Nov 2017 20:55:03 GMT

Hi Slavik,

I originally applied the script to the trunk between CUCM and Expressway which made no difference So had a change of plan and applied the script to the trunk to the SIP Provider and it worked first time. Time permitting I might investigate further why it didn’t work on Expressway trunk.

I suppose the way it’s working now is the ideal way - we only have problems with calls out via the ITSP so no reason to amend calls just between Expressway and internal devices.

Yes, all MRA devices can now make external calls 🙂

I do have one outstanding issue - the trace.enable() and trace.format() commands in the script either aren’t writing to the SDL file or I’m not looking in the right place. Trace is enabled on the script option on the trunk and I’ve checked Serviceability where detailed logging is enabled on all CM nodes. Any thoughts?

thanks

Lee