Questions over Jabber provisioned users using ICE/TURN

[Chris Swinney]

Hi All,

TMS - 14.2.2

Jabber Video - 4.6

TMSPE - v1

VCS - x7.1

Basic Network Idea:

Now, I have set up ICE on the Jabber provisioning template and point the client to the VCS-E as a TURN server. Internal "Default Mediatype Candidate" provisioning parameter is set to "host", whilst the "Public Mediatype Candidate" is set to "relay". ICE is not actually needed for internal clients as they all are assigned publicly routable IP address. However, we need this for certain remote users, and there seem to be no way to apply this setting to JUST the remote workers - it is either ON or OFF.

Now here's the rub. We have been having odd issue when the local Jabber clients have been trying to connect to other local clients (be they other Jabber user or H.323 endpoints). We where getting some really wird things such as:

• connected video and audio on both sides, then almost immediatly disconnected on one side but the other can still see and hear;
• no video/audio in one direction (no firewalls in between)
• continuous ringing even have a call has been cancelled.
• long delays in connecting clients
• and other oddities

If I turn ICE off, the problem goes away for at the local site, but of course, certian remote users can no longer communicate..

But here the interesting bit. If I turn ICE back on (and ensure that all the provisioning templates have replicated), log out of Jabber, then back in and then try to ring a client within around 10 seconds of logging in (either Jabber or H.323), it will take the Jabber client maybe 10 seconds to connect. If I hang up and redial, it takes around 90 seconds to connect! No matter how long I wait, this problem seems to be around untill I log out.

If I run a Wireshark trace during these connections, I see multiple STUN packets being sent from the Jabber PC to the VCS-C referring to either a "Binding Indication or "Allocate Request". I have also played with the "Default Mediatype Candidate" set to the other setting, which will point the client to the VCS-E but essentially I see the same behaviours.

Now, if I log out of Jabber, then back in, but this time WAIT more than around 10 seconds, when I dial, I am connected immediately.

With ICE off, it make no odds how quickly I dial - I am connected immediately.

So - what's going on? I would love it someone could replicate this issue (unless its a known bug).

Regards,

Chris

[Paulo Souza]

Hi Cris,

I am trying to understand why you need to use TURN/ICE features. ICE is good to avoid external users to route media through your VCSe (as happens in a common traversal call), so that external users will be able to route media directly between them without using VCSe.

But ICE is not a required feature to make an external Jabber (behind NAT/Firewall) to sucessfull connect to VCSe via Internet and make and receive calls. ICE is just to optmizing media routing.

For example, I have my jabber 4.6, behind NAT/Firewall at home, registered to VCSe via internet, the topology is exactly jus like your environment. There is no ICE/TURN enable. I am able to make and receive calls normally without restriction.

Have you tried to setup your external users without using ICE?

In my opinion, Cisco documentation related to ICE/TURN is very poor. For example, this is the only thing you can found related to how to configure ICE for Jabber:

http://www.cisco.com/en/US/docs/telepresence/endpoint/Jabber_Video/4_6/CJAB_BK_C89F6C9E_00_cisco-jabber-video-for-telepresence_chapter_0111.html#CJAB_RF_MF2FBD6E_00

See the detail level of the doc, it is really poor. I don't think it is enough to make you understand, configure and troubleshooting ICE, that is why I prefer to go ahead without ICE.

Regards

Paulo Souza

Please rate replies and mark question as "answered" if applicable.

[Chris Swinney]

Hi Paulo,

I always wonder when I draw a diagram if it will be sufficient to explain a problem. You always assume you have covered all aspects in order to put the point across but not leave out too much detail. I think that I might have failed in this basic scenario. I have also read the document you link to and agree that it doesn't really explain what we might need to do.

So, we have a little more going on here than diagramed. There are external forces involved here - other organisations that both the external and internal clients would like to communicate with.

Indeed, I can connect from home via NAT successfully without ICE. However, I have seen other instances where this has not been possible. We had a remote worker connect via 3G recently in Africa where without TURN with connection did not function. Closer to home we have had other user connect in from corporate guest LANs where a connection has been an issue.

Further, both the remote and local client will need to contact user in outside organisations or facilities where ICE may play and important part.

However, i think were missing an important point here. That is, why does the client fail after a short login period. If this is a limitation of ICE/TURN registration, then surely the client should be limited from making a call until this process has been completed.

Cheers

Chris

[Paulo Souza]

Hi Cris,

Which clients are your external users using? Are all they using Jabber or are there generic SIP clients involved?

Let me explain you why external jabber clients can sucessfull connect to VCSe without ICE. In general, the inteligence for this works is not on any mechanism used by the client, the inteligence is on VCS-e traversall server features. There are two behaviors of VCSe that makes the connection from a client behind NAT without ICE to work:

1. When you have SIP client (not only Jabber) behind a firewall/NAT registered to VCSe, VCS-e will use traversall call everytime this client receives or makes calls. That means, VCS-e will insert its own IP address in the media routing IP address inside SIP/SDP header, so that media will always be routed through VCSe.
2. When VCSe detects that the registered SIP client is behind a NAT, when this client makes or receive a call, VCSe will inteligently replace the source ports used by RTP according to the "NATed" source ports regardless of the ports negotiated in SDP header. So that VCSe will be able to send RTP to the endpoint even when it is behind NAT.

If you set diagnostic "network.sip" and "network.mediarouting" parameters to debug, you will be able to take the logs and understand better this behavior. And this is applicable for Jabber client and another generic SIP clients too, because, as I stated before, the inteligence is on VCS-e and not on the client itself.

Then, as far as I know, even without ICE/TURN the SIP connection should work fine, unless you have following situations:

• The client has ICE/TURN/STUN features enable but VCSe has not
• The client is behind a NAT/Firewall with some kind of inspection/ALG enable. It may cause conflict because of the behavior of VCSe described above
• The client uses TCP media relay. This is not supported by VCS, only UDP is supported
• The client is behind firewall/NAT which has some kind of proxy or content filter that may block the RTP ports used by the client

With regards ICE/TURN time out, this is really a issue. The client will try to negotiate the best path for media routing before start the call, this will take a while. But I guess you can decrease the timeout by changing the parameter "Turn fallback time out" (or something like this) in provisioning template configuration, I guess I saw this configuration in TMS, not sure.

An important point is, if you are using ICE, If the ICE negotiation fails or if the remote does not understand ICE, the endpoint will make a common call without any media routing opitimization, so that the "Default Mediatype Candidate" fields are applied. I would suggest to configure as following:

Default Mediatype Candidate: Host

Public Default Mediatype Candidate: Relay

Regards       

Paulo Souza

Please rate replies and mark question as "answered" if applicable.

[Paulo Souza]

Additionally, I think you can create different provisioning directories in TMS to put your external users that demand ICE/TURN features. So you will be able to enable ICE only to specific user groups.

Unfortunately, as you stated, in the same template we cannot enable ICE for external clients and totally disable it for internal. But I think that TURN parameteres is only applied to external clients and not to internal. Well, anyhow, you can use separated provisioning directories and set ICE on only to desirable users. That could be something to investigate and test.

Regards

Paulo Souza

Please rate replies and mark question as "answered" if applicable.

[Chris Swinney]

Hi Paulo

Paulo Souza wrote:
Additionally, I think you can create different provisioning directories in TMS to put your external users that demand ICE/TURN features. So you will be able to enable ICE only to specific user groups.
Unfortunately, as you stated, in the same template we cannot enable ICE for external clients and totally disable it for internal. But I think that TURN parameteres is only applied to external clients and not to internal. Well, anyhow, you can use separated provisioning directories and set ICE on only to desirable users. That could be something to investigate and test.

Yep - was thinking the same thing (specific user groups) - although of course the remote workers in 99% of the cases will also need to be local users - its a real pain to figure.

[Chris Swinney]

Hey Paulo,

Once again thanks for the replies. It is a great resource to have willing participants to enable thoughtful discussion in order to  help find resolutions to what can be, complex queries.

Paulo Souza wrote:
Hi Cris,
Which clients are your external users using? Are all they using Jabber or are there generic SIP clients involved?

In our case, we need to be able to support both Jabber and standards based SIP clients - we are involved mainly with Education and so the Linux desktop environment can be common, but there are other reasons. We did operate without ICE for quite some time, however, we have seen issues with Jabber when connecting through specific environments which we DON'T have control over. However, I think you elude to this below.

Paulo Souza wrote:
If you set diagnostic "network.sip" and "network.mediarouting" parameters to debug, you will be able to take the logs and understand better this behavior. And this is applicable for Jabber client and another generic SIP clients too, because, as I stated before, the inteligence is on VCS-e and not on the client itself. 

Will Do.

Then, as far as I know, even without ICE/TURN the SIP connection should work fine, unless you have following situations:
The client has ICE/TURN/STUN features enable but VCSe has not
The client is behind a NAT/Firewall with some kind of inspection/ALG enable. It may cause conflict because of the behavior of VCSe described above
The client uses TCP media relay. This is not supported by VCS, only UDP is supported
The client is behind firewall/NAT which has some kind of proxy or content filter that may block the RTP ports used by the client

Yes - I think this would be an issue. Often we don't have control on the environment the remote users walk into. Certainly enabling ICE/TURN in these case resolved the issue.

With regards ICE/TURN time out, this is really a issue. The client will try to negotiate the best path for media routing before start the call, this will take a while. But I guess you can decrease the timeout by changing the parameter "Turn fallback time out" (or something like this) in provisioning template configuration, I guess I saw this configuration in TMS, not sure.

Ah ha, at least this is not me going mad. I would be great it the client could bar call from being made until the negotiation is complete. I have set the TURN time out to 3 seconds, but it still take 10+ to sort itself out. More anoyinglyly, if you call too soon, the client then become corrupt until the user logs out and back in again!!! Sounds like a bug to me

An important point is, if you are using ICE, If the ICE negotiation fails or if the remote does not understand ICE, the endpoint will make a common call without any media routing opitimization, so that the "Default Mediatype Candidate" fields are applied. I would suggest to configure as following:
Default Mediatype Candidate: Host
Public Default Mediatype Candidate: Relay

Yep - had this (but also tested other settings just in case), but still we have issues on local calls. With ICE turned on, when a local Jabber client calls another local Jabber client, both video and audio connect in both directions, but then the caller drops audio and video - the called party so continues to recieve both audio and video (I did reboot both the VCS-C and VCS-E last night but have not tested this since). As soon as I turn ICE off, the problem disapears. I will see if I can replicate the issue and Wireshark the trace from the caller - but I bet I will see an odd STUN request to the VCS-C (yes Control). I don't see why this should be case though.

For this reason, I have to turn ICE off as we obvioulsy have more internal calls than remote that actually require ICE.

[Paulo Souza]

Hi Cris,

I really think that Jabber clients should not send any STUN/TURN messages to VCS Control, because certain parameters in the provisioning template settings are only applicable to endpoints externally connected, just like other paremeters, such as Public Server, Public bandwidth and so on, including turn parameters, they should be applicable only to jabber connected from external.

Are you sure you have received STUN/TURN messages in VCS Control from local endpoints??

Btw, this kind of issue is very interesting for me. I have an environment with VCSc, VCSe and TMS to make tests. I will try to replicate your configuration here to see if a get the same problem. I will test ICE functionality by using Jabber and X-Lite clients. Of course I wont be able reproduce your whole environment, mainly the part related to external users behind "strange" firewall/NAT devices, but at least we can have a try to check what happens.

Give me some time. I will post a feedback soon.

Regards

Paulo Souza

Please rate replies and mark question as "answered" if applicable.