CUBE Rejects SIP invite with ICMP Destination unreachable (Port unreachable)

kdhartstrom · ‎02-26-2020

So this morning I arrive and discover that incoming calls are failing. Architecture is

[CUCM] -<sip>-[CUBE]--<sip>--[onpremITSProtuer w/NAT]----[ITSP] all via sip. So I log into my cube router, and I can successfully ping my ITSP, and "show sip-ua register status" shows that my trunk is registered.

CUBE#show sip-ua register status 
Line                             peer       expires(sec) reg survival P-Associ-URI
================================ ========== ============ === ======== ============
sipuser                          -1         736          yes normal

I do a quick debug and see zero log messages showing traffic coming from the ITSP when someone calls in. However, when I did a packet capture, I could see SIP invites coming from the ITSP, and then my cube router rejecting them with ICMP Destination unreachable (Port unreachable)

No.	Time	Source	Destination	Protocol	Length	Info
7	7.803019	xxx.xxx.xxx.x	yyy.yyy.yyy.yy	SIP/SDP	1437	Request: INVITE sip:sipuser@xxx.xxx.xxx.x:63055;transport=UDP | 
8	7.803019	yyy.yyy.yyy.yy	xxx.xxx.xxx.x	ICMP	74	Destination unreachable (Port unreachable)

Now my ITSP provides the [onpremITSProtuer w/NAT] device which is an on premise router performing NAT translations and an automatic fail-over between two circuits where it's programmed to clear the NAT translation if a fail-over happens. It does not have any SIP ALG functionality enabled. I suspect this may have something to do with the ICMP port unreachable messages. The UDP packets had a source port of 5060 and a destination port of 63055,

I called my ITSP and we rest the [onpremITSProtuer w/NAT] device, and once it came back up incoming calls succeeded.

I again did a packet capture, and now see that instead of rejecting the SIP invite with UDP source 5060, destination 63721, the cube router replies with a "SIP 100 Trying" as expected and the call goes on to succeed.

So I have two questions:

1. Why does my CUBE router show registered when incoming calls are failing? Can I have it do a sip options ping to make sure that the CUBE and ITSP are talking?

2. Why is my CUBE rejecting SIP invites? can I configure it to listen to more UDP ports?

Thanks for any help.

Kirk

Mike_Brezicky · ‎02-26-2020

Can you provide the SIP debug logs for a failed and successful call?
It does sound like a NAT issue, but if the ITSP is providing the device which handles the NAT< that may be difficult to get any insight in with out the provider.

kdhartstrom · ‎02-26-2020

So I have the debugs, but the one for failed calls shows nothing from the ITSP, and the one for a working call shows normal SIP signalling as expected. What specifically should I be looking for?

TONY SMITH · ‎02-27-2020

@kdhartstrom wrote:
1. Why does my CUBE router show registered when incoming calls are failing? Can I have it do a sip options ping to make sure that the CUBE and ITSP are talking?

Registration is initiated by the CUBE, so that shows that it is able to send to the ITSP, and that the ITSP replies. Options ping is the same, initiated from the CUBE and replied by the ITSP.

2. Why is my CUBE rejecting SIP invites? can I configure it to listen to more UDP ports?

As far as I'm aware the CUBE just listens for SIP on one port, by default 5060, but can be configured to a different value.

kdhartstrom · ‎02-27-2020

So the issue appears to be related to UDP Nat/Pat and open IP Sockets. If I run the command 'show ip sockets' I see an record that lists a local port in the 60,000 range when calls are working, which corresponds to what my carrier is seeing on their router's NAT translation table, as well as the SIP registration at the ITSP.

x.x.x.x is my LAN IP, y.y.y.y is my ITSP, z.z.z.z is an internet public IP

CUBE#show ip sockets
Proto        Remote      Port      Local       Port  In Out  Stat TTY OutputIF
<truncated>
 17     0.0.0.0             0 x.x.x.x    5060   0   0    41   0
 17     y.y.y.y          5060 x.x.x.x   61560   0   0   205   0

carrier router#sho ip nat translations
Pro Inside global         Inside local          Outside local         Outside global
<truncated>
udp z.z.z.z:61560         x.x.x.x:61560          y.y.y.y:5060          y.y.y.y:5060

I'm seeing the ICMP Destination unreachable (Port unreachable) rejects when I do a packet capture on my interface facing the ITSP, but nothing of use in the debug logs. What would cause my ip socket to be closed, but the sip registration on by my side and the carrier side to remain open, and how could I prevent that?

kdhartstrom · ‎02-27-2020

I should add that the issue happened again yesterday afternoon, and as soon as the "show sip-ua register status" expires timer hit 0, and subsequently re-registered calls started flowing again.

CUBE#show sip-ua register status 
Line                             peer       expires(sec) reg survival P-Associ-URI
================================ ========== ============ === ======== ============
sipuser                          -1         10           yes normal

TONY SMITH · ‎02-28-2020

@kdhartstrom wrote:
So the issue appears to be related to UDP Nat/Pat and open IP Sockets.

Yes. Whenever I've put a CUBE inside a NAT firewall we've had a static NAT entry mapping an external IP address directly to the CUBE's internal IP, together with an ACL or policy that permits only the ITSP's gateway to use that translation. You don't want your CUBE open to the Internet as a whole.

With this configuration the service provider can send to our external address, on port 5060, whenever it feels like it, and the CUBE will receive on 5060.

Regarding registration, and outbound option pings for that matter, I think what's happening in your case is that the initial message from the CUBE is being permitted out with normal stateful "inspect" type firewall rules the same as normal user Internet access. Accordingly the firewall is then happy to receive what appear to be reply packets in response. So assuming your CUBE sources on port 5060 then that behaviour will allow it to receive on 5060. This will only apply until the firewall NAT times out.

So in fact thinking again maybe Options Ping might help, by keeping that port open.

What would cause my ip socket to be closed, but the sip registration on by my side and the carrier side to remain open, and how could I prevent that?

As above I suspect the IP socket is being closed by NAT timeout on the firewall. Registration will have its own separate timeout/