SIP BYE packets not received from remote party after 7 minutes

[Drew T]

Unusual one that's just recently surfaced. 

We are using a certain SIP provider for outbound calls, which up until recently appears to have been working fine.

We have two gateways, which sit directly behind our border routers to the internet. They're 3825's with CUBE on them, both running a variant of 15.1. They are used exclusively for this one provider. 

Both have simple unauthenticated SIP Trunks, with ACL restrictions. 

We can initiate a call, and it works fine. If the called party hangs up first, under 7 minutes, we receive the BYE from the remote party. 

When the call exceeds 7 minutes (approximately, not a hard number, just somewhere over 7-9 minutes), we do not see the BYE back. The SIP trunk provider has given us pcap's of the call, and they are sending the BYE repeatedly when the called party terminates. At 9 minutes plus it's happening 100% of the time.

There is not NAT at play, and the remote SIP provider is connecting to the binds on each gateay (via a loopback). These two gateways have a near identical configuration to every other gateway in our network on other providers, all of which are working fine. The provider with the problem says it's not their problem, and the fact that it seems to be only longer calls with the issue would suggest it could be a Cisco problem, but i'm not so sure. 

Anyone seen anything like this? We do see the following in the logs, but it's not a very helpful error (and happens frequently in other places)

Aug 13 22:03:31: //-1/xxxxxxxxxxxx/SIP/Msg/ccsipDisplayMsg:
Received: 



Aug 13 22:03:31: //-1/xxxxxxxxxxxx/SIP/Error/HandleUdpIPv4SocketReads: SIP Message incomplete, trashed
Aug 13 22:03:31: //-1/xxxxxxxxxxxx/SIP/Msg/ccsipDisplayMsg:
Received: 



Aug 13 22:03:31: //-1/xxxxxxxxxxxx/SIP/Error/HandleUdpIPv4SocketReads: SIP Message incomplete, trashedterm no mon

I don't think these messages are related, but included them as it's all that the debugs are showing. 

Any help/suggestions appreciated. 

[Dennis Mink]

do you see the BYE come into your border router, at all?

I mean you probably did a debug ccsip on your cube, and saw no BYE.

so look for the problem upstream to your provider, ie your border router(s)

[Drew T]

The border routers are not likely to be the culprit as shorter length calls are fine. There is no NAT and it's just passing a routered packet through to a directly connected subnet. 

I'm not able to do a full capture on the border routers at present, but the fact it's working for every provider but this one, suggests it's not likely to be that.

First thought is they were not reaching us, but now we've verified it's only on calls 7-9 minutes plus, and only when the called party terminates, it's making it a little more strange. 

[Dan Lukes]

A statefull firewall on the path may have the same effect as NAT.

[Dennis Mink]

move up the stream and capture sip to rule components out. 

[ashok_boin]

It may be possible that the CUBE ignores SIP Bye messages if considered as malformed.

As suggested in earlier post, the best thing to do is to do SPAN monitoring through Wireshark for the traffic b/w Border and CUBE routers. If you see SIP bye packet is getting forwarded to CUBE, then they are getting dropped at CUBE itself.

https://stackoverflow.com/questions/1632499/problem-with-sip-bye-message

[Drew T]

Thanks Ashok,

I don't have the ability to span anything at this site (no boxes to use for tcpdump/wireshark). 

The BYE packets get through if the calls are under 7 or so minutes. The SIP provider is in AWS (if that makes any difference). 

There are also no stateful firewalls anywhere. We have ACL's on the border router ingress (to our providers) and permit gt 1023 for all IP's in the providers ranges to the gateways (bi-directional). 

I'll continue to search and see if I can find anything with what I have. 

[Ishan Zutshi]

there are just 2 possibilities. Either the BYE packet didnt reach the cube router or it reached but it was discarded. either way you need a packet capture and ccsip debugs to find out the root cause.
there could be a difference in the way the SIP BYE message is sent from the provider side in case of <9 minute calls versus >9 minute calls.
if you can get the packet capture from the provider side comparing the 2 BYE messages, maybe you could get a clue (compare VIA headers, URIs, packet size etc)

[Dalton-Covene]

Drew T, 

Can you please check the TCP/UDP transport settings for the outbound dial-peer to the SIP Provider? 

dial-peer voice 101 voip

session transport tcp 

Regards,

Dalton Woolsey