We use a Verizon Business SIP Trunk, with 3845 CUBE 12.4.24T. CallManager 7.0(2) and 7962G(SCCP) handset. All calls have dynamic MTP and are G711.
Had the exact same symptoms again twice, only I was dialed out to 704-XXX-XXXX.
I hit the QRT button to record the issue.
The destination can still hear me but I can not hear them. Hang-up call back, all is ok. Then it happens, again an undetermined amount of time later in the call.
well you are using like the latest and greatest IOS, which as many know can be prone to bugs. is there a reason you choose this over 12.4.22 train? We have a good CUBE / SIP Trunk project running with 12.4.22YB.
Intermittent one-way audio mid-call are pretty tricky.
Since this is a sip trunk you really need a packet capture mid-call to see if you're still receiving the packets or not. That's going to be the major factor. I don't know of any bugs in CCM/CUBE that would do this. If you have a firewall you may want to start there though.
i work with Jason, this issue keeps happening but now its more dropped calls than anything. Is packet capture still the best method to find this issue?
Dropped calls can be one of the most misleading things to troubleshoot.
There are two types of dropped calls:
1) Signaling level dropped calls. This is when one of the devices has an error and tears the call down. These will usually have a disconnect value of:
Network out of order
Interworking Error; Unspecified
Resources Unavailable; Unspecified
No Circuit/Channel Available
These are easier to troubleshoot because you can look for the odd disconnect values that shouldn't be happening.
2) The other cause is mid call one-way/no-way audio. Say you have an operator on an IP phone and a customer on the PSTN. The operator is giving instructions, and then then the PSTN user stops hearing the other user. They don't hear anything for 6 seconds, and they hang up. The operator has meanwhile kept on talking, and then they believe the call has dropped.
If you look at debugs for this, it will show a normal call disconnect value of 16.
For this reason, it's very important to know what you're troubleshooting. Very little overlap in possible causes between these two situations.
One thing you can do is capture debug ccsip and see if you can reproduce the issues.
I have done VzB SIP trunk for 800 sites with two clusters. A couple of things I ran into was routing through the MPLS and the marking verizon does.
VzB uses juniper routers (Sorry Cisco! if you are listening) in their cloud and it will only mark at AF 41 or 42 for speech path. During congestion you may find more drops than normal. also are you using a single path out to the MPLS from the central site gatway or multiple paths for redundancy.
Make sure Fast Start is turned on and G711 is the CODEC. Do not get caught in the g722 wideband trap because the newer phone nego g722 first by default.
Nick, I used to work with David before I worked with Chris. Funny how few degrees of seperation there are!
David is your customer running thru CUBE and what version or direct to CallManager and what version CallManager? This was 7.0.2.
Also ran into CSCta21337 (blind transfer bug) at another customer that is Open bug with no fix yet. Consult transfer works and it seems a IOS mtp resource in MRG may be a workaround as well.
I'd suggest the customer debug ccsip to a syslog server like CatTools SyslogD, recommended QRT for tracking when it happens, and try to get some Wireshark capture off back of phone if possible.
They are running a IPIPGW which is now the CUBE. I can not remeber the IOS but I was doing H323 to SIP with UCM 6.x. WireShark will be your best bet.
so we are using VzB SIP trunk to a CUBE through a MPLS cloud. On each exit point to the MPLS their are Verizon Managed Cisco routers which mark and prioritize based on the correct Cisco values (EF, AF31). Now in the service policy I see no drops of RTP or control traffic. Now that being said their are two exit points on each end, both secondary routers seem to not be in use but I'm going to double check. Unless Verizon is rewriting the traffic in the cloud (which I would doubt because these are Verizon routers doing the marking as they leave the site) the problem would have to be routing somewhere. The majority of all our calls are G.729 so we rarely have to worry about g722 out/in SIP calls except to unity and our uccx.
Jason asked me reply to you since I was involved in fixing the issue. The issue I ran into was a bug in the CUBE with codec re-negotation. The fix was to lock down the dial peers in the CUBE so they only did g729. Before I let the dial peer negotiate from g711 or g729 and in mid call the CUBE would ask VBiz for a negotiate and it would just do g711 while Vbiz would stay at g729. I'm sure it work if you just left the dial peer at G711 as well.
We ran into this same issue with Verizon as the SIP trunk provider, CUCM 6.1, and a CUBE router running 12.4(24)T3 (IPIPGW). If Verizon detected what it thought was a fax tone, they would send a codec renegotiation to our CUBE which our CUBE accepted. CUCM 6.1 doesn't accept mid-call codec changes and this resulted in one-way audio. We found that the default Blackberry ring tone, or a fax machine in the background would consistently cause Verizon to initiate a codec renegotiation.
The first solution we implemented was to lock down the dial-peer to G.729. This worked but had the undesirable side-effect of not allowing informational recordings sent from Verizon (invalid number, etc) to be played. These apparently were sent G.711 only.
The second solution came from Verizon and it fixed our problem for good. We actually had to downgrade the code on our 3845 CUBE to 12.4(20)T6 and this fixed the issue. This version of code will reject the mid-call codec change thereby preventing the one-way audio. According to Verizon, we could run any version of code as long as it was 12.4(20)Tx.