Voice traffic "NO SOUND" Cisco 3850 stack

[r.lundrigan]

Hi, I have a Cisco 3850 8-switch stack with a Mitel phone environment(mixture of 5300 phones). Users will be on the phone and then suddenly all sound cuts out on both the receiving and sending end for about 8seconds and then returns. One thing I have noticed is that ping requests to the stack will show latency for about 5-10ms at a time and then drop down to 1-3ms. I have not found any correlation between the ping latency and the sound issue yet.

-The stack is currently linked up via 10gb fibre to a Nexus 6000.

-Phone and data traffic is converged

[Richard Bradfield]

Hi,

has QOS been setup on the 3850 for the phones? please share the  3850 configuration

it could be a device that is flooding the network, do you have any form of network monitoring?

[Dennis Mink]

In general, don't get to fixated on it being a Voice/Mittel related issue. 95% of the time when I get voice issues reported to me, its is symptomatic.

Does this issue happens between phones on that same switch?  in which case QoS does not seem to be the likely cause.  

start graphing cpu, errors on interfaces, and use syslog, so that if you have a timestamp of the issue occuring you can match it with another measured event.

I am sorry to be so generic, but that is all I can advise atm,.

HTH

[devils_advocate]

Firstly run a packet capture on one of the ports going to the phones.
Make sure you replicate the issue whilst the capture is taking place.
Check the capture to see if the RTP traffic is still being sent/received during the 8 seconds. 

Switches do not prioritise ICMP responses which are destined for one of their interfaces so pinging them is not a reliable method of checking for latency. The difference between 1-3ms and 5-10ms when pinging a switch is nothing to worry about, as mentioned, the switch does not prioritise its responses to these requests. 

[r.lundrigan]

Thanks for the reply

Another thing I started to notice is that some of the pings would jump up to 400-500ms for about 6 seconds and then drop down. I ran a show proc cpu sort and noticed that the IOSD process was running high. I ran a show proc cpu detail iosd sort | ex 0.0, and that show the NGWC L2M process running at 96%. I Googled this process, and apparently it has something do to with the software on the switch trying to process a flood of requests for IPv6. What was recommended was to enable IPv6 snooping and that will force the switch hardware to process the requests instead of the switch software.

command to run: switch#(config) ipv6 mld snooping

I am not 100% positive that this was the cause of the voice issue, but so far i haven't heard any complaints. I will monitor and keep you guys updated.

[devils_advocate]

If that does fix the issue then great.

If not, don't focus on the ping times to the switch.
Run a packet capture to a phone whilst the issue is occurring.

When it comes to troubleshooting, don't focus on something that may or may not be an issue. Get a packet capture to see what is actually happening to the RTP packets.

You don't want to be lead down a blind path of ping times when we know that they are not reliable when the destination is the switch itself. 

[r.lundrigan]

Agreed. I appreciate the advice.