ā09-04-2024 02:17 AM
Hello everyone,
I am trying to troubleshoot VOIP related problems when voice during calls is intermittent
The phones are connected to separate switches and there is Cisco 9300 in between those switches.
I have read through this article:
And increased qos queue-softmax-multiplier 1200
Now drops are not increasing anymore in this table, but TH0/TH1/TH2 are still increasing
AQM Global counters
GlobalHardLimit: 5176 | GlobalHardBufCount: 0
GlobalSoftLimit: 14672 | GlobalSoftBufCount: 0
----------------------------------------------------------------------------------------------
High Watermark Soft Buffers: Port Monitor Disabled
----------------------------------------------------------------------------------------------
Asic:0 Core:1 DATA Port:15 Hardware Enqueue Counters
-------------------------------------------------------------------------
Q Buffers Enqueue-TH0 Enqueue-TH1 Enqueue-TH2
(Count) (Bytes) (Bytes) (Bytes)
-- ------- -------------------- -------------------- --------------------
0 0 15626884274 32907700571 2516556011
1 0 0 0 7226027781
2 0 0 0 0
3 0 0 0 0
4 0 0 0 0
5 0 0 0 0
6 0 0 0 0
7 0 0 0 0
Asic:0 Core:1 DATA Port:15 Hardware Drop Counters
-----------------------------------------------------------------------------------------------------------
Q Drop-TH0 Drop-TH1 Drop-TH2 SBufDrop QebDrop
(Bytes) (Bytes) (Bytes) (Bytes) (Bytes)
-- -------------------- -------------------- -------------------- -------------------- --------------------
0 14586413831 614569504 0 0 0
1 0 0 15293510335 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
6 0 0 0 0 0
7 0 0 0 0 0
Can this queuing affect voice traffic?
ā09-11-2024 03:13 AM
To your first question, the later Cisco switches usually have a default active QoS config, where different CoS or ToS markings are equally divided between egress queues, each with equal resources, but possibly with different drop thresholds. Whether your traffic hits a drop threshold would depend on how your traffic is tagged, how much traffic you have, and the device's defaults.
I presume if you check similar ports on the same switch, or like model switches, you'll also see different port utilization stats too. ; )
To your second question, yes, you might increase a buffer allocation, but it's also a QoS command. Consider you would change it to reduce drops, i.e. it impacts the quality of service.
Might it help? Possibly.
Understand, often QoS is a zero sum game. One pie slice size increase is less available for all the possible pie slices.
Ideally, though, you take advantage of excess resources, those not actually needed. E.g. you slice a pie into 8 slices, but 1 of your expected 8 guests is a no show. Knowing that, you can dole out more pie to one or more of your guests.
Possibly you have a guest of honor who gets their choice for their pie slice size and everyone else gets to divide up the remainder.
When we adjust resources, we often do so at the expense of some traffic to the benefit of other traffic.
ā09-11-2024 03:31 AM
To your first question, the later Cisco switches usually have a default active QoS config, where different CoS or ToS markings are equally divided between egress queues, each with equal resources, but possibly with different drop thresholds. Whether your traffic hits a drop threshold would depend on how your traffic is tagged, how much traffic you have, and the device's defaults.
I checked all ports on 2 switches (different models) and on all of them only Queue 3 Weight 2 has drops. Does it have anything to do with CS3 Qos marking that is configured on IP Phones by default?
Is it possible to check the drop threshold on specific ports and increase it somehow?
ā09-11-2024 03:43 AM
"Does it have anything to do with CS3 Qos marking that is configured on IP Phones by default?"
It may, although VoIP traffic tends to usually use relatively little bandwidth. Other traffic might be marked CS3 and other traffic might have a different marking but directed to the same place. There should be a way to query the device what traffic get directed there.
"Is it possible to check the drop threshold on specific ports and increase it somehow?"
Usually yes, but I suggest a useful policy be implemented on all ports unless your exceptional policy is only applied to exceptionally few ports.
ā09-04-2024 04:21 AM
Do you use multi Gige for ingress and one gige for egress?
You need to add PO I think for this issue
MHM
ā09-04-2024 06:37 AM
Hello,
Topology is like this:
Cisco Phone 1 - Switch A - 9300 Switch - Switch 8 - Cisco Phone 2
All uplinks are Gig. Cisco 9300 has other switches connected to it as well
What is PO?
ā09-04-2024 06:47 AM
PO meaning port channel'
You need to faster transfer traffic before it full Queue.
MHM
ā09-04-2024 06:57 AM
Well, according to Solarwinds, these Gig interfaces are hardly getting 100mb/s of traffic in Gig uplink, do you think PO will help?
ā09-04-2024 06:59 AM
100 mbps maybe from traffic drop and drop in queue.
So sure try use PO abd check status.
Hope this help you in your issue
MHM
ā09-04-2024 07:02 AM
Sorry, did not understand you comment about 100mbs
ā09-04-2024 09:27 AM
For 100 mbps first check if interface work as 1 Gbps or 100 mbps
Do show interface
Show interface status
MHM
ā09-04-2024 09:24 AM
@FuadG wrote:
Well, according to Solarwinds, these Gig interfaces are hardly getting 100mb/s of traffic in Gig uplink, do you think PO will help?
Unfortunately, that's a very, very (very) common misinterpretation of interface stats.
Laugh - so ponder on why a gig interface hardly ever even showing 10% utilization drop packets? If that doesn't seem to make sense - exactly! Are those stats wrong? Probably not. Again, often they are misinterpreted (or misunderstood).
ā09-04-2024 09:26 AM - edited ā09-04-2024 09:28 AM
Hello @FuadG ,
you should implement QoS on all switches in the path as suggested by @Joseph W. Doherty .
>> Cisco Phone 1 - Switch A - 9300 Switch - Switch 8 - Cisco Phone 2
You need to find out what kind of switches are Switch A, Switch B and their OS system version.
Ideally , you should be able to call also without QoS applied if the network load is low but you may have production data traffic and without QoS you cannot avoid issues like big data packets being served = put on the wire before the small delay sensitive VOIP packets. This can create delay variance = jitter that impairs voice quality.
Hope to help
Giuseppe
ā09-04-2024 04:04 PM - edited ā09-04-2024 04:07 PM
@Giuseppe Larosa wrote:
you should implement QoS on all switches in the path as suggested by @Joseph W. Doherty .
I don't believe I actually explicitly suggested that, although I may have inadvertently implied that. ; )
As to whether QoS is actually needs to be configured depends on whether it actually is also needed and can deliver the results needed. I.e. often it's not actually needed to be configured end-to-end; still, though, it's very advisable to do so.
That said, the "book" suggests that it should be, and the "book" recommendation is good because only with QoS configured end-to-end can you provide end-to-end service guarantees. Also, networks, and their usage, can change. If you don't have end-to-end QoS, what works fine today, might not work well tomorrow.
So am I now, recommending end-to-end QoS? Well . . .
QoS has its pros and cons. Cons include, QoS has its own "care and feeding" requirements. Further, depending on your needs, a one size fits all QoS model might not work for you.
So, what I actually often recommend with QoS, is a case by case analysis of its RoI. If some cases, usually like WANs, QoS often has a great RoI. In some other cases, usually like LANs, other approaches might have a better RoI.
Also, understand, you can mix and match. For example, a complex QoS model for WAN interfaces, and a simple QoS model for LAN interfaces.
@Giuseppe Larosa wrote:
Ideally , you should be able to call also without QoS applied if the network load is low but you may have production data traffic and without QoS you cannot avoid issues like big data packets being served = put on the wire before the small delay sensitive VOIP packets. This can create delay variance = jitter that impairs voice quality.
There's much truth in the forgoing, but "low" traffic levels don't guarantee VoIP success and conversely "high" traffic level don't guarantee VoIP problems. Also "big" data packets aren't, in themselves, even jumbos, necessarily (or in modern networks often) adverse to "small" VoIP packets.
Although often traffic utilization level and/or packets sizes seem to go hand-in-hand with VoIP issues, there are other variables that matter that are often not considered.
Simple example:
On a 64 Kbps link, it takes 187.5 ms to serialize a single 1518 byte Ethernet frame. Consider, the usual maximum bound for VoIP latency is 250 ms, and 150 ms or under is recommended, 187.5 ms isn't particularly good. This is an example of the "big" packet issue, Giuseppe mentions.
For low bandwidth interfaces, Cisco had/has a Link Fragmentation and Interleaving (LFI) feature.
So if 1500 is borderline bad, a 9000 byte jumbo must be out of bounds, correct?
Yes it would be on a 64 Kbps link, but what if used on a gig link?
Then its serialization delay is 72 Āµs. As VoIP latency tolerance hasn't changed, 72 Āµs isn't much of a concern.
What impacts VoIP quality is packet loss, overall latency and (as mentioned by Giuseppe) jitter, but problem causes are a bit more than just packet sizes (which is a factor) or average load utilizations, the latter usually measured in lots of seconds to minutes. With VoIP, we need to understand, and allow for, "things" happening in milliseconds.
In other replies, I've remarked that Cisco's QoS information has greatly improved, over the years.
In Cisco's Technote, Troubleshoot Output Drops on Catalyst 9000 Switches, refer to the last section which shows how drop issues cannot be seen even at 1 second intervals, but can be seen at 1 ms intervals.
ā09-05-2024 12:39 AM
Thank you for detailed explanation
When I use this formula:
Percentage of output drops = Total output drops / Total output bytes X 100 I get an extremely low value.
Max Jitter on the web interface of the phone is between 1-10
Drops count are no longer increasing on 9300 switch in the table from my initial post, but Enqueue numbers are increasing constantly - is it something abnormal or it is as it should be?
ā09-05-2024 12:51 AM
If one side use 100 M and other use 1 G
Then sure there is drop
Some interface not support autonego you need to hardcoded speed.
MHM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide