Choppy voice with QoS enabled

raymondtun · ‎12-19-2012

Hi All,

I have CUCM cluster in Singapore and Gateways/IP phones across asia pac region using MPLS network.

I am having this weird issue with QoS at the newly setup office at australia.

It has 10mbps WAN link to our corporate MPLS network with cisco 2921 router. And not much data traffic since no one has moved in yet.

When I enable QoS on the WAN link, the voice becomes choppy and ping dropping to voice VLAN.

When I remove the ACL "no access-list 120", the voice is loud and clear.

I know it sounds funny.

But I have this same setup at other sites working perfectly and this issue is driving me crazy for a week now. Here is the config on the router...

access-list 120 remark QOS - VOICE - VOIP ACL

access-list 120 permit ip 10.146.0.0 0.0.255.255 any

!

class-map match-any 120-VOICE

match access-group 120

!

policy-map QOS

class 120-VOICE

priority 2560

set ip dscp ef

!

interface GigabitEthernet0/1

description MPLS link 10Mbps

bandwidth 10240

ip address X.X.X.X 255.255.255.252

ip virtual-reassembly in

load-interval 30

duplex full

speed 100

service-policy output QOS

!

#show policy-map interface gigabitEthernet 0/1 output class 120-VOICE

GigabitEthernet0/1

Service-policy output: QOS

queue stats for all priority classes:

queue limit 64 packets

(queue depth/total drops/no-buffer drops) 0/0/0

(pkts output/bytes output) 80054/9612252

Class-map: 120-VOICE (match-any)

80054 packets, 9612252 bytes

30 second offered rate 80520 bps, drop rate 0 bps

Match: access-group 120

356 packets, 30160 bytes

30 second rate 0 bps

Priority: 2560 kbps, burst bytes 64000, b/w exceed drops: 0

QoS Set

dscp ef

Packets marked 80054

#

During the call, I check the Call Statistics Screen of my Singapore IP Phone.

Rcvr Lost Packets counter is always increasing.

MOS is 2 (using G711)

Max Conceal Ratio is 0.7 to 1

Conceal Secs and Severely Conceal Secs are increasing every second.

I hope someone can give me some pointers for troubleshooting.

Where else should I check or look at?

brmeade · ‎12-19-2012

Tun Aung San,

Most likely this issue here is that you are not throttling the transmit queue when you have a 10Mbps MPLS connection connected to a Gigabit interface. You will need to use a parent policer to police your traffic down to your CIR (10 Mbps) so that your own queueing mechanisms will be effective.

Try this:

policy-map parent-policy

class class-default

police 10240000

exceed-action drop

violate-action drop

exit

policy-map QOS

int gig0/1

service-policy output parent-policy

Thanks,

Brian Meade

raymondtun · ‎12-19-2012

Brian Meade,

Thanks. I tried your advice.

But my actual configuration is like this... I have more than one class map. I couldn't use policing. It gave me error.

"Cannot attach queuing-based child policy to a non-queuing based class"

I used shaping instead.

Sad to say, it still doesn't resolve the issue.

When I did "sh policy-map int gi0/1" , I don't see any packet drops. But still I got ping drops and alien voice.

class-map match-any 122-MEDIUM

match access-group 122

class-map match-any 121-HIGH

match access-group 121

class-map match-any 120-VOICE

match access-group 120

class-map match-any 123-LOW

match access-group 123

!

policy-map QOS

class 120-VOICE

priority 2560

set ip dscp ef

class 121-HIGH

set ip dscp af41

bandwidth percent 5

class 123-LOW

set ip dscp default

bandwidth percent 3

class 122-MEDIUM

set ip dscp af21

bandwidth percent 35

class class-default

set ip dscp af11

bandwidth percent 15

policy-map Parent

class class-default

shape average 10240000

service-policy QOS

!

interface GigabitEthernet0/1

service-policy output Parent

!

#sh policy-map int gi0/1

GigabitEthernet0/1

Service-policy output: Parent

Class-map: class-default (match-any)

16080010 packets, 9169966477 bytes

30 second offered rate 1777000 bps, drop rate 0 bps

Match: any

Queueing

queue limit 64 packets

(queue depth/total drops/no-buffer drops) 0/1875/0

(pkts output/bytes output) 15522633/8773105768

shape (average) cir 10240000, bc 40960, be 40960

target shape rate 10240000

Service-policy : QOS

queue stats for all priority classes:

queue limit 64 packets

(queue depth/total drops/no-buffer drops) 0/0/0

(pkts output/bytes output) 21436/1795699

Class-map: 120-VOICE (match-any)

21436 packets, 1795699 bytes

30 second offered rate 32000 bps, drop rate 0 bps

Match: access-group 120

4071 packets, 321154 bytes

30 second rate 32000 bps

Priority: 2560 kbps, burst bytes 64000, b/w exceed drops: 0

QoS Set

dscp ef

Packets marked 21436

........

Thanks,

Raymond

Terry Cheema · ‎12-20-2012

On your QoS, you are matching all your voice subnet ip and priortising it as CoS1 (both signaling and media as ef), from australia. SG WAN router is marking correctly the traffic from CUCM/phones at SG back to AU?

There are no packet drops anywhere in your outputs. This clearly indicates the QoS is not dropping your packets. What is the ping test you are doing? Are you trying to ping from this router in australia to singapore? If you are trying to ping cucm server it will always drop every about sixth ping due to rate limiting on the cucm. If you remove the QoS policy if things improve, did you check ping as well?? Try ping tests with QoS removed to rule out any problem with circuit as its newly provisioned. Try extended ping tests with both QoS on and removed, with repeat count set to 100. Also with normal ping test, if you are doing from router may not (if its not matching cos1 traffic) give you the result for CoS1 priority traffic unless you set tos byte to ef in hexadecimal (0xB8 ?). Can you post the ping results?

Sent from Cisco Technical Support iPad App

raymondtun · ‎12-20-2012

Yes, our voice infrastracture is 10.146.0.0/16 for asia pacific.

Yes, all routers have the same QoS settings.

The ping test is from Singapore (LAN, from my laptop) to Asutraila voice VLAN interface on the core switch. It was a continuous ping.

Ping drop only when QoS is enalbled and at least one call is in progress. 70-80% ping drop.

No ping drop when QoS is enabled and no call.

No ping drop when QoS is disabled and call is in progress.

No ping drop when QoS is disabled and no call.

Terry Cheema · ‎12-21-2012

Hi Tun Aung San,

Its very strange behavior. Is it happening only from new AU site to one of the SG site? How about new site to any other sites inside\outside of Australia? I think i have looked at few times. You are matching your Voice subnet and priortising with 2.5 Mb bandwidth. Which looks correct.

To troubleshoot further i think we will need more information. At the minimum sh run from SG HQ site and AU site.

I think better to log a case with your provider, speak to them (check with them they are not applying any policer or anything like that.. safe to check)? Also if you try to ping with increase the packet size, maximum or close to that does that goes through ok?

Terry

paolo bevilacqua · ‎12-19-2012

In addition to the correct advice given by Brian above, there is something else you can do: nothing. This effective philoshoy is called "if ain't broken, don't touch it".