QoS over T-1

[pcanters]

Hello all.

I wonder if anyone has a moment to look at what I am seeing

I am getting reports of voice quality between two sites being "mushy"

I immidiately thought of my qos settings between the two sites

I just don't see any major issues here with the voice not having enough bandwidth or capacity

Perhaps I have misconfigured the Qos

I have applied the service policies to the correct serial interfaces

Here is what I see on both ends of the T-1 and also the policies that I am applying

I wonder if anyone sees something wrong here that could explain the mushy voice

Remote site Policy Map

sh policy-map interface serial 0/0/0:1

Serial0/0/0:1

  Service-policy output: QoS_Policy

    queue stats for all priority classes:

      Queueing

      queue limit 64 packets

      (queue depth/total drops/no-buffer drops) 0/0/0

      (pkts output/bytes output) 7553453/1540901232

    Class-map: voice-signal (match-all) 

      3654998 packets, 191051299 bytes

      5 minute offered rate 1000 bps, drop rate 0000 bps

      Match: access-group name VoIP_Control

      Queueing

      queue limit 64 packets

      (queue depth/total drops/no-buffer drops) 0/0/0

      (pkts output/bytes output) 3654998/190194851

      QoS Set

        dscp af31

          Packets marked 3654998

      bandwidth 50 kbps

    Class-map: voice-traffic (match-all) 

      7553453 packets, 1540901232 bytes

      5 minute offered rate 42000 bps, drop rate 0000 bps

      Match: ip rtp 16384 16383

      QoS Set

        dscp ef

          Packets marked 7553453

      Priority: 500 kbps, burst bytes 12500, b/w exceed drops: 0

    Class-map: class-default (match-any) 

      23953059 packets, 5356704274 bytes

      5 minute offered rate 34000 bps, drop rate 0000 bps

      Match: any

      Queueing

      queue limit 64 packets

      (queue depth/total drops/no-buffer drops/flowdrops) 0/12992/0/12991

      (pkts output/bytes output) 23940068/5337017437

      Fair-queue: per-flow queue limit 16 packets

------------------------------------------------------------------------------------------------------------

Head End policy Map

Gateway#sh policy-map interface serial 0/3/0:1

Serial0/3/0:1

  Service-policy output: QoS_Policy

    Class-map: voice-signal (match-all)

      2338141 packets, 138455767 bytes

      5 minute offered rate 0 bps, drop rate 0 bps

      Match: access-group name VoIP_Control

      QoS Set

        dscp af31

          Packets marked 2338141

      Queueing

        Output Queue: Conversation 265

        Bandwidth 50 (kbps) Max Threshold 64 (packets)

        (pkts matched/bytes matched) 195145/16283834

        (depth/total drops/no-buffer drops) 0/0/0

    Class-map: voice-traffic (match-all)

      8494849 packets, 1732758007 bytes

      5 minute offered rate 17000 bps, drop rate 0 bps

      Match: ip rtp 16384 16383

      QoS Set

        dscp ef

          Packets marked 8494849

      Queueing

        Strict Priority

        Output Queue: Conversation 264

        Bandwidth 500 (kbps) Burst 12500 (Bytes)

        (pkts matched/bytes matched) 1380453/281580578

        (total drops/bytes drops) 0/0

    Class-map: class-default (match-any)

      24704739 packets, 14011278776 bytes

      5 minute offered rate 147000 bps, drop rate 0 bps

      Match: any

      Queueing

        Flow Based Fair Queueing

        Maximum Number of Hashed Queues 256

        (total queued/total drops/no-buffer drops) 0/4802/0

--------------------------------------------------------------------------------------------------

QOS config used on both ends

!       

class-map match-all voice-traffic

match ip rtp 16384 16383

class-map match-all voice-signal

match access-group name VoIP_Control

!

!

policy-map QoS_Policy

class voice-signal

  set ip dscp af31

  bandwidth 50

class voice-traffic

  set ip dscp ef

  priority 500

class class-default

  fair-queue

ip access-list extended VoIP_Control

permit tcp any any eq 1720

permit tcp any any range 11000 11999

permit udp any any eq 2427

permit tcp any any eq 2428

permit tcp any any range 2000 2002

permit udp any any eq 1719

permit udp any any eq 5060

permit tcp any eq 1720 any

permit tcp any range 11000 11999 any

permit udp any eq 2427 any

permit tcp any eq 2428 any

permit tcp any range 2000 2002 any

permit udp any eq 1719 any

permit udp any eq 5060 any

[stephenshaw]

Hi pcanters,

a few of questions to ask .....

• Is the quality affected only for one particular call or are all calls affected at the same time?
  
• what codec is being used for Voice and how much bandwidth are you provisioning per call?
  
• I see you have set a priority of 500Kbps; how do you control the number of concurrent calls? i.e. is it possible that 500Kbps is insufficient for the number of concurrent calls? Keep in mind if you don't have sufficient bandwidth, all of the calls suffer in quality, not just the call that potentially pushes the bandwidth above 500Kbps
  
• I didn't notice anything out of the ordinary in terms of the config or the outputs posted
  

I won't be able to follow up on this post but hope these questions lead you in a direction that solves the problem and potentially another person can help out.

Cheers,

Steve

[pcanters]

first off...thanks for the response

Valid questions

• Is the quality affected only for one particular call or are all calls affected at the same time?

I am actually not clear on the answer here. Might have to investigate further

• what codec is being used for Voice and how much bandwidth are you provisioning per call?

I am using the g.711 codec and not using compression at this time

I am thinking I might just go do this now though to see if it fixes it

• I  see you have set a priority of 500Kbps; how do you control the number  of concurrent calls? i.e. is it possible that 500Kbps is insufficient  for the number of concurrent calls? Keep in mind if you don't have  sufficient bandwidth, all of the calls suffer in quality, not just the call that potentially pushes the bandwidth above 500Kbps

I would think that the 500 Kbps is sufficient otherwise I might see drops in the Qos Policy. I could be wrong in this line of thinking.

[glbeach]

Hi Pcanters,

A couple more questions and comments that  I hope can help you out.

1) Is the call quality issue experienced in only ONE or in BOTH directions of the call? This can be really helpfull in ruling or out many variables as the possible source of the issue.

2) Do we know if the call disruption is related to dropped packets or delay?

     - I would guess delay but you may be able to confirm this with the end point appliances if they show jitter numbers. IF in fact we believe it could be jitter then we should look into the following.

2a. Are we possibly getting non voip packets between the packets?

2b. Is there some form of delay being introduced through the network?

-=  For reference 4 Types of delay include,

      (1)Transit Delay = End to end time of travel like RTT (just below speed of light) and is affected by didstance between endpoint.  Likely not our issue here or it would be consistant and not commonly the problem with voip call quality in general.

     (2)Serialization Delay = The delay introduced by the amount of time it takes to put the packet on the wire from the hardware queue to pulses on the wire. This form of delay is most relevent on links with speeds 768 and below.

     (3)Processing Delay = Delay introduced by intermediate devices as the result of internal processing of the packet. Things like High CPU would be an example of an event that could introduce and increased delay due to processing.,

     (4) Queueing Delay = The delay resulting from congestion and buffering (What low latency queueing mechanisms guard against)

3) The interface is a Serial T1, but is this a Point to Point T1 all channels at 1536 kbps?

     - If so great!  Then as there are no drops in the output classes for voip here we know that the traffic is being low latency queued.

3a. Do we know that all the traffic we are matching and marking is in fact voip Call data? Is there a possibility some other applications in the network could be getting matched in with or spoofing these ports?  Bittorent or some other port swapping app?  If the voip appliances you have can mark DSCP, COS or PREC, matching one of these values in the policy map instead of RTP ports might help to test and eliminate the possibility. You could also use a sniffer capture on both sides to look at a bad call and validate if there are missing packets and the intergap delay as well.

     -If not...

3b. If the link is not "Point-To-Point" T1 but maybe something like MPLS or Frame-Relay, then we would want to check the provider is honoring low latency queued traffic at hte 500K rate and that they are in fact looking for the same markings we are setting in our policy.

Hope this gives some food for thought and helps you in your discovery.

[pcanters]

A lot of info here !!

It is indeed a point to point T1...no MPLS

so I also assume this :

If so great!  Then as there are no drops in the output classes for voip  here we know that the traffic is being low latency queued.

I do not suspect any port hopping or spoofing

I do not know if the call quality issue is one way or both ways

All things I need to get to the bottom of

I am changing to a g729 codec to see if it makes any difference

I don't think it's delay as my RTT times seem well below the 150 ms requirement on a continual basis

I suspect I will need to continue to look at that to see if it is true

I suspect that maybe we are going over the 500 Kbps limit which could affect the call quality

The easy answer would be to increase the priority for the voice payload and see if that fixes it

Instead I think I will try the xcoding and the 729 codec

Thanks so much for the help !!

I'll let you know what I find

[bbaillie]

Hi,

Three things come to mind, serialization delay, excessive jitter. , and queuing delay, bear with me on this.

VOIP on a T1 shouldn't suffer from serialization delay, but the operative word is shouldn't. The clocking out of the largest packets does take a finite amount of time, and large packets are usually TCP based.

So to verify this issue ensure the TCP packets are made smaller by using the command "ip tcp adjust-mss 1000". This will cause the end stations to negotiate a smaller MSS and therefore smaller TCP packets meanss less jitter.

.

Next issue is the number of transmit ring buffers on the serial interface, after the packets are placed in the transmit ring buffers, the router insert VOIP packets between the other packets to ensure the VOIP doesn't suffer excess delay/jitter. To view the current transmit ring buffer setting use the command "show controllers s0/0/1. To help this use the command "tx-ring-limit 2" on the interface.

Both of thes commands are easily removed with the commands "no ip tcp adjust-mss 1000", and "no tx-ring-limit 2", "no tx-queue-limit 2"

Try one or both to determine if this is your issue, it is still possible that the VOIP packets aren't marked correctly, so when you are sure a VOIP call is in progress, use the command "show policy-map interface serial 0/0/0" to determine for certain if the traffic is actually flowing through the VOIP queue.

Cheers,

Brian