Problems on FR PVC with LLQ & FRF.12

[dragos.stroescu]

Actually implementing a voice solution FR PVC based between one central and manny distant sites. The QOS for voice is ensured using LLQ with FRF.12. CRTP is also used. Platforms used are 3660 and 26xx. We are encountering BIG voice quality issues on the 3660 platform using 12.2 mainline IOS as soon as the data rate on PVC goes near to CIR value (80% CIR). We are very concerned about that and actually searching for a more stable IOS release which can ensure the QOS needs or if there are other similar experiences in same environment.

Thanks

[tepatel]

We need to troubleshoot that voice quality issue when pvc goes near CIR.

Need to see what is happening with those voice packets..For that pl. post running config with following show commands when the problem is there regarding voice

sh run

sh call active voice

sh frame pvc (number)

[dragos.stroescu]

Thanks for answer.

I will be able to have that information only on monday!

I can add that from the voice perspective the 26xx and 3660 configurations are IDENTICAL! We have ALWAYS problems only in the direction 3660->26xx which indicates problems in handling the interleaving. At 3660 end we could always hear perfectly the voice originating from the 26xx end regardless of the load.

I have tested almost all IOS images available on 12.2 mainline! Also lab tests with identical configs were unable to reproduce the problem! FR provider was asked to increase the PVC CIR above the one configured on routers (on routers 128Kbps and SP has used 135Kbps) - same problems. Some times the problems are appearing at about 30-40Kbps traffic (near CIR voice becomes almost silence...).

Could that be HW related?

Thanks in advance

[tepatel]

I don't think that the problem can be because of hardware. Now the question is

1)was this a new installation or the exhisting one?

2)Was this setup ever worked? (Meaning was this problem there in the past or its newer one)

Now if the problem is there from bigining, then i don't doubt on hardware.

Now good to know that the issue is there in one direction only. so voice packets are getting dropped at either on 3660 or 26xx router. So those commands "sh frame pvc x/y" with "sh interface " should prove something.

[dragos.stroescu]

1) New installation!

2) Worked only once with 12.2.6a for some days (according to customer) but never after!

Found A LOT of problems in fast switching in 12.2 code which does NOT match packets to corresponding classes but to default one... Disabling fast switching - all starts matching correctly but voice quality problems still manifest. I suspect LFI working from time to time (like the VoIP on ISDN bug?)

I'll get the outputs and publish them!

Thanks

[dragos.stroescu]

Serial1/1:0 is up, line protocol is up

Hardware is DSCC4 Serial

Internet address is 172.16.27.1/30

MTU 1500 bytes, BW 128 Kbit, DLY 20000 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation FRAME-RELAY, loopback not set

Keepalive set (10 sec)

Restart-Delay is 65 secs

LMI enq sent 24047, LMI stat recvd 24047, LMI upd recvd 0, DTE LMI up

LMI enq recvd 0, LMI stat sent 0, LMI upd sent 0

LMI DLCI 0 LMI type is ANSI Annex D frame relay DTE

Broadcast queue 0/64, broadcasts sent/dropped 51959/0, interface

broadcasts 51

967

Last input 00:00:00, output 00:00:01, output hang never

Last clearing of "show interface" counters 2d18h

Input queue: 0/100/0/0 (size/max/drops/flushes); Total output drops: 8

Queueing strategy: dual fifo

Output queue: high size/max/dropped 0/2048/0

Output queue: 0/1024 (size/max)

30 second input rate 1000 bits/sec, 1 packets/sec

30 second output rate 1000 bits/sec, 1 packets/sec

546271 packets input, 47622376 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

2 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 2 abort

932808 packets output, 86627968 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 output buffer failures, 0 output buffers swapped out

1 carrier transitions

Timeslot(s) Used:1-31, SCC: 3, Transmitter delay is 0 flags

PVC Statistics for interface Serial1/1:0 (Frame Relay DTE)

DLCI = 100, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE =

Serial1/1:0

input pkts 175938 output pkts 191342 in bytes 44975033

out bytes 83119755 dropped pkts 0 in pkts dropped 0

out pkts dropped 8 out bytes dropped 11720

late-dropped out pkts 8 late-dropped out bytes 11720

in FECN pkts 0 in BECN pkts 0 out FECN pkts 0

out BECN pkts 0 in DE pkts 0 out DE pkts 0

out bcast pkts 51961 out bcast bytes 3325504

pvc create time 2d18h, last time pvc status changed 00:04:21

fragment type end-to-end fragment size 100

cir 128000 bc 12800 be 0 limit 1600 interval 100

mincir 128000 byte increment 1600 BECN response no

frags 908765 bytes 86244356 frags delayed 811955 bytes delayed

79526616

shaping inactive

traffic shaping drops 8

service policy DLSW_BUC_PRI

Serial1/1:0: DLCI 100 -

Service-policy output: DLSW_BUC_PRI

Class-map: BUC_TOS0 (match-all)

3482 packets, 1026312 bytes

30 second offered rate 0 bps, drop rate 0 bps

Match: access-group 2632

Queueing

Output Queue: Conversation 25

Bandwidth 40 (kbps) Max Threshold 64 (packets)

(pkts matched/bytes matched) 3482/1026312

(depth/total drops/no-buffer drops) 0/0/0

Class-map: BUC_TOS5 (match-all)

98994 packets, 74738584 bytes

30 second offered rate 0 bps, drop rate 0 bps

Match: access-group 2633

Queueing

Output Queue: Conversation 26

Bandwidth 52 (kbps) Max Threshold 64 (packets)

(pkts matched/bytes matched) 98994/74738584

(depth/total drops/no-buffer drops) 0/8/0

Class-map: BUC_VOICE (match-all)

42 packets, 2999 bytes

30 second offered rate 0 bps, drop rate 0 bps

Match: access-group 2634

Queueing

Strict Priority

Output Queue: Conversation 24

Bandwidth 36 (kbps) Burst 900 (Bytes)

(pkts matched/bytes matched) 42/2787

(total drops/bytes drops) 0/0

Class-map: class-default (match-any)

85917 packets, 6355686 bytes

30 second offered rate 0 bps, drop rate 0 bps

Match: any

Output queue size 0/max total 600/drops 8

[dragos.stroescu]

Looks like the issue was identified into the SP area on the respective PVC.

Thanks