Help understanding interface statistics?

[somwicked]

Recently setup a PPP WAN with T1 between 2 1750 routers.

I'm watching the line and noticing what I believe to be errors - but being a noob, need help in understanding what I'm seeing.

I cleared the counters on both routers today @ 11:30a. These statistics were taken @ 4:15p. Run time of approx 5 hours.

Main router#show interface serial0

Serial0 is up, line protocol is up

Hardware is PQUICC with Fractional T1 CSU/DSU

Description: WAN to Bear Lake

Internet address is 192.168.33.1/24

MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation PPP, loopback not set

Keepalive set (10 sec)

LCP Open

Open: IPCP, CDPCP

Last input 00:00:03, output 00:00:00, output hang never

Last clearing of "show interface" counters 04:43:04

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 67

Queueing strategy: weighted fair

Output queue: 0/1000/64/0 (size/max total/threshold/drops)

Conversations 0/12/256 (active/max active/max total)

Reserved Conversations 0/0 (allocated/max allocated)

Available Bandwidth 1158 kilobits/sec

5 minute input rate 8000 bits/sec, 22 packets/sec

5 minute output rate 7000 bits/sec, 22 packets/sec

212764 packets input, 18813675 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

9140 input errors, 2569 CRC, 5371 frame, 0 overrun, 0 ignored, 1200 abort

211632 packets output, 27433416 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 output buffer failures, 0 output buffers swapped out

2 carrier transitions

DCD=up DSR=up DTR=up RTS=up CTS=up

main router#show service-module serial0

Module type is T1/fractional

Hardware revision is 0.96, Software revision is 0.2,

Image checksum is 0x73D70058, Protocol revision is 0.1

Receiver has no alarms.

Framing is ESF, Line Code is B8ZS, Current clock source is line,

Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.

Last module self-test (done at startup): Passed

Last clearing of alarm counters 04:44:50

loss of signal : 0,

loss of frame : 0,

AIS alarm : 0,

Remote alarm : 0,

Module access errors : 0,

Total Data (last 96 15 minute intervals):

0 Line Code Violations, 1652 Path Code Violations

0 Slip Secs, 31 Fr Loss Secs, 0 Line Err Secs, 196 Degraded Mins

243 Errored Secs, 230 Bursty Err Secs, 31 Severely Err Secs, 0 Unavail Secs

Data in current interval (118 seconds elapsed):

0 Line Code Violations, 0 Path Code Violations

0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

[somwicked]

Due to message length limit, here is the remote router info.

remote router#show interface serial0

Serial0 is up, line protocol is up

Hardware is PQUICC with Fractional T1 CSU/DSU

Description: WAN to Zeeland

Internet address is 192.168.33.2/24

MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation PPP, loopback not set

Keepalive set (10 sec)

LCP Open

Open: IPCP, CDPCP

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 04:42:59

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 70

Queueing strategy: weighted fair

Output queue: 0/1000/64/0 (size/max total/threshold/drops)

Conversations 0/8/256 (active/max active/max total)

Reserved Conversations 0/0 (allocated/max allocated)

Available Bandwidth 1158 kilobits/sec

5 minute input rate 1000 bits/sec, 14 packets/sec

5 minute output rate 1000 bits/sec, 14 packets/sec

211635 packets input, 27433443 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

212962 packets output, 18847262 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 output buffer failures, 0 output buffers swapped out

2 carrier transitions

DCD=up DSR=up DTR=up RTS=up CTS=up

remote router#show service-module serial0

Module type is T1/fractional

Hardware revision is 0.96, Software revision is 0.2,

Image checksum is 0x73D70058, Protocol revision is 0.1

Receiver has no alarms.

Framing is ESF, Line Code is B8ZS, Current clock source is line,

Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.

Last module self-test (done at startup): Passed

Last clearing of alarm counters 04:43:37

loss of signal : 0,

loss of frame : 0,

AIS alarm : 0,

Remote alarm : 1, last occurred 01:41:24

Module access errors : 0,

Total Data (last 96 15 minute intervals):

14999 Line Code Violations, 7 Path Code Violations

0 Slip Secs, 30 Fr Loss Secs, 7007 Line Err Secs, 1373 Degraded Mins

6972 Errored Secs, 0 Bursty Err Secs, 35 Severely Err Secs, 0 Unavail Secs

Data in current interval (412 seconds elapsed):

0 Line Code Violations, 0 Path Code Violations

0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

I'm mainly concerned about QoS on this line due to running VoIP between the sites.

Pointers? Guidance? Much appreciated.

[teyobanilom]

Looks like you have a noisy link.

The link explains each counters.

http://www.cisco.com/en/US/products/sw/iosswrel/ps1835/products_command_reference_chapter09186a00800874c4.html#wp1020497

HTH.

[ggatten]

You definetely have issues. I'd have your telco do intrusive testing on this circuit.

Also, double check with them and make sure the circuit is provisioned as your router is config'd:

- ESF

- B8ZS

- Make sure they are providing clock.

Good luck!

Gary

[somwicked]

Thanks for the replies guys.

Exactly the information I was looking for.

Now to figure out the problem.

[dbellaze]

Both routers are set to accept clock from the line. In a point to point circuit like this you want to set one side of the T1's to clock source internal.

service-module t1 clock source internal

I have seen many T1's where the clock seems to generate errors on one end of the link more then the other so I would first set the main site to internal and then watch the error rate. If there are still many errors than try setting the remote router to internal.

Daniel

[somwicked]

Daniel,

Just tried setting the main site to an internal clock source with the remote site set to line.

As soon as I changed the main site clock source to internal, i lost all connectivity - the line dropped and never came back up.

I would love to change the remote site to internal and test, but am afraid of losing the line and my access if it goes down.

[teyobanilom]

I believe this link will help.

http://www.cisco.com/univercd/cc/td/doc/cisintwk/itg_v1/tr1915.htm

I think there's a lot of possible sources. One thing's for sure, you have to work it out with your service provider. Here are my suggestions;

1. Just make sure the integrity of your port is OK

2. Make sure that configuration on your routers (clocking, framing and linecoding) correspond to the link (your service provider should know).

3. If problem still exist, perform loopback test.

HTH.

[somwicked]

Ok. Been doing some digging into further documentation (while I wait for the telcos) and have a question regarding clocking.

I know I can set the controller on one end to clock source internal - which was recommended previously.

When I attempted this, the line dropped out completely.

I was reading about SCTE and having to enable it. From what I understand, it has to be enabled for the internal clock to function. (I may be wrong on this)

Is there a way to check if this enabled or disabled?

If not enabled, does the command "transmit-clock-internal" enable it on the 1750 (as I read it does on other devices)?

[somwicked]

More digging - since I am loosing my mind with this (being the weekend and all).

I've been watching the line and statistics all weekend. Sometime Saturday/Sunday, I noticed the CRC and frame errors practically disappeared. Only to be replaced with lots of line code and path code violations. My guess is the line was totally hosed and not passing any data at that point. During this time, nothing changed on the routers other than clearing the counters periodically.

I've also ran a show controllers serial command on each router. Both of them indicate that a TX and RX clock is detected - both are configured for line. At least it makes sense why the line dropped when I changed the clock source.

[somwicked]

Been working with the telco on this and they insist they are not providing clocking, and have ran tests and ensure the line is clean.

If I set either end to clock source internal - the line drops out completely.

I've also noticed that if I set the clock rate (i.e. clock rate 64000) for the serial interface with clocking internal, nothing is shown in the config indicating it was set.

Am I overlooking something here?

Would the line even function if it was not getting clocking from somewhere?

[ggatten]

Who is your telco? Maybe you can call your sales rep (if you have one) and tell them to cancel the order because it doesn't work? That usually gets someone off their a$$ to fix the problem.

1.) Using a T1 Loopback, ENSURE you can loop up the CSU/DSU in each router and it tests clean. I made some female loopback connectors so I can DC from the SmartJack and loop there. This ensures ALL CPE, including the cable, is good.

2.) If you have a long cable between demarc and CSU/DSU, check your LBO. The loopback test above should find any problems here as well.

3.) Look at the Smartjack and post which lights are red and/or green.

4.) Maybe post your configs again as an attachment?

Gary