T1 / PRI drops occassionally (AIS detected)

Frank den Haan · ‎02-01-2012

Hello,

For the last few months, we have been experiencing occassional, brief drops on one of our T1 / PRI circuits. It's probably once every 3 weeks or so. Not too frequent, but nevertheless we would like it resolved.

It's connected to a Cisco 2851 with IOS c2800nm-spservicesk9-mz.124-24.T5.bin

Actions taken on our end:

- Replaced VWIC-1MFT-T1 card

- Changed cable from the VWIC-1MFT-T1 card to the Telco's PRI circuit/card.

- Router has been reloaded

Telco's actions:

- Changed cable pair several times

- Actually installed a new PRI

- Confirmed framing and line code settings are the same as our equipment: Framing is ESF, Line Code is B8ZS, Clock Source is Line

But, it's still happening. The configuration at this problematic site is the same as our many other sites with PRIs/Gateways.

Although I am currently taking the issue up with the Telco yet again, I was wondering if there is anything else I could do on my end to troubleshoot this. Any help is appreciated, thanks!

Recent sh log output:

Feb 1 05:24:51.811 AST: %CONTROLLER-5-UPDOWN: Controller T1 0/0/0, changed state to down (AIS detected)

Feb 1 05:24:51.815 AST: %ISDN-6-LAYER2DOWN: Layer 2 for Interface Se0/0/0:23, TEI 0 changed to down

Feb 1 05:24:52.063 AST: %MARS_NETCLK-3-HOLDOVER: Entering Holdover for Controller T1 0/0/0

Feb 1 05:24:53.811 AST: %LINK-3-UPDOWN: Interface Serial0/0/0:23, changed state to down

Feb 1 05:25:02.311 AST: %MARS_NETCLK-3-HOLDOVER_TRANS: Holdover timer exceeded for Controller T1 0/0/0

Feb 1 05:25:02.311 AST: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 1 to priority 10

Feb 1 05:25:39.811 AST: %CONTROLLER-5-UPDOWN: Controller T1 0/0/0, changed state to up

Feb 1 05:25:40.815 AST: %ISDN-6-LAYER2UP: Layer 2 for Interface Se0/0/0:23, TEI 0 changed to up

Feb 1 05:25:41.811 AST: %LINK-3-UPDOWN: Interface Serial0/0/0:23, changed state to up

Feb 1 05:25:50.311 AST: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 10 to priority 1

Feb 1 05:34:31.814 AST: %CONTROLLER-5-UPDOWN: Controller T1 0/0/0, changed state to down (AIS detected)

Feb 1 05:34:31.818 AST: %ISDN-6-LAYER2DOWN: Layer 2 for Interface Se0/0/0:23, TEI 0 changed to down

Feb 1 05:34:31.822 AST: %MARS_NETCLK-3-HOLDOVER: Entering Holdover for Controller T1 0/0/0

Feb 1 05:34:33.814 AST: %LINK-3-UPDOWN: Interface Serial0/0/0:23, changed state to down

Feb 1 05:34:42.074 AST: %MARS_NETCLK-3-HOLDOVER_TRANS: Holdover timer exceeded for Controller T1 0/0/0

Feb 1 05:34:42.074 AST: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 1 to priority 10

Feb 1 05:35:16.814 AST: %CONTROLLER-5-UPDOWN: Controller T1 0/0/0, changed state to up

Feb 1 05:35:17.818 AST: %ISDN-6-LAYER2UP: Layer 2 for Interface Se0/0/0:23, TEI 0 changed to up

Feb 1 05:35:18.814 AST: %LINK-3-UPDOWN: Interface Serial0/0/0:23, changed state to up

Feb 1 05:35:27.074 AST: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 10 to priority 1

Feb 1 05:41:55.816 AST: %CONTROLLER-5-UPDOWN: Controller T1 0/0/0, changed state to down (AIS detected)

Feb 1 05:41:55.820 AST: %ISDN-6-LAYER2DOWN: Layer 2 for Interface Se0/0/0:23, TEI 0 changed to down

Feb 1 05:41:55.836 AST: %MARS_NETCLK-3-HOLDOVER: Entering Holdover for Controller T1 0/0/0

Feb 1 05:41:57.816 AST: %LINK-3-UPDOWN: Interface Serial0/0/0:23, changed state to down

Feb 1 05:42:06.084 AST: %MARS_NETCLK-3-HOLDOVER_TRANS: Holdover timer exceeded for Controller T1 0/0/0

Feb 1 05:42:06.084 AST: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 1 to priority 10

Feb 1 05:42:41.816 AST: %CONTROLLER-5-UPDOWN: Controller T1 0/0/0, changed state to up

Feb 1 05:42:42.824 AST: %ISDN-6-LAYER2UP: Layer 2 for Interface Se0/0/0:23, TEI 0 changed to up

Feb 1 05:42:43.816 AST: %LINK-3-UPDOWN: Interface Serial0/0/0:23, changed state to up

Feb 1 05:42:52.084 AST: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 10 to priority 1

_________________

Router name#sh int serial 0/0/0:23
Serial0/0/0:23 is up, line protocol is up (spoofing)
Hardware is DSX1
Description: XXXXXXXXXXXXX
MTU 1500 bytes, BW 64 Kbit/sec, DLY 20000 usec,
     reliability 253/255, txload 1/255, rxload 1/255
Encapsulation HDLC, loopback not set
Last input 00:00:27, output never, output hang never
Last clearing of "show interface" counters 6w0d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations 0/1/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 48 kilobits/sec
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
     180498 packets input, 1062948 bytes, 0 no buffer
     Received 0 broadcasts, 3324 runts, 0 giants, 0 throttles
     9407 input errors, 9407 CRC, 0 frame, 0 overrun, 0 ignored, 2232 abort
     180505 packets output, 1140153 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
     6 carrier transitions
Timeslot(s) Used:24, SCC: 0, Transmitter delay is 0 flags

________________________

Router name#sh controller t1 0/0/0
T1 0/0/0 is up.
Applique type is Channelized T1
Cablelength is long gain36 0db
Description: XXXXXXXXXXXXXXXXX
No alarms detected.
alarm-trigger is not set
Soaking time: 3, Clearance time: 10
AIS State:Clear LOS State:Clear LOF State:Clear
Version info Firmware: 20090113, FPGA: 20, spm_count = 0
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
CRC Threshold is 320. Reported from firmware is 320.
Data in current interval (825 seconds elapsed):
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 1:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 2:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 3:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 4:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 5:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 6:
     0 Line Code Violations, 41 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 9 Degraded Mins
     15 Errored Secs, 8 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 7:
     2 Line Code Violations, 1412 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 7 Degraded Mins
     14 Errored Secs, 10 Bursty Err Secs, 1 Severely Err Secs, 18 Unavail Secs
Data in Interval 8:
     0 Line Code Violations, 4390 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 8 Degraded Mins
     23 Errored Secs, 15 Bursty Err Secs, 6 Severely Err Secs, 25 Unavail Secs
Data in Interval 9:
     0 Line Code Violations, 26 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 4 Degraded Mins
     9 Errored Secs, 6 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 10:
     0 Line Code Violations, 19 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 5 Degraded Mins
     6 Errored Secs, 4 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 11:
     0 Line Code Violations, 13 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 5 Degraded Mins
     8 Errored Secs, 2 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 12:
     0 Line Code Violations, 18 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 3 Degraded Mins
     6 Errored Secs, 5 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 13:
     0 Line Code Violations, 5 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 5 Degraded Mins
     5 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 14:
     0 Line Code Violations, 136 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 8 Degraded Mins
     15 Errored Secs, 11 Bursty Err Secs, 1 Severely Err Secs, 0 Unavail Secs
Data in Interval 15:
     0 Line Code Violations, 107 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 7 Degraded Mins
     18 Errored Secs, 13 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 16:
     0 Line Code Violations, 50 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 6 Degraded Mins
     16 Errored Secs, 11 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 17:
     0 Line Code Violations, 216 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 8 Degraded Mins
     40 Errored Secs, 20 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 18:
     0 Line Code Violations, 71 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 6 Degraded Mins
     19 Errored Secs, 9 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 19:
     0 Line Code Violations, 204 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 8 Degraded Mins
     40 Errored Secs, 17 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 20:
     0 Line Code Violations, 228 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 12 Degraded Mins
     64 Errored Secs, 25 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 21:
     0 Line Code Violations, 62 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 6 Degraded Mins
     13 Errored Secs, 8 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 22:
     0 Line Code Violations, 39 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 6 Degraded Mins
     7 Errored Secs, 4 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 23:
     0 Line Code Violations, 32 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 9 Degraded Mins
     10 Errored Secs, 4 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 24:
     0 Line Code Violations, 182 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 11 Degraded Mins
     31 Errored Secs, 17 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 25:
     0 Line Code Violations, 3668 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 14 Degraded Mins
     121 Errored Secs, 74 Bursty Err Secs, 5 Severely Err Secs, 46 Unavail Secs
Data in Interval 26:
     1 Line Code Violations, 8761 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 12 Degraded Mins
     97 Errored Secs, 61 Bursty Err Secs, 7 Severely Err Secs, 93 Unavail Secs
Data in Interval 27:
     0 Line Code Violations, 10 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 2 Degraded Mins
     5 Errored Secs, 1 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 28:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 29:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 30:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 31:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 32:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 33:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 34:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 35:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 36:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 37:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 38:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 39:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 40:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 41:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 42:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 43:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 44:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 45:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 46:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 47:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 48:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 49:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 50:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 51:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 52:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 53:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 54:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 55:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 56:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 57:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 58:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 59:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 60:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 61:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 62:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 63:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 64:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 65:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 66:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 67:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 68:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 69:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 70:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 71:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 72:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 73:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 74:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 75:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 76:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 77:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 78:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 79:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 80:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 81:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 82:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 83:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 84:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 85:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 86:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 87:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 88:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 89:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 90:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 91:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 92:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 93:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 94:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 95:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in Interval 96:
     0 Line Code Violations, 0 Path Code Violations
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
     0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Total Data (last 24 hours)
     3 Line Code Violations, 19690 Path Code Violations,
     0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 161 Degraded Mins,
     582 Errored Secs, 325 Bursty Err Secs, 20 Severely Err Secs, 182 Unavail Secs

______________________________________

Some parts of the config:

interface Serial0/0/0:23
description XXXXXXXXXXXXXXX
no ip address
encapsulation hdlc
isdn switch-type primary-ni
isdn incoming-voice voice
no cdp enable

controller T1 0/0/0
pri-group timeslots 1-24
description XXXXXXXXXXXXX

network-clock-participate wic 0
network-clock-select 1 T1 0/0/0

YHZBUR-R0#sh int serial 0/0/0:23

Serial0/0/0:23 is up, line protocol is up (spoofing)

Hardware is DSX1

Description: PRI ID 07/IPCD/131808/000/MRTM/000

MTU 1500 bytes, BW 64 Kbit/sec, DLY 20000 usec,

reliability 253/255, txload 1/255, rxload 1/255

Encapsulation HDLC, loopback not set

Last input 00:00:27, output never, output hang never

Last clearing of "show interface" counters 6w0d

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: weighted fair

Output queue: 0/1000/64/0 (size/max total/threshold/drops)

Conversations 0/1/256 (active/max active/max total)

Reserved Conversations 0/0 (allocated/max allocated)

Available Bandwidth 48 kilobits/sec

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

180498 packets input, 1062948 bytes, 0 no buffer

Received 0 broadcasts, 3324 runts, 0 giants, 0 throttles

9407 input errors, 9407 CRC, 0 frame, 0 overrun, 0 ignored, 2232 abort

180505 packets output, 1140153 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 unknown protocol drops

0 output buffer failures, 0 output buffers swapped out

6 carrier transitions

Timeslot(s) Used:24, SCC: 0, Transmitter delay is 0 flags

Frank den Haan · ‎07-03-2012

Just to close this one out, in the end the problem was on the Telco's side. They finally replaced the circuit with a fibre-based T1 and that resolved it.