Multilink has one line with errors

[qbakies11]

SiteA and Site B are connected Point-to-Point with two 1.5 Mbps T1 lines.  In order to bundle the traffic into a single 3 Mbps line I have created a Multilink group and added both lines to it on the router at each site.  Recently we have been having slowness on the connection where it wasn't before.  I did a 'sh controller t1' command to see if there were errors and the first one I did (0/2/0) was fin and showed everything fine but the second line (0/2/1) has a bunch of errors:

SiteAGW#sh controller t1 0/2/1

T1 0/2/1 is up.

  Applique type is Channelized T1

  Cablelength is long gain36 0db

  No alarms detected.

  alarm-trigger is not set

  Soaking time: 3, Clearance time: 10

  AIS State:Clear  LOS State:Clear  LOF State:Clear

  Version info Firmware: 20070320, FPGA: 20, spm_count = 0

  Framing is ESF, Line Code is B8ZS, Clock Source is Line.

  CRC Threshold is 320. Reported from firmware  is 320.

  Data in current interval (461 seconds elapsed):

     0 Line Code Violations, 0 Path Code Violations

     461 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

     461 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 1:

     0 Line Code Violations, 1371 Path Code Violations

     897 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

     895 Errored Secs, 0 Bursty Err Secs, 2 Severely Err Secs, 0 Unavail Secs

  Data in Interval 2:

     0 Line Code Violations, 2800 Path Code Violations

     895 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

     889 Errored Secs, 0 Bursty Err Secs, 8 Severely Err Secs, 0 Unavail Secs

  Data in Interval 3:

     0 Line Code Violations, 2854 Path Code Violations

     890 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

     885 Errored Secs, 0 Bursty Err Secs, 8 Severely Err Secs, 0 Unavail Secs

I logged onto SiteB's router and found the same thing.  The first T1 I looked at (0/0/0) was fine but the second one (0/0/1) had a bunch of errors:

SiteBGW#sh controller t1 0/0/1

T1 0/0/1 is up.

  Applique type is Channelized T1

  Cablelength is long 0db

  No alarms detected.

  alarm-trigger is not set

  Soaking time: 3, Clearance time: 10

  AIS State:Clear  LOS State:Clear  LOF State:Clear

  Version info Firmware: 20071011, FPGA: 13, spm_count = 0

  Framing is ESF, Line Code is B8ZS, Clock Source is Line.

  CRC Threshold is 320. Reported from firmware  is 320.

  Data in current interval (559 seconds elapsed):

     0 Line Code Violations, 57140 Path Code Violations

     556 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 10 Degraded Mins

     559 Errored Secs, 559 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 1:

     0 Line Code Violations, 88777 Path Code Violations

     900 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 15 Degraded Mins

     900 Errored Secs, 900 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 2:

     0 Line Code Violations, 94691 Path Code Violations

     895 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 15 Degraded Mins

     900 Errored Secs, 900 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 3:

     0 Line Code Violations, 88393 Path Code Violations

     894 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 15 Degraded Mins

     900 Errored Secs, 900 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 4:

     0 Line Code Violations, 80340 Path Code Violations

     897 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 15 Degraded Mins

     900 Errored Secs, 900 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

Could this be causing our connection issues?  What could be causing just one of the lines on the Multilink team to do this and how do I fix it?

[Giuseppe Larosa · Accepted solution]

Hello,

>>

Data in current interval (559 seconds elapsed):

     0 Line Code Violations, 57140 Path Code Violations

     556 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 10 Degraded Mins

   >>>>>>>>>  559 Errored Secs, 559 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 1:

     0 Line Code Violations, 88777 Path Code Violations

     900 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 15 Degraded Mins

    >>>>> 900 Errored Secs, 900 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

  Data in Interval 2:

the line is really errored and this is limiting the capability to send and receive traffic over the whole multilink bundle.

every second is classified as an errored second!

each line is indipendent at physical layer so one line may be affected and the other may be in good status, however the multilink bundle tries to use both in an even manner

open a service request with WAN provider, in order to avoid problems as first step remove the line with troubles from the ppp multilink configuration so that loopback tests can be performed without impact on the bundle.

Hope to help

Giuseppe

[qbakies11]

I have removed the bad lines from the ppp multilink bundle and I'm still  getting errros showing on the circuit, is this normal?  Are they  framing errors?

[Giuseppe Larosa]

Hello Gbakies11,

>> I have removed the bad lines from the ppp multilink bundle and I'm still  getting errros showing on the circuit, is this normal?

yes, it is expected as they are OSI layer1 errors and are not related to the way you use/configure the line

see the following document about troubleshooting serial lines, E1/T1

http://www.cisco.com/en/US/tech/tk713/tk628/technologies_tech_note09186a00800a758d.shtml

Edit:

http://www.cisco.com/en/US/docs/internetworking/troubleshooting/guide/tr1915.html

Hope to help

Giuseppe

[qbakies11]

Our carrier replaced a smartjack at one of the sites and states that they are not seeing errors now but I'm still getting just as many when I do a 'sh controller t1' command.  They stated that isn't their issue because all they provide is the transport for the P2P and it is clear.  They stated that the timing may be wrong and I stated that the clock source was the line and then they told me I couldn't use that because they don't provide clocking.  I have the clock source as the line on all of my P2P lines configured though this provider (5 in total) and the others are fine.

Could this be a timing issue?  I've never manually set a clock source before as I've just always used the line.

Also, I show:

Serial0/0/1:0 is up, line protocol is down

Shouldn't that be correct since I have this serial interface setup for ppp multilink but don't have it added to a multilink group or should the line protocaol be up since it is plugged in at both ends of the P2P?