ā05-11-2010 06:24 AM - edited ā03-04-2019 08:26 AM
Hello all,
I'm bringing up a new DS3 line, and i've got some questions. Basically the circut shows up / up, and the carrier is saying there are no problems on their end of the circut. I'm having a very hard time believing them because we've had issues all along bringing up this line, and they keep telling me the problem is not on their end, but thusfar, the problems have been at the carrier.
One of my serial interfaces has ALOT of CRC / framing problems:
Serial1/0 is up, line protocol is up
Hardware is DSXPNM Serial
Internet address is X.X.X.X
MTU 4470 bytes, BW 44210 Kbit/sec, DLY 200 usec,
reliability 200/255, txload 1/255, rxload 1/255
Encapsulation HDLC, crc 16, loopback not set
Keepalive set (10 sec)
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 08:26:52
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 11
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 37000 bits/sec, 8 packets/sec
5 minute output rate 25000 bits/sec, 7 packets/sec
254192 packets input, 101865695 bytes, 0 no buffer
Received 3546 broadcasts, 0 runts, 4022 giants, 0 throttles
343934 input errors, 343934 CRC, 205200 frame, 156868 overrun, 0 ignored, 185754 abort
249527 packets output, 89821686 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
6 carrier transitions
DSU mode 0, bandwidth 44210, real bandwidth 44210, scramble 0
The other interface looks good (I cleared the counters, but there were very few CRC errors on this side):
Serial4/0 is up, line protocol is up
Hardware is DSXPNM Serial
Internet address is X.X.X.X
MTU 4470 bytes, BW 44210 Kbit, DLY 200 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation HDLC, crc 16, loopback not set
Keepalive set (10 sec)
Last input 00:00:08, output 00:00:01, output hang never
Last clearing of "show interface" counters 01:04:40
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
289 packets input, 39491 bytes, 0 no buffer
Received 89 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
134 packets output, 12036 bytes, 0 underruns
0 output errors, 0 collisions, 105 interface resets
0 output buffer failures, 0 output buffers swapped out
1 carrier transitions
Any help would be very appreciated, because now it's turning into a blame game, where they are blaming our equipment / cabling, even though it is all brand new.
Thanks
Jon
ā05-12-2010 04:10 AM
Please send "show controllers T3" taken on both sides.
However, when there are such debates, ask SP to bring in their own testing equipment on premises in front of your eyes, do a 48h BER test (must give 1 * 10^-9 or better), and be sure to understand how is conducted and personally observe the results.
Otherwise, do not accept circuit delivery, and no not pays the bills.
ā05-12-2010 05:01 AM
You could perform a BERT test on a DS3 port on the router. The URL is for 7304 platform..
http://cisco.biz/en/US/docs/ios/12_1/12_1ex/feature/guide/e_t3bert.html#wp1045298
Francisco
ā05-12-2010 05:19 AM
But the point is exactly not using the router, beside such an high number of errors makes the test unnecessary.
The point is having telco to use their own equipment so they must stop blaming the router.
ā05-12-2010 05:28 AM
I guess you are right. telco should take full responsiblity and use their own equipment to test and rectify the issue..
Francisco.
ā05-12-2010 09:24 AM
I'm sorry for not getting back to you folks sooner, been having conference calls with my carrier to no avail. Basically what I've been doing is running through different types of ping tests, sending different types of data patterns & such, so far my findings have been pretty interesting.
If I try to ping through the link using data patterns of 0x0000, I get 95 - 98% loss of the pings, barely any go through. But any other data patterns such as 0xFFFF, 0x5555, 0xaaaa, I get 100% response on the pings. I've found a very interesting article that was written by someone having almost the exact same issue as myself, and their fix was that the alcatel mux's that some carriers use, by default, don't pass all zero's patterns, and this causes translation & linecode errors, thus producing CRC errors on the interface of the Cisco boxes.
Just an FYI, controller output is listed below.
Controller from the router I'm seeing all the CRC errors on.
T3 1/0 is up.
Applique type is Subrate T3
Description: DS3 Controller
No alarms detected.
MDL transmission is disabled
FEAC code received: No code is being received
Framing is C-BIT Parity, Line Code is B3ZS, Clock Source is Internal
Data in current interval (260 seconds elapsed):
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 1:
0 Line Code Violations, 247 P-bit Coding Violation
5 C-bit Coding Violation, 89 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
5 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 2:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 3:
0 Line Code Violations, 225 P-bit Coding Violation
6 C-bit Coding Violation, 86 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
5 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 4:
0 Line Code Violations, 146 P-bit Coding Violation
2 C-bit Coding Violation, 57 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
2 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 5:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Total Data (last 5 15 minute intervals):
0 Line Code Violations, 618 P-bit Coding Violation,
13 C-bit Coding Violation, 232 P-bit Err Secs,
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs,
0 Unavailable Secs, 0 Line Errored Secs,
12 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Controller from the router I'm not seeing any errors on:
T3 4/0 is up.
Applique type is Subrate T3
Description: APG - Lakehurst Controller
No alarms detected.
MDL transmission is disabled
FEAC code received: No code is being received
Framing is C-BIT Parity, Line Code is B3ZS, Clock Source is Internal
Data in current interval (290 seconds elapsed):
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 1:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 2:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 3:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 4:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 5:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 6:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Data in Interval 7:
0 Line Code Violations, 0 P-bit Coding Violation
0 C-bit Coding Violation, 0 P-bit Err Secs
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs
0 Unavailable Secs, 0 Line Errored Secs
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Total Data (last 7 15 minute intervals):
0 Line Code Violations, 0 P-bit Coding Violation,
0 C-bit Coding Violation, 0 P-bit Err Secs,
0 P-bit Severely Err Secs, 0 Severely Err Framing Secs,
0 Unavailable Secs, 0 Line Errored Secs,
0 C-bit Errored Secs, 0 C-bit Severely Errored Secs
Any help or opinions are always appreciated!
Thanks,
Jon
ā05-12-2010 09:47 AM
You have clock source internal on both sides, that is never correct.
If circuit provides clock, set both sides to external.
If it doesn't, set one side only to internal.
If SP can't reliably tell if clock is provided or not (happens often), try and compare the two modes above.
Note, circuit must be clean of any C-bit and P-bit errors to be acceptable.
Clock-breaking tests come after that.
ā05-12-2010 11:12 AM
Thanks!
I've got the clocking set to line on both sides, because the carrier cannot seem to tell me if they provide clock. The controller statistics look ok now. Also, just fyi, it seems they may have a bad cross connect, they're dispatching someone to take a look at the line.
Also, why would ping times be so slow on a ds3 link, they aren't rate limiting, it's a point to point line, but serial interface to serial interface, this is what i'm seeing... On my other DS3 lines it's much faster - 4/4/4.
ping
Protocol [ip]:
Target IP address: X.X.X.2
Repeat count [5]: 1000
Datagram size [100]: 1500
Timeout in seconds [2]:
Extended commands [n]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 1000, 1500-byte ICMP Echos to X.X.X.X, timeout is 2 seconds:
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 16/19/28 ms
ping
Protocol [ip]:
Target IP address: X.X.X.1
Repeat count [5]: 1000
Datagram size [100]: 1500
Timeout in seconds [2]:
Extended commands [n]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 1000, 1500-byte ICMP Echos to X.X.X.1, timeout is 2 seconds:
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 16/19/24 ms
Any thoughts?
Thanks!
ā05-12-2010 11:18 AM
How long is the circuit? SP can be using some funky transport like microwave, handing off to other carriers that influence delay.
Please remember to rate useful posts clicking on the stars below.
ā05-12-2010 11:27 AM
Of course I'll rate .. Thanks again for all of your help. The circuit goes from NJ, to Maryland, but it bounces around a few different places beforehand.
I guess I'll attribute the slow ping times to how many hops data is actually traveling through..
Jon
ā05-12-2010 01:31 PM
Well, in theory delay in TDM circuits is not influenced by number of hops or nodes, but by overall length only.
Not even a change in transport technologu, eg circuit emaulation would explaing such an high delay.
Anyway, it should not cause you any problem.
Thanks for the rating.
ā05-12-2010 03:06 PM
Jon,
If you are running a point-to-point circuit, then the carrier usually DOESN'T provide clocking. At this point set the host side to internal and the remote side to line to pull from the host and any clocking errors should go away.
Good rule of thumb for clocking: For Frame Relay and MPLS circuits the Telco usually provides the clocking on the line. For most to all point-to-point type circuits the Telco will not provide it and the Client Premise Equipment (CPE) will need to provide it.
I thought I would throw in my two cents!
Thanks,
Kimberly
ā05-12-2010 02:32 PM
Try having your carrier enable scrambling and do the same on your router for that circuit. If you are seeing these errors as bursting and not constant that might resolve it for you.
ā05-12-2010 02:41 PM
Try having your carrier enable scrambling and do the same on your router for that circuit.
That is a good suggestion. I don't remember if Telco does need to do anything, as it's transparent to them.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide