DS3 circuit issues

[mataalfredo]

We have a DS3 connection to one of our routers in other country. A week ago we started to see the following message in our logs:

5/20/2011 9:24 PM : ENTITY_ALARM-6-INFO  2162: May 20 21:22:54 cst: %ENTITY_ALARM-6-INFO: CLEAR MAJOR Se1/0 Receiver has remote alarm

5/20/2011 9:40 PM : ENTITY_ALARM-6-INFO  2163: .May 20 21:38:34 cst: %ENTITY_ALARM-6-INFO: ASSERT MAJOR Se1/0 Receiver has remote alarm

5/20/2011 9:58 PM : ENTITY_ALARM-6-INFO  2165: .May 20 21:57:22 cst: %ENTITY_ALARM-6-INFO: ASSERT MAJOR Se1/0 Receiver has remote alarm

5/20/2011 9:58 PM : ENTITY_ALARM-6-INFO  2166: .May 20 21:57:23 cst: %ENTITY_ALARM-6-INFO: CLEAR MAJOR Se1/0 Receiver has remote alarm

Our ISP made some test and said that the circuit is ok , they said that is not from their side.

I asked them about the framing they are using and they said is a clear channel and they dont use any framing standar. We have C-Bit configured in our serial interface.

We also already replace DS3 card with another one and the issue persist. Any idea how to solve this?

Regads,

[paolo bevilacqua]

It's a circuit issue, ask for a 48h BER test.

Telco can say what they want, but that's what it is.

[mataalfredo]

Thanks Paolo. The ISP made the test but everything looks fine from their side.

We changed the configuration on the interface and removed the command scramble on both sides. The alerts has been reduced dramatically and we received very few now. We may replace the card to make sure if the hardware is ok.

Do you think that the scramble command could be the issue or maybe just de card?

Regards,

[paolo bevilacqua · Accepted solution]

SPs will always say everything is OK until you prove otherwise.

Scramble helps with marginal circuits. You should always have scramble on. Check your controller and interface statistics for errors.

Yet, if controller flaps, that means faulty circuit, not faulty router.

[mataalfredo]

Thanks Paolo we are still having issues. Here is the output from controllers command.

ESV-WAN-01#sh controllers serial 1/0
M1T-T3+ pa: show controller:
PAS unit 0, subunit 0, f/w version 3-93, rev ID 0x2800001, version 3
idb = 0x65CF3424, ds = 0x65CF44CC, ssb=0x65CF4888
Clock mux=0x30, ucmd_ctrl=0x0, port_status=0x1
Serial config=0xA, line config=0x1B0202
maxdgram=4584, bufpool=128Kb, 256 particles

   rxLOS inactive, rxLOF inactive, rxAIS inactive
   txAIS inactive, rxRAI inactive, txRAI inactive
line state: up
cable type : T3  cable, received clockrate 44202795

base0 registers=0x3C800000, base1 registers=0x3C802000
mxt_ds=0x677B0830, rx ring entries=124, tx ring entries=254
statring=0xE332E60, statr shadow=0x65CF7068, stat_head=53
rxring=0xE332C20, rxr shadow=0x65CF6A34, rx_head=30
txring=0xE3334A0, txr shadow=0x65CF7C9C, tx_head=91, tx_tail=91, tx_count=0
throttled=0, enabled=0
halted=0, last halt reason=0
Microcode fatal errors=0
rx_no_eop_err=0, rx_no_stp_err=0, rx_no_eop_stp_err=0
rx_no_buf=0, rx_soft_overrun_err=300, dump_err= 0, bogus=0, mxt_flags=0x2C
tx_underrun_err=0, tx_soft_underrun_err=0, tx_limited=1(64)
tx_fullring=1757750, tx_started=799335547, mxt_flush_count=0
rx_int_count=1726747897, tx_int_count=894013546
   Framing is c-bit, Clock Source is Internal
   Bandwidth limit is 44210, DSU mode 0, Cable length is 10
   rx FEBE since last clear counter 477, since reset 230415
   Data in current interval (641 seconds elapsed):
     0 Line Code Violations, 886 P-bit Coding Violation
     881 C-bit Coding Violation
     440 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 78 Unavailable Secs
     440 Line Errored Secs, 440 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 1:
     0 Line Code Violations, 1047 P-bit Coding Violation
     1041 C-bit Coding Violation
     557 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 142 Unavailable Secs
     557 Line Errored Secs, 556 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 2:
     0 Line Code Violations, 1037 P-bit Coding Violation
     1032 C-bit Coding Violation
     532 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 130 Unavailable Secs
     532 Line Errored Secs, 531 C-bit Errored Secs, 0 C-bit Sev Err Secs
Data in Interval 3:
     0 Line Code Violations, 1325 P-bit Coding Violation
     1315 C-bit Coding Violation
     640 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 161 Unavailable Secs
     640 Line Errored Secs, 639 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 4:
     0 Line Code Violations, 845 P-bit Coding Violation
     843 C-bit Coding Violation
     460 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 91 Unavailable Secs
     460 Line Errored Secs, 460 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 5:
     0 Line Code Violations, 96 P-bit Coding Violation
     95 C-bit Coding Violation
     80 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 10 Unavailable Secs
     80 Line Errored Secs, 80 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 6:
     0 Line Code Violations, 150 P-bit Coding Violation
     150 C-bit Coding Violation
     137 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 20 Unavailable Secs
     137 Line Errored Secs, 137 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 7:
     0 Line Code Violations, 262 P-bit Coding Violation
     262 C-bit Coding Violation
     220 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 10 Unavailable Secs
     220 Line Errored Secs, 220 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 8:
     0 Line Code Violations, 319 P-bit Coding Violation
     317 C-bit Coding Violation
     251 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 30 Unavailable Secs
     251 Line Errored Secs, 249 C-bit Errored Secs, 0 C-bit Sev Err Secs
  Data in Interval 9:
     0 Line Code Violations, 329 P-bit Coding Violation
     328 C-bit Coding Violation
     274 P-bit Err Secs, 0 P-bit Sev Err Secs
     0 Sev Err Framing Secs, 30 Unavailable Secs
     274 Line Errored Secs, 273 C-bit Errored Secs, 0 C-bit Sev Err Secs

We are going to replace the card just to make sure that is not from our side.I really believe is from ISP side but they dont want to admit and said in their logs and all the test they made everything was fine.

Regards,

[paolo bevilacqua]

There is no need to replace the card.

As mentioned before, ask for a BER test made on your premises with all your equipment removed from circuit, meaning, harware loopback one side, BER test device (telco-provided) on the other site.

If they say they do other tests differently from the above, refuse.

Be sure you're present on site and can observe results with your own eyes.

If it is an international circuit thing will be a lot more difficult, as each telco will start blaming each other, and communication will be very slow and complicated.

This test is the same as done when a circuit is delivered, aka customer acceptance test.

Thanks for the nice rating and good luck.

[Calin C. · Accepted solution]

In addition to what Paolo said:

2. %ENTITY_ALARM-6-INFO: CLEAR MAJOR Se1/0 Physical Port Link Down # This means that the interface set a Major alarm, due to having  received an indication of a problem at the remote end. From the time  stamps, this created brief flaps. This is an information message, since  the problem is elsewhere. You need not need to do anything at this end.

Recommended Action: Check the remote peer for any errors / warnings.

Related documents- No specific documents apply to this error message.

So, whatever the problem is on that circuit, it is caused by the equipment on the provider edge.

Calin