DSL Disconnection With ATM interface

Elie Bassil · ‎07-14-2011

Hello,

I have a scenario where multiple branches are connected to Head Office via DSL links. The branches are located in differenct exchanges, everyone once in a while ALL branches go down at the exact same time except 3 branches and those 3 branches don't have ATM interface on the router, rather they have external DSL modem, all other branches that face the disconnection (for couple of minutes) have either HWIC-1ADSL-M or HWIC-2SHDSL and the DSL line is directly plugged into the card. Here is a sample configuration of the routers facing the disruption:

interface ATM0/0/0

no ip address

no ip redirects

no ip unreachables

no ip proxy-arp

no atm ilmi-keepalive

!

interface ATM0/0/0.1 point-to-point

description ADSL

mtu 1500

bandwidth 1000

ip address 192.168.241.17 255.255.255.0

no ip redirects

no ip unreachables

no ip proxy-arp

atm route-bridged ip

crypto map DSL

pvc 0/35

encapsulation aal5snap

!

I tried so many solutions like adding "operating-mode auto" command under the subinterface, IOS upgrade, changed dynamic routing protocol from EIGRP to OSPF... Nothing worked, on the logs of the router it just shows "EIGRP neighbor down", at the time of the problem I can't ping any of the affected branches on their WAN ATM interface although it stays physically "up up".

Could the problem be a missing command that I need to add under the subinterface? Something maybe related to clocking or timeout or..?

Note that hte problem occurs randomly, every few days, sometimes every 10 days, and all branches with ATM interface go down at the exact same time.

E.B:.

paolo bevilacqua · ‎07-14-2011

Note that hte problem occurs randomly, every few days, sometimes every 10 days, and all branches with ATM interface go down at the exact same time.

How you can tell they go down at the same time?

Any, send "show dsl interface", taken on the troubled branches only.

Elie Bassil · ‎07-14-2011

How you can tell they go down at the same time?

I have a monitoring tool that monitors all the branches, it goes all red when the problem occurs except for the 3 branches where there is an external DSL modem.

Any, send "show dsl interface", taken on the troubled branches only.

command not supported, find "sh interface" output if that helps

ATM0/0/0 is up, line protocol is up

Hardware is DSLSAR, address is c84c.75fd.1fa3 (bia c84c.75fd.1fa3)

MTU 4470 bytes, sub MTU 4470, BW 2304 Kbit/sec, DLY 110 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ATM, loopback not set

Keepalive not supported

Encapsulation(s): AAL5

8 maximum active VCs, 256 VCs per VP, 1 current VCCs

VC Auto Creation Disabled.

VC idle disconnect time: 300 seconds

Last input 00:00:00, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: Per VC Queueing

5 minute input rate 7000 bits/sec, 5 packets/sec

5 minute output rate 2000 bits/sec, 1 packets/sec

1410023 packets input, 197277896 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 1509 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

103046 packets output, 19325143 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 unknown protocol drops

0 output buffer failures, 0 output buffers swapped out

ATM0/0/0.1 is up, line protocol is up

Hardware is DSLSAR, address is c84c.75fd.1fa3 (bia c84c.75fd.1fa3)

Internet address is 192.168.241.12/24

MTU 1500 bytes, BW 2000 Kbit/sec, DLY 110 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ATM

Keepalive not supported

1410023 packets input, 197277896 bytes

103046 packets output, 18912959 bytes

0 OAM cells input, 0 OAM cells output

AAL5 CRC errors : 0

AAL5 SAR Timeouts : 0

AAL5 Oversized SDUs : 0

ARP type: ARPA, ARP Timeout 04:00:00

Last clearing of "show interface" counters never

10x for looking into this dazzling issue

E.B:.

paolo bevilacqua · ‎07-14-2011

You should have "show dsl" commands because you have an ADSL interface. It is crucial to look at ADSL line and firmware information. Please check again. Alse send show ver, show diag.

Elie Bassil · ‎07-14-2011

yes you are right:

------------------ show dsl interface ------------------

ATM0/0/0

Alcatel 20190 chipset information

Line not activated: displaying cached data from last activation

ATU-R (DS) ATU-C (US)

Modem Status: Showtime (DMTDSL_STOP)

DSL Mode: ANSI T1.413

ANSI Issue/rev: 2/1 2/1

Vendor ID: 0x003A 0x0054

Chip ID: C196 (3) capability-enabled

DFE BOM: DFE3.0 Annex M (3)

Capacity Used: 62% 28%

Noise Margin: 12.5 dB 28.0 dB

Output Power: 15.0 dBm 11.5 dBm

Attenuation: 66.0 dB 31.5 dB

FEC ES Errors: 0 0

ES Errors: 4582 310

SES Errors: 0 0

LOSES Errors: 0 0

UES Errors: 0 0

Defect Status: None None

Last Fail Code: None

Watchdog Counter: 0xD5

Watchdog Resets: 0

Selftest Result: 0x00

Subfunction: 0x00

Interrupts: 28939 (0 spurious)

PHY Access Err: 0

Activations: 28

LED Status: OFF

LED On Time: 0

LED Off Time: 0

Init FW: init_AMR-4.0.015_no_bist.bin

Operation FW: AMR-4.0.015.bin

FW Source: embedded

FW Version: 4.0.15

Interleave Fast Interleave Fast

Speed (kbps): 512 0 128 0

Cells: 6358834 0 27079623 0

Reed-Solomon EC: 47318 0 341 343

CRC Errors: 970 0 348 327

Header Errors: 272 0 325 333

Total BER: 0E-0 0E-0

Leakage Average BER: 12288E-117 0E-0

ATU-R (DS) ATU-C (US)

Bitswap: enabled enabled

LOM Monitoring : Disabled

DMT Bits Per Bin

000: 0 0 0 0 0 0 0 0 2 3 4 4 4 4 4 4

010: 0 3 3 3 3 3 3 2 2 2 2 2 2 2 2 0

020: 0 2 2 2 2 2 2 3 3 4 2 3 4 3 3 3

030: 4 3 3 3 2 2 2 0 0 0 2 3 3 4 4 4

040: 0 4 2 0 2 2 2 3 5 5 4 3 4 4 4 4

050: 2 4 4 2 0 0 4 4 4 3 3 3 4 3 3 3

060: 4 3 3 0 0 0 0 2 2 0 0 0 0 0 0 0

070: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

080: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

090: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0A0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0B0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0C0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0D0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0E0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0F0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

DSL: Training log buffer capability is not enabled

------------------ show version ------------------

Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.0(1)M1, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Compiled Wed 02-Dec-09 15:23 by prod_rel_team

ROM: System Bootstrap, Version 15.0(1r)M1, RELEASE SOFTWARE (fc1)

Bekaata uptime is 14 weeks, 3 hours, 13 minutes

System returned to ROM by power-on

System image file is "flash0:c2900-universalk9-mz.SPA.150-1.M1.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

Cisco CISCO2911/K9 (revision 1.0) with 475136K/49152K bytes of memory.

Processor board ID FHK1352F36T

3 Gigabit Ethernet interfaces

2 Low-speed serial(sync/async) interfaces

1 ATM interface

1 Virtual Private Network (VPN) Module

2 Voice FXO interfaces

2 Voice FXS interfaces

DRAM configuration is 64 bits wide with parity enabled.

255K bytes of non-volatile configuration memory.

254464K bytes of ATA System CompactFlash 0 (Read/Write)

License Info:

License UDI:

-------------------------------------------------

Device# PID SN

-------------------------------------------------

*0 CISCO2911/K9

Technology Package License Information for Module:'c2900'

----------------------------------------------------------------

Technology Technology-package Technology-package

Current Type Next reboot

-----------------------------------------------------------------

ipbase ipbasek9 Permanent ipbasek9

security securityk9 Permanent securityk9

uc uck9 Permanent uck9

data None None None

Configuration register is 0x2102

------------------ show diag ------------------

Slot 0:

C2911 Mother board 3GE, integrated VPN and 4W Port adapter, 10 ports

Port adapter is analyzed

Port adapter insertion time 14w0d ago

EEPROM contents at hardware discovery:

PCB Serial Number :

Hardware Revision : 1.0

Part Number : 73-11833-03

Top Assy. Part Number : 800-30797-01

Board Revision : B0

Deviation Number : 0

Fab Version : 03

Product (FRU) Number : CISCO2911/K9

Version Identifier : V01

CLEI Code : COMGM00ARA

Processor type : C1

Chassis Serial Number :

Chassis MAC Address : 8843.e116.ef40

MAC Address block size : 80

Manufacturing Test Data : 00 00 00 00 00 00 00 00

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF C1 8B 46 4F 43 31 33 34 39 35 31 50 56 40

0x10: 06 18 41 01 00 82 49 2E 39 03 C0 46 03 20 00 78

0x20: 4D 01 42 42 30 88 00 00 00 00 02 03 CB 8C 43 49

0x30: 53 43 4F 32 39 31 31 2F 4B 39 89 56 30 31 20 D9

0x40: 04 40 C1 CB C2 C6 8A 43 4F 4D 47 4D 30 30 41 52

0x50: 41 09 C1 C2 8B 46 48 4B 31 33 35 32 46 33 36 54

0x60: C3 06 88 43 E1 16 EF 40 43 00 50 C4 08 00 00 00

0x70: 00 00 00 00 00 F3 00 03 40 01 25 FF FF FF FF FF

0x80: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x90: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xA0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xB0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xC0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xD0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xE0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xF0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

Internal power Supply information

Top Assy. Part Number : 341-0235-03

Deviation Number : 0

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Version Identifier : V01

Product (FRU) Number : PWR-2911-AC

CLEI Code : TYCOCONNEC

Board Revision : A0

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 05 E5 DF 45 01 55 00 EB 03 88 00 00 00

0x10: 00 C1 8B 41 5A 53 31 33 34 36 30 35 36 35 03 00

0x20: 81 00 00 00 00 04 00 89 56 30 31 20 CB 8B 50 57

0x30: 52 2D 32 39 31 31 2D 41 43 C6 8A 54 59 43 4F 43

0x40: 4F 4E 4E 45 43 F3 00 59 41 00 BE 42 00 05 F8 00

0x50: 50 01 F3 17 32 02 EE 1A 9F 03 E6 1B B4 04 E3 1C

0x60: 99 05 DC 1D 7B 06 D6 1E 3C 07 D3 1E E7 08 C9 1F

0x70: 5A 09 C6 1F D5 0A BC 20 20 0B B9 20 74 0D 92 20

0x80: A0 0F 8C 21 28 11 85 21 5F 13 7F 21 9D 17 71 21

0x90: F3 1B 57 22 08 1F 40 22 18 23 37 22 27 27 10 22

0xA0: 09 42 41 30 FF FF FF FF FF FF FF FF FF FF FF FF

0xB0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xC0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xD0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xE0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0xF0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PVDM Slot 0:

16-channel (G.711) Voice/Fax PVDM3 DSP DIMM PVDM daughter card

Hardware Revision : 1.0

Part Number : 73-11576-03

Board Revision : B0

Deviation Number : 0

Fab Version : 03

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Processor type : 00

Product (FRU) Number : PVDM3-16

Version Identifier : V01

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 05 D8 41 01 00 82 49 2D 38 03 42 42 30

0x10: 88 00 00 00 00 02 03 C1 8B 46 4F 43 31 33 35 31

0x20: 33 5A 48 35 03 00 81 00 00 00 00 04 00 09 00 CB

0x30: 8F 50 56 44 4D 33 2D 31 36 20 20 20 20 20 20 20

0x40: 89 56 30 31 20 D9 02 40 C1 FF FF FF FF FF FF FF

0x50: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

WIC Slot 0:

ADSL over POTS Annex M

Hardware Revision : 3.0

Top Assy. Part Number : 800-28201-03

Part Number : 73-10885-03

Board Revision : B0

Deviation Number : 0

Fab Version : 07

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Product (FRU) Number : HWIC-1ADSL-M

Version Identifier : V03

CLEI Code : COUIAFDCAB

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 05 66 41 03 00 C0 46 03 20 00 6E 29 03

0x10: 82 49 2A 85 03 42 42 30 88 00 00 00 00 02 07 C1

0x20: 8B 46 4F 43 31 34 32 33 32 4B 32 54 03 00 81 00

0x30: 00 00 00 04 00 CB 92 48 57 49 43 2D 31 41 44 53

0x40: 4C 2D 4D 20 20 20 20 20 20 89 56 30 33 20 D9 02

0x50: 40 C1 C6 8A 43 4F 55 49 41 46 44 43 41 42 FF FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

EM Slot 0:

ADSL over POTS Annex M non-removable daughtercard

Hardware Revision : 3.0

Part Number : 73-10886-03

Board Revision : C0

Deviation Number : 0

Fab Version : 01

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Fab Part Number : 28-8925-01

Manufacturing Test Data : 00 00 00 00 00 00 00 00

Field Diagnostics Data : 00 00 00 00 00 00 00 00

Connector Type : 01

Version Identifier : V03

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 05 67 41 03 00 82 49 2A 86 03 42 43 30

0x10: 88 00 00 00 00 02 01 C1 8B 46 4F 43 31 34 32 32

0x20: 34 53 51 4E 03 00 81 00 00 00 00 04 00 85 1C 22

0x30: DD 01 C4 08 00 00 00 00 00 00 00 00 C5 08 00 00

0x40: 00 00 00 00 00 00 05 01 89 56 30 33 20 FF FF FF

0x50: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

WIC Slot 1:

HWIC Serial 2A/S

Hardware Revision : 1.0

Top Assy. Part Number : 800-28900-02

Board Revision : A0

Deviation Number : 0

Fab Version : 01

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Fab Part Number : 28-8141-01

Product (FRU) Number : HWIC-2A/S

Version Identifier : V02

CLEI Code : COUIA1PCAA

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 05 8E 41 01 00 C0 46 03 20 00 70 E4 02

0x10: 42 41 30 88 00 00 00 00 02 01 C1 8B 46 4F 43 31

0x20: 33 35 30 32 36 35 52 03 00 81 00 00 00 00 04 00

0x30: 85 1C 1F CD 01 CB 89 48 57 49 43 2D 32 41 2F 53

0x40: 89 56 30 32 20 D9 02 40 C1 C6 8A 43 4F 55 49 41

0x50: 31 50 43 41 41 FF FF FF FF FF FF FF FF FF FF FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

WIC Slot 2:

3rd generation - FXS DID Voice daughter card (2 port)

Hardware Revision : 1.0

Top Assy. Part Number : 800-30501-02

Board Revision : A0

Deviation Number : 0

Fab Version : 07

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Product (FRU) Number : VIC3-2FXS/DID

Version Identifier : V02

CLEI Code : IP9IAB3CAB

Longitudinal Calibration : 01 10 10 94 9E 10 10 9F

8A

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 06 21 41 01 00 C0 46 03 20 00 77 25 02

0x10: 42 41 30 88 00 00 00 00 02 07 C1 8B 46 4F 43 31

0x20: 33 34 38 32 35 54 44 03 00 81 00 00 00 00 04 00

0x30: CB 8D 56 49 43 33 2D 32 46 58 53 2F 44 49 44 89

0x40: 56 30 32 20 D9 02 40 C1 C6 8A 49 50 39 49 41 42

0x50: 33 43 41 42 D3 09 01 10 10 94 9E 10 10 9F 8A FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

WIC Slot 3:

2nd generation - FXO Voice daughter card (2 port)

Hardware Revision : 5.0

Top Assy. Part Number : 800-21597-03

Board Revision : B0

Deviation Number : 0-0

Fab Version : 04

PCB Serial Number :

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

Product (FRU) Number : VIC2-2FXO

Version Identifier : V02

CLEI Code : CNUIARTAAB

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 00 4D 41 05 00 C0 46 03 20 00 54 5D 03

0x10: 42 42 30 80 00 00 00 00 02 04 C1 8B 46 4F 43 31

0x20: 33 34 34 35 39 5A 38 03 00 81 00 00 00 00 04 00

0x30: CB 89 56 49 43 32 2D 32 46 58 4F 89 56 30 32 20

0x40: C6 8A 43 4E 55 49 41 52 54 41 41 42 FF FF FF FF

0x50: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

paolo bevilacqua · ‎07-14-2011

The ADSL line seems troubled, a lot of activations, some errors, very low speed despite attentuation and noise margine within acceptable limits.

And DSL mode should be ITU T not ANSI.

The most important thing is that you try ADSL firmware 4.0.17, that requires a support contract to be downloaded.

If you cannot easily get the support contract, try asking TAC is they can open the case on the basis that you have a new product malfuctioning. TAC may also have newer, unpublish ADSL firmware to address your problem.

Also you can try rehauling all internal wiring, ADSL filters, etc. Here a link

http://www.kitz.co.uk/adsl/lowSNR.htm

What is puzzling is that you see branches connected to different COs go down at the same time, you would have to look at logs with timestamps to verify that. It might be something flapping in ISP network between you and the nodes,

Thanks for the nice rating and good luck!