Channelized DS3 / DS1's strange after power outage

[bbuhlman]

Hi,

We had about a 2 second power outage yesterday and while our 7606 stayed up the batteries failed in the AT&T OC12 cabinet where our DS3's are. It took about 20 minutes for all the telco circuits to come back up but I'm still having trouble with 5 (out of 32)DS1 circuits connected to the 2 channelized DS3's in the 7606. They all exhibit the same limited connectivity, and only respond to every other ICMP packet. <wierd> I've had AT&T look at the circuits, they say they are fine. I can telnet across the links to the remote routers and while the connectivity is slow, I dont see anything wrong with the circuit ie: framing or line code violation. I've rebooted remote routers with no help. I've played with clocking and everything else I can think of short of rebooting the 7600. Thanks,

Bill

[wochanda]

Could you post the 'show controllers t3' output as well as 'show interface' for the T1's having the issue?

What kind of module are you terminating the DS3's on? Flexwan, OSM, SIP/SPA?

If you console into the linecard "ipc-con " do you see anything funky in the 'show log' around the time the problem started? What about in the MSFC log?

Is anything in common with these 5 T1 circuits? Are they going to the same general location? It could be that the power failure did some damage to a remote telco switch.

Also, if you can take the customer down for a while, try some loopback tests on the individual T1's. Start out with software loops on the DS3:

controller t3 x/y

t1 z loopback internal

and send 10000 1500-byte pings to yourself. If that works, stick a hard loopback plug in the remote smartjack and see if you can still ping clean to yourself.

I anticipate the software loop to be clean, and the remote loop to be dirty, but if i'm wrong it'll tell us a lot. If i'm right, we need to trace out the problem between those points. Call your telco and ask them to throw you a loop on an individual T1 at the closest point to you. If that is clean, have them keep moving the loop towards the end location until it goes dirty, and then tell them to fix it.