Strange issue moving a T-1 from 2600 to 2911 router

Andrew Cink · ‎07-24-2012

I'm kind of at a loss. We have an old Cisco 2600 series router, running a single T-1 line, AT&T is the carrier. Very standard stuff.

The T-1 works perfectly on the old router. Cisco 2600 with a WIC-1DSU-T1 card. We have a brand new 2911 router with a VWIC3-1MFT-T1/E1 card in it. The circuit runs 100% clean and trouble free on the old router. When I plug it into the new router, it shows up/up but has a massive number of errors on the line, and the circuit is not usable. There are no timing slips or anything like that.

I have another site using the same router and card with the same configuration on that T-1 and it works perfectly. Zero errors. This site (in a different city) is totally slammed with errors. We already replaced the T-1 card and still have the exact same issue.

I've actually been doing this a long time, and I've tried all of the usual things, adjusting line build out, changing clock source, checking and re-checking configs. Nothing seems to reduce the error rate.

I personally think it has something to do with the Smartjack. Like maybe their LBO is too high or too low, and the old router card was more/less sensitive than the new one?

Does anyone have any insight? I mean no offense when I say, please don't look for a configuration issue here, this is a layer 1 issue and is not defective hardware or defective configuration. I am looking for T-1 insights or bugs/issues.

Thank you in advance if you can help!

Andrew

Georg Pauwen · ‎07-25-2012

Andrew,

this sounds like the problem is at the AT&T side, what do they say ? Does the circuit check out clean to the CSU ? Since you checked and double-checked everything on your side (including the framing on the controller, I assume), I would ask AT&T to perform (intrusive if necessary) testing.

I know this is probably not the answer you are looking for, but having worked both for providers and large customers, in my experience, the Telco is usually at fault.

Good luck,

Georg

jhindmarsh · ‎11-13-2013

Andrew,

I was wondering if you made any progress with this issue? We just replaced 23 routers at sites for a customer, and we only have issues with sites having multiple T1 circuits. This includes both MPLS as well as PRI circuits. We confirmed the circuits were not at fault using an older 2800 series router with older VWIC-2MFT cards. We also proved this to not be a provider issue by replacing the 2 port VWIC3 cards with older VWIC2 MFT cards for a voice circuit.

This is most frustrating as the provider is testing clean with their test equipment, and Cisco is telling us it is not a problem with the new VWIC3 cards when we can use old hardware and not have this issue.

I hope Cisco is trolling these forums and can provide some insight.

Thanks
Jesse

Andrew Cink · ‎11-14-2013

This is the worst kind of answer to give, but someone else wound up looking at it, they say they replaced the T1 card and it worked. Not sure if they used an older model of card or what. I still think it has to be an issue with the signal on the line being too strong or too weak. I would call the carrier and ask to do a vendor meet and check the smart jack/NIU settings. That would have been my next step, anyways.

Good luck!

Andrew

Andrew Cink · ‎11-14-2013

Ugh, one other thing to mention. I have seen issues in the past with multilink t1s when the circuits ride different spans and have different clock sources as a result. This has been a problem at my current job which the carrier has resolved by moving all t1s to the same t3/span.

Here is an explanation of the issue....

http://www.raltron.com/products/pdfspecs/sync_an02-stratumleveldefined.pdf

Andrew

Andrew Cink · ‎11-14-2013

Double ugh, that was not the right link... Try these for a better explanation. I think this applies equally to Cisco even though Adtran is a competitor of theirs...

https://supportforums.adtran.com/servlet/JiveServlet/previewBody/1401-102-1-1428/ATLAS%20Timing%20White%20Paper.pdf

https://supportforums.adtran.com/thread/2939

Long story short, most equipment can only sync to one clock. If you have multiple devices synced to different clocks you will get slips which can cause problems...

jhindmarsh · ‎11-14-2013

We looked into the clocking and LBO parameters but that did not resolve the issue. What is more frustrating is that a version 1 or version 2 MFT card works as expected. We have also had the issue with PRI circuits with the VWIC3-MFT cards. In every case, switching back to a version 2 card works.

In one particular case we tried to take advantage of the new 4 port MFT cards for density, only to have to populate the router with multiple version 2 cards.

I have a pile of version 3 cards that are useless. TAC has been less that supportive saying it is the provider. However, I am working with MULTIPLE providers, all seeing the same issue.

I challenge Cisco TAC to prove it! Show me a diagnostic report that only Cisco TAC can run. Our in house CCIE is working with TAC and he can't even escalate the case to anyone higher.

When all this is figured out, who do I send the bill to?

Thanks
Jesse

Sent from Cisco Technical Support iPhone App

Verizon Verizon · ‎12-18-2013

All,

This appears to be very related to an issue I have. I can plug in any single T1 and bring them up but when I plug in all 3, only the third T1 will come up with no errors. Router presently has VWIC3-2MFT-T1/E1s I am asking Cisco to ship a two port version of a HWIC-1DSU-T1 for my 2951 router.