Verizon PRI issue

ALIAOF_ · ‎01-03-2014

So unfortunately we have a local PRI from Verizon at one of our locations. This site has been a nightmare with multiple issues. Each time as usual Verizon has blamed everything on our equipment and it ended up being their issue. Any ways so the latest issue is this:

- PRI circuit has lots of errors but "show isdn status" is showing it up (Layer 1, 2)

Layer 1 Status:

ACTIVE

Layer 2 Status:

TEI = 0, Ces = 1, SAPI = 0, State = MULTIPLE_FRAME_ESTABLISHED

- I do see these messages when running a debug here and there:

ISDN Se0/3/1:23 Q921: L2_EstablishDataLink: sending SABME

- All outbound calling works

- Any inbound calling does not work. I have tried calling the DID's from Cell, Landline and we get either a fastbusy tone, or Non working number message and running a "debug isdn q921 and q931" doesn't even show the call hitting our router.

- And as expected first thing Verizon said it is our equipment and our equipment is not accepting the call

After talking to the manager of this tech he said that they inbound calling is not working because there are lot of errors on the T1. I told him regardless I should still atleast see the calls hitting the router because the PRI is showing up. And he insisted that there is an issue with either our equipment and the demarc extension. Any thoughts, notions, suggestions on why the inbound calling not working?

Also these messages were showing up in the logs but its been 20 mins so far haven't seen them again:

Jan 3 09:28:55.557: %CONTROLLER-5-UPDOWN: Controller T1 0/3/1, changed state to up

Jan 3 09:28:55.565: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 10 to priority 1

Jan 3 09:29:06.557: %CONTROLLER-5-UPDOWN: Controller T1 0/3/1, changed state to down (LOF detected)

Jan 3 09:29:06.557: %MARS_NETCLK-3-CLK_TRANS: Network clock source transitioned from priority 1 to priority 10

Jan 3 09:29:07.557: %CONTROLLER-5-UPDOWN: Controller T1 0/3/1, changed state to up

Manish Gogna · ‎01-03-2014

Hi Mohammad,

There is very little that could be done if the call does not even reach the gateway. In these kind of scenarios it is a good idea to open a TAC case and have the TAC engineer get on a conference call with you and the service provider to sort things out. Any other suggestions are welcome.

HTH

Manish

Aman Soi · ‎01-03-2014

Hi Ali,

you can also try running the loopback local line command under E1 controller and ask the Telco to check the media at their end.

what is the router model and E1 card u are using? share show version and sh diag from router.

regds,

aman

ALIAOF_ · ‎01-03-2014

Thank you for the reply here is the info:

Router = Cisco 2911

Cisco IOS Software, C2900 Software (C2900-UNIVERSALK9-M), Version 15.2(4)M3, RELEASE SOFTWARE (fc2)

Card = VWIC3-2MFT-T1/E1 (We have T1 not E1)

WIC Slot 3:

VWIC3-2MFT-T1/E1 - 2-Port RJ-48 Multiflex Trunk - T1/E1

Hardware Revision : 1.0

Top Assy. Part Number : 800-34658-01

Part Number : 73-13420-01

Board Revision : B0

Deviation Number : 0

Fab Version : 05

PCB Serial Number : xxxxxxxxxxxx

Version Identifier : V01

Product (FRU) Number : VWIC3-2MFT-T1/E1

CLEI Code : COUIA7NCAA

RMA Test History : 00

RMA Number : 0-0-0-0

RMA History : 00

EEPROM format version 4

EEPROM contents (hex):

0x00: 04 FF 40 06 EF 41 01 00 C0 46 03 20 00 87 62 01

0x10: 82 49 34 6C 01 42 42 30 88 00 00 00 00 02 05 C1

0x20: 8B 46 4F 43 31 37 32 30 35 55 53 56 89 56 30 31

0x30: 20 CB 90 56 57 49 43 33 2D 32 4D 46 54 2D 54 31

0x40: 2F 45 31 C6 8A 43 4F 55 49 41 37 4E 43 41 41 03

0x50: 00 81 00 00 00 00 04 00 D9 03 40 C1 CB FF FF FF

0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

Suresh Subramanian · ‎01-03-2014

Check this thread if it helps please

https://supportforums.cisco.com/message/3996514#3996514

//Suresh Please rate all the useful posts.

ALIAOF_ · ‎01-03-2014

Thank you good post, however my router already has the IOS with the fix. Still seeing errors on it. This still does not explain why the inbound calls aren't working though. Verizon is seriously a worst provider of any services period and so is their tech support.

Aman Soi · ‎01-03-2014

Hi Ali,

As Mr. Suresh has already shared the discussion,just have a look .May ne your problem gets resolved.

this could be a bug.

T1 interface behavior inconsistent across WIC Modules

CSCua50697

Description

Symptoms: After unplugging and reconnecting a T1 cable, the T1 controller
remains down or report continuous errors. After a router reload, the T1
controller remains up until the cable is disconnected again.

Conditions: This symptom affects only the following cards: HWIC-xCE1T1-PRI,
NM-8CE1T1-PRI, VWIC3-xMFT-T1/E1, and GRWIC-xCE1T1-PRI.
Also, the T1 signal must be somewhat out-of-specification according to T1.403
standards.

Workaround 1: Reload the router with the T1 cable plugged in.

Workaround 2:
Step 1) Upgrade to a fixed-in Cisco IOS version.
Step 2) Issue the following commands (hidden, so tab complete will not work):

enable
config t
controller ! ( example: controller t1 0/0/0 )
hwic_t1e1 equalize

Step 3) Shut/no shut the T1 controller, or reload the router to allow the CLI to
take effect.

regds,

aman

ALIAOF_ · ‎01-06-2014

Like I mentioned that the router has the IOS in which this bug was fixed. Verizon called me and while I was on the phone with them we were doing some testing. They told me that the calls are just sitting there and our equipment is not accepting them. Any ways after I got done talking to them I decided to reset our gateway and controller again.

As soon as I did that everything started working. Going to keep an eye on it. So far there is no root cause of why this happened. I did end up adding this command as well by the way just in case hwic_t1e1 equalize. I am thinking if this might be the reason but we'll see.