PRI (PRIMARY RATE INTERFACE) Troubleshooting
Before I start with main content on how to troubleshoot PRI, let’s go through a small introduction of PRI.
What is PRI?
A PRI – or Primary Rate Interface – is an end-to-end, digital telecommunications connection that allows for 23/30(depending on T1/E1) concurrent transmissions of voice, data, or video traffic between the network and the user. The PRI line, or circuit, is a physical piece of equipment.
Features of a PRI line
- PRI technology has been around since the 1980s
- A PRI line is made up of two pairs of copper wires connecting the service provider to the customer.
- A PRI is technically one line, but it can simultaneously transmit 23 (or 30) separate communications (voice, data, or video.) This means up to 23 (or 30) people in your company can be on the phone (or sending an IM or having a video chat) at the same time.
- Each of a PRI’s channels (both T1 and E1) has a 64 kbps capacity for data transmissions.
The PRI circuit – the actual cable that physically connects your telecom system – is of two types: T1 and E1. Please see the differences below:
|
E1 |
T1 |
|
Used in Europe, Asia and Australia |
Used in United State, Canada and Japan |
|
Provides 64 Kbps for data transmission |
Provides 64 Kbps for data transmission |
|
Transmit and receive rate of 2.048 Mbps |
Transmit and receive rate of 1.544 Mbps |
|
Physically contains two pairs of copper wires |
Physically contains two pairs of copper wires |
|
32 total channels |
24 total channels |
|
30 channels used for voice, data, video |
23 channels used for voice, data, video |
|
1 channel (1st channel) is mainly used for clocking and synchronization and 1 channel (16th channel) used for signalling. |
1 channel used for signalling. |
Now, lets start the TROUBLEshooting part.
The first thing to examine when troubleshooting a T1 or E1 connection is the physical layer statistics. Use the command show controllers [t1 | e1] to show all the T1 or E1 interfaces on the gateway. You can also specify a particular port at the end of the command to view the statistics for a single port.
Router#show controllers t1
T1 1/0 is down.
Applique type is Channelized T1
Cablelength is long gain36 0db
Transmitter is sending remote alarm.
Receiver has loss of signal.
alarm-trigger is not set
Version info Firmware: 20010315, FPGA: 15
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
Data in current interval (246 seconds elapsed):
0 Line Code Violations, 0 Path Code Violations
0 Slip Secs, 246 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 246 Unavail Secs
Total Data (last 24 hours)
0 Line Code Violations, 0 Path Code Violations,
0 Slip Secs, 86400 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins,
0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 86400 Unavail Secs
You can see that T1 1/0 is down. Note that T1 1/0 indicates that Receiver has loss of signal, so it is sending a remote alarm. This is called a red alarm. A red alarm informs the remote side that the local interface is not receiving any framing.
ISDN PRI signaling is probably the easiest of the TDM protocols to troubleshoot. This is mainly because ISDN is a message-based protocol and therefore provides a great deal of information regarding error conditions.
The most commonly used commands are “show isdn status” and “show controllers T1 or E1”. The command “show isdn status” shows you the status of each ISDN layers, please see below:
Router#show isdn status
ISDN Serial1:23 interface
dsl 1, interface ISDN Switchtype = primary-5ess
Layer 1 Status:
ACTIVE
Layer 2 Status:
TEI = 0, Ces = 1, SAPI = 0, State = TEI_ASSIGNED>>>AWAITING ESTABLISHMENT>>>>MFE
Layer 3 Status:
0 Active Layer 3 Call(s)
Activated dsl 1 CCBs = 0
The Free Channel Mask: 0x807FFFFF
Total Allocated ISDN CCBs = 5
You can see that the Layer 1 is ACTIVE on this PRI. If Layer 1 is not ACTIVE (could be in SHUTDOWN/DEACTIVATED state), then please check the output of “show controllers” and if it is shows as DOWN, the problem is with the physical layer.
If Layer 2 is stable (Multiple_Frame_Established), the router and switch must begin to synchronize with each other. The Set Asynchronous Balanced Mode Extended (SABME) message appears on the display. This message indicates that Layer 2 tries to initialize with the other side. Either side can send the message and try to initialize with the other side. If the router receives the SABME message, it must send back an Unnumbered Acknowledge frame (UAf). The router then changes the Layer 2 status to MULTIPLE_FRAME_ESTABLISHED from TEI_ASSIGNED. Here is an example:
Sep 15 22:45:54.061: ISDN Se1/1:23: TX -> SABMEp c/r = 0 sapi = 0 tei = 0
Sep 15 22:45:54.065: ISDN Se1/1:23: RX <- UAf c/r = 0 sapi = 0 tei = 0
In case D-channel is not enabled, you would see the ISDN Layer 2 as TEI_ASSIGNED. You will see SABME messages being sent but not acknowledged with a UAf, check the equipment attached to the gateway to ensure that the D-channel is enabled. This might involve calling the service provider.
Sep 14 21:54:57.090: ISDN Se0/0/0:23 Q921: Net TX -> SABMEp sapi=0 tei=0
Sep 14 21:54:57.590: ISDN Se0/2/0:23 Q921: User TX -> SABMEp sapi=0 tei=0
Sep 14 21:54:58.090: ISDN Se0/0/0:23 Q921: Net TX -> SABMEp sapi=0 tei=0
At times, the D-channel does not come up correctly and stays in TEI_ASSIGNED state, or Layer 2 bounces up and down. This could be caused by one way transmission or missed keepalive packets. When either side misses four consecutive keepalives, the respective side tries to initialize the Layer 2 link again. To isolate the problem, we can go for loopback test. Loopback tests can be both software and hardware. Hardware loopback tests are more reliable (which I too recommend) and it can be done using a loopback plug. Please refer to below document for loopback testing.
https://www.cisco.com/c/en/us/support/docs/voice/device-signaling/116492-trouble-t1e1-00.html
If the switch receives and recognizes the UAf, both devices synchronize, and periodic keepalives are exchanged between the router and the ISDN switch. These messages are in the form of Receiver Ready (RRf and RRp). These keepalives are seen 10 seconds apart, and ensure that both sides are able to communicate with each other. For example:
*Apr 12 05:19:56.183: ISDN Se0:23: RX <- RRp sapi=0 tei=0 nr=18
*Apr 12 05:19:56.183: ISDN Se0:23: TX -> RRf sapi=0 tei=0 nr=18*
Apr 12 05:20:06.247: ISDN Se0:23: RX <- RRp sapi=0 tei=0 nr=18*
Apr 12 05:20:06.247: ISDN Se0:23: TX -> RRf sapi=0 tei=0 nr=18*
Apr 12 05:20:16.311: ISDN Se0:23: RX <- RRp sapi=0 tei=0 nr=18*
Apr 12 05:20:16.311: ISDN Se0:23: TX -> RRf sapi=0 tei=0 nr=18
Now, as our D-channel is up, debugging Q.931 messages is easy.
Look at the ISDN call flow:
After enabling debug isdn q931, try to reproduce the issue and collect logs. So, if you see below, it’s an incoming ISDN SETUP message (see the direction of ARROW in front of SETUP). So, it’s an incoming call.
Please make a note of calling/called number with time when issue occured. After collecting debugs, best way to find the call is by calling or called number. And then to track an ISDN call leg, highlight the last three characters of callref value. These last three characters remains same per call.
Please find the sample debug analyis below:
//INCOMING ISDN SETUP RECEIVED BY THE GW FROM THE PROVIDER//
105990: Jan 31 20:12:46.873: ISDN Se0/0/0:15 Q931: RX <- SETUP pd = 8 callref = 0x00BA
Sending Complete
Bearer Capability i = 0x9090A3
Standard = CCITT
Transfer Capability = 3.1kHz Audio
Transfer Mode = Circuit
Transfer Rate = 64 kbit/s
Channel ID i = 0xA18381
Preferred, Channel 1
Progress Ind i = 0x8A81 - Call not end-to-end ISDN, may have in-band info
Calling Party Number i = 0x2180, '2071388387'
Plan:ISDN, Type:National
Called Party Number i = 0x81, '7523'
Plan:ISDN, Type:Unknown
Important information to mark in the above SETUP message is Called/Calling number, Plan and Type of Called/Calling number.
//CALL PROCEEEDING SENT BY THE GW TO PROVIDER//
106115: Jan 31 20:12:46.877: ISDN Se0/0/0:15 Q931: TX -> CALL_PROC pd = 8 callref = 0x80BA Channel ID i = 0xA98381
//ALERTING SENT BY THE GW TO PROVIDER//
106169: Jan 31 20:12:47.193: ISDN Se0/0/0:15 Q931: TX -> ALERTING pd = 8 callref = 0x80BA Progress Ind i = 0x8188 - In-band info or appropriate now available
//CONNECT SENT BY THE GW TO PROVIDER//
106263: Jan 31 20:12:59.140: ISDN Se0/0/0:15 Q931: TX -> CONNECT pd = 8 callref = 0x80BA Progress Ind i = 0x8188 - In-band info or appropriate now available
//CONNECT_ACK RECEIVED BY THE GW FROM THE PROVIDER//
106264: Jan 31 20:12:59.162: ISDN Se0/0/0:15 Q931: RX <- CONNECT_ACK pd = 8 callref = 0x00BA
