911 calls failing

[pbrackett1]

I am working with a network phone system running CUCM 10, alongside CER and some other similar systems. Inside the phone network, calls to 911 are failing. The caller gets a rapid busy signal when they try. I can see the attempts in the CER call history, and I do receive the emergency alert notification ("Alert, caller at extension XXXX is in emergency."). The call router (a Cisco C2911) shows the calls being made (see the attached log for an example). My telecom is reporting that the calls are being denied because they are not properly defined. They have pointed to a couple of specific lines in the log file from my router:

Called Party Number i = 0x80, '911'
Plan:Unknown, Type:Unknown

Anyone have any ideas. I'm in over my head and could really use some assistance.

[Roger Kallberg]

If your service provider is saying that the call is denied because it is not properly defined they need to tell you how they expect the call to be defined. If I where to guess I would think that they want to get the called number in this plan and type, Plan:ISDN, Type:National.

[pbrackett1]

So, I just got off the phone with the telecom and they are suggesting that the call from the Cisco phone system is being forwarded to the ISDN and that forwarding is causing the ISDN to put a diversion header on the SIP packet. This eventually causes the call to be rejected. The technician at the telecom said that something inside of the Cisco phone system is passing the 911 calls to the ISDN as forwarded calls. The technician referenced these lines in the ISDN debug:

09:47:13.358 ISDN.L2_FMT PRI 1 IE - 74 REDIRECTING # Len=6
09:47:13.358 ISDN.L2_FMT PRI 1 21 Numb. Type:NATIONAL
09:47:13.358 ISDN.L2_FMT PRI 1 Numb. Plan:E.164
09:47:13.359 ISDN.L2_FMT PRI 1 00 Presentation:ALLOWED
09:47:13.360 ISDN.L2_FMT PRI 1 Screening:USER PROVIDED
09:47:13.360 ISDN.L2_FMT PRI 1 82 Redir reason:CALL FWD NO REPLY

So, where should I look to find out why this call is being passed as a forwarded call? CER? CUCM? Other calls work, but not 911. This failed for us in a real emergency last week, but I couldn't tell you the last time before that when I got an emergency notifcation from the system.

[Roger Kallberg]

In your shared debug it is visible that you send redirecting information and your service provider is rejecting the call.

Name = HS Server Room x3885 
Calling Party Number i = 0x2181, '2694610065' 
	Plan:ISDN, Type:National 
Called Party Number i = 0x80, '911' 
	Plan:Unknown, Type:Unknown 
Redirecting Number i = '!', 0x0082, '911' 
	Plan:ISDN, Type:National
Mar 14 10:50:43.328 EDT: ISDN Se0/0/0:23 Q931: RX <- CALL_PROC pd = 8 callref = 0xBE59
Channel ID i = 0xA98381 
	Exclusive, Channel 1
Mar 14 10:50:43.428 EDT: ISDN Se0/0/0:23 Q931: RX <- DISCONNECT pd = 8 callref = 0xBE59
Cause i = 0x8295 - Call rejected

Not sure if it's applicable to your situation, but at one of our sites in USA where we use a SIP trunk we had to make sure that we did not pass any diversion information for emergency calls to 911 by having it removed in the SIP profile with this "request INVITE sip-header Diversion remove". Obviously not directly applicable to your situation as you're using a PRI, but that could be something to look into to not pass redirecting information in the 911 calls.

[Roger Kallberg]

Can you please share the details on how you are routing the calls to CER and then to the PSTN. Provide screenshots if possible as that is the most detailed way to outline it.

[Steven L]

some providers require 911 to be formatted as "+1911"

[pbrackett1]

Thanks, Steven. I don't think that's the case, here. The telecom technician didn't mention it on the call and I've never had it configured that way before (at least, not that I know of).

[Steven L]

do the rest of your calls use +1?

if so, humor me and change it.

[pbrackett1]

I'm sorry, Steven. I thought I replied to this already. I'm not sure how to make the change that you're suggesting, and I thought I replied to you already with the information that I ended up posting yesterday at 12:52 p.m. (see the attached route plan report and route patterns list). I didn't mean to drop our conversation.

[Steven L]

no worries. 

I am not sure what the cause of your issue is but, if this has tested successfully in the past, SOMETHING changed

A setting on your side or a setting on the Carrier side.

If you are 100% certain nothing changed on your side, then you have to look at what the carrier changed.

Requiring +e164 is a change i have seen, they do typically notify you before they start requiring this.

I am not saying it is the reason, it is just a logical test in my opinion

[pbrackett1]

I've attached my Route Plan report from CUCM. It's not clear to me how the 911 calls are being routed. Our callers usually dial an 8 to reach the outside world. Then a ten digit number for local calls and a +1 for long distance. The route plan includes several entries that mention 911, and since we have CER in the mix, I'd imagine some of those entries have to do with CER. There is an entry in the route plan report that just says "911", but that looks like it connects to our CER server. I've also tried to include my route patterns in a text file.

Having said all that, I can't get through to 911 no matter how I dial it. "911" & "1911" & "8911" & "81911" all fail. I think it might be related to CER.

[Daniel Bosch]

How is CUCM connected to the voice router - SIP or MGCP?  If you have a SIP trunk configured, you can use and inbound SIP dial-peer to strip it, as Roger mentioned.   I've had to do this as well but applied to outbound dial-peer to SIP carrier instead.   In your case this may not be possible on a POTS dial-peer, but the same result could be accomplished by stripping the diversion header on a matching inbound SIP peer from CUCM.

Here's an example config that would probably work.  Our CUCM strips any prepended digits before sending 911 to the router.  If your CUCM does not do this, then you'll need inbound peers to match any/all combination of 911 outbound peers you have.

voice service voip
sip
sip-profiles inbound
!
voice class sip-profiles 911
response ANY sip-header Diversion remove
request ANY sip-header Diversion remove
!
dial-peer voice 911 voip
description *** Inbound from CUCM ***
session protocol sipv2
incoming called-number 911
voice-class sip profiles 911 inbound
voice-class sip bind control source-interface GigabitEthernet0/0/1
voice-class sip bind media source-interface GigabitEthernet0/0/1
dtmf-relay rtp-nte
codec g711ulaw
fax-relay ecm disable
fax-relay sg3-to-g3
fax rate 14400
fax protocol pass-through g711ulaw
no vad

I haven't dealt with ISDN or MGCP for years so can't offer much help in those protocols any longer. If you're using MGCP and there's no way to strip the diversion header, you might be forced to configure a CUCM SIP trunk for 911 calls.  But I'd recommend checking with Cisco TAC before taking a drastic measure like that.

[Roger Kallberg]

Great post. @Daniel Bosch

If the OP do need to have multiple patterns match on the inbound dial peer this can be done with an e164 pattern map in the inbound direction. Something like this should work.

voice class e164-pattern-map 911
 description Emergency numbers to PSTN
 e164 911
 e164 9.911 ! add the permutations you have
!
dial-peer voice 911 voip
 no incoming called-number
 incoming called e164-pattern-map 911

Apart from this I would not recommend to use a single codec on the dial peer, but use a codec list instead.

[Daniel Bosch]

You betcha! Using e164-pattern-map is way better than multiple dial-peers.

[pbrackett1]

I appreciate what all of you have been trying to do to help me. Part of the problem that I'm having is that I don't understand what you're saying when you discuss these things --> I am way out of my league here. Dial-peers and pattern maps and OPs and codecs --> sounds like Greek. I can say that I have a Cisco environment (version 10 of CUCM and CER and Unity and such) that connects to a Cisco C2911 router and then to an AdTran Total Access router. This AdTran router was installed in the spring of 2022 when we switched from a TDM PRI to an IP PRI. So, I would guess we are running MGCP. It looks like I need to figure out a way to get CUCM to strip the diversion header, if I'm reading the comments correctly.

The other problem that I'm having is that I don't have command line access to be able to execute any of the commands that you guys are putting in your comments. I have webGUI access to the pieces of the environment, but I am getting the sense that you are wanting me to be able to access CLI and I don't know that I can do that.

I'm sorry to be such a pain. I am sure that you'll just abandon me at some point for being this difficult. But, I really do appreciate any and all advice.