IOS upgrade causes VoIP phones Busy signal calling out

[jwornstaff]

I have strange issue on small VoIP network running on 3 CME routers. The network is pretty static and unchanged for 10 plus years, and working fine as configured. I upgrade the 4451x routers fro 17.9.6 to 17.12.05a recently and after doing so, the VoIP phones on the site 3 router (333) were unable to call internally to the other IP phones to 333xxxx or  the other site routers on 222xxxx, they receive busy signal after the first digit is entered. But they can call outside POTs line when dialing 9, and if they use the recently called menu to redial the call goes through.  If put the router back on IOS-XE 17.9.6 the phones work again. 

The phones on the other site routers do not have this problem on the new IOS-XE, the only difference is they do not have FXO cards and no attached POTS lines.

I have Cisco TAC case open but thus far they have had no solutions or answers. Attached are is the config for the affected router and some debug logs, the debugs indicate no dial-peer found but the dial peer is configured and works as configured under the older IOS.

Any thoughts or help would be appreciated.

Site 3 router voice configuration: Note IPs and phone numbers changed for security

voice iec syslog
!
voice register global
mode cme
source-address 3.3.2.3 port 5060
max-dn 100
max-pool 100
authenticate realm al
timezone 21
tftp-path flash:
create profile sync 2147160237244596
ntp-server 3.3.1.1 mode directedbroadcast
auto-register
!
!
voice register dn 2
number 3331001
label S3 OPS MGR
!
voice register dn 3
number 3331002
call-forward b2bua busy 98034774592
call-forward b2bua noan 98034774592 timeout 60
label S3 OPS Cube
!
voice register dn 31
number 3331000
name SP3NPHN01
label S3 Conf RM
!
voice register dn 33
number 3331003
name Conf RM Table
label Conf RM Table
!
voice register dn 34
number 3330100
label S3 All Phones
!
voice register dn 35
number 3335000
name SP5PHN05
label CMOC (SOH)
!
voice register dn 36
number 3330549
name SP3PHN03
label GH-OCE
!
voice register template 1
!
voice register template 2
!
voice register pool 1
busy-trigger-per-button 2
id mac 006C.BCBD.66E0
type 8811
number 1 dn 31
number 2 dn 34
dtmf-relay sip-notify
username netadmin1 password 6 gFCRLRNBcXVXVPaIAWFFB`XDRbBRSObLbAAB
codec g711ulaw
no vad
after-hour exempt
!
voice register pool 2
busy-trigger-per-button 2
id mac D0EC.35C1.3AB6
type 8811
number 1 dn 2
number 2 dn 34
dtmf-relay sip-notify
username netadmin2 password 6 AKCNFM[Wh`c\_WbQdCfCBULFGNE[]bMWaAAB
codec g711ulaw
no vad
!
voice register pool 3
busy-trigger-per-button 2
id mac D0EC.35C1.44DF
type 8811
number 1 dn 3
number 2 dn 34
number 3 dn 36
dtmf-relay sip-notify
username netadmin3 password 6 TPE`^RdWaX[TRZKW`cZNhRceSZUGURcTFAAB
codec g711ulaw
no vad
!
voice register pool 5
busy-trigger-per-button 2
id mac D0EC.35C1.3E42
type 8811
number 1 dn 35
dtmf-relay sip-notify
username netadmin5 password 6 ROb_IISIGUchAILg_gL`YFKaC[JWUVPKdAAB
codec g711ulaw
no vad
speed-dial 1 3331002 label "SPEED DIAL to S3 Phones"
!
voice register pool 6
busy-trigger-per-button 2
id mac 1096.C615.0BEF
type 8832
number 1 dn 33
dtmf-relay sip-notify
username netadmin6 password 6 gVaOSRefOeWFHCXPWRXbaFOC]ESUcY``TAAB
codec g711ulaw
no vad
!

dial-peer voice 222 voip
destination-pattern 222....
session protocol sipv2
session target ipv4:20.0.2.1
incoming called-number .
codec g711ulaw
!
dial-peer voice 777 voip
destination-pattern 7771...
session protocol sipv2
session target ipv4:70.0.2.1
codec g711ulaw
!
dial-peer voice 1 pots
preference 1
destination-pattern 9T
port 0/1/0
!
dial-peer voice 2 pots
preference 2
destination-pattern 9T
port 0/2/0
!
dial-peer voice 3 pots
preference 3
destination-pattern 9T
port 0/1/1
!
dial-peer voice 4 pots
preference 4
destination-pattern 9T
port 0/2/1
!
dial-peer voice 5 pots
preference 5
destination-pattern 9T
port 0/1/2
!
dial-peer voice 6 pots
preference 6
destination-pattern 9T
port 0/2/2
!
!
sip-ua

!
!
!
!
!
!
voice-card 0/1
no watchdog
!
voice-card 0/2
no watchdog
!
voice-card 0/4
no watchdog

voice-port 0/1/0
connection plar 2224173
description POTS Line RK3 Port 1
caller-id enable
!
voice-port 0/1/1
connection plar 2224196
description POTS Line RK3 Port 3
caller-id enable
!
voice-port 0/1/2
connection plar 2224219
description POTS Line RK3 Port 5
caller-id enable
!
voice-port 0/1/3
!
voice-port 0/2/0
connection plar 2224371
description POTS Line RK3 Port 7
caller-id enable
!
voice-port 0/2/1
secondary dialtone
description POTS Line RK3 Port 9
caller-id enable
!
voice-port 0/2/2
connection plar 3330549
description POTS Line RK3 Port 11
caller-id enable
!
voice-port 0/2/3

telephony-service
max-ephones 4
max-dn 4
ip source-address 3.3.2.3 port 2000 secondary 3.3.2.4
cnf-file location flash:
dialplan-pattern 1 703377.... extension-length 7 extension-pattern 333....
max-conferences 12 gain -6
transfer-system full-consult

[Alexis Amaro · Accepted solution]

The phone is sending only 1 digit in the Invite and CME is not entering digit collection, so the call fails with no dp match:

000840: Jul 1 12:11:33.368: %VOICE_IEC-3-GW: C SCRIPTS: Internal Error (No dialpeer match): IEC=1.1.128.11.5.0 on callID 41 GUID=5CCF175955AB11F0802A8F8C321A65E1
000841: Jul 1 12:11:33.368: //41/5CCF1759802A/CCAPI/ccCallDisconnect:
Cause Value=1, Tag=0x0, Call Entry(Previous Disconnect Cause=0, Disconnect Cause=0)

Can you try disabling digit collect KPML in a phone?

Enter "no digit collect kpml" in one voice register pool and save/apply the config.

[Roger Kallberg]

I find this odd.

000824: Jul 1 12:11:33.364: //-1/5CCF1759802A/CCAPI/cc_api_call_setup_ind_common:
Interface=0x7095BF212648, Call Info(
Calling Number=3331002,(Calling Name=)(TON=Unknown, NPI=Unknown, Screening=User, Passed, Presentation=Allowed),
Called Number=2(TON=Unknown, NPI=Unknown),
Calling Translated=FALSE, Subscriber Type Str=Unknown, FinalDestinationFlag=TRUE,
Incoming Dial-peer=222, Progress Indication=NULL(0), Calling IE Present=TRUE,
Source Trkgrp Route Label=, Target Trkgrp Route Label=, CLID Transparent=FALSE), Call Id=41

When making a call from a CME phone it should match the phones created dial peer on the inbound direction, not a configured dial peer that is intended to be used by other call scenarios. I would suggest that you remove the incoming called-number . from DP 222 and use inbound match on VIA information instead. How to do this is outlined in the document. Explain Cisco IOS and IOS XE Call Routing 

I would also suggest that you put bind statements on all your VoIP dial peers as that is best practices.

[jwornstaff]

Thanks for the reply and info. The incoming-call number . Was added sometime after another IOS broke two way audio for some reason. I also looked at link you sent but unsure how I would apply with this scenario. Any example?

[Roger Kallberg]

Look at this chapter. Inbound URI Dial-Peer Matching.

[jwornstaff]

Cisco TAC suggested I had "no digit collect kpml" under the voice pools, which seems to have worked and the phones can now make calls again. It just does not make sense that this has been working fine under previous IOS versions but not on 17.12.05a and my other CME routers that don't have VOIP FXO cards work fine without the "no digit collect kpml" but oh well working now.