Cisco 2811 Router will not activate paging system via FXO Port - No more Routes in RouteListName, Ca...

GLIDE · ‎09-14-2018

Hello, we’re are making a last-ditch effort to see if someone in the community has some advice on this problem as we are stumped and it’s looking like we will need additional equipment that we didn’t plan on getting yet. We have CUCM (V. 8.6.2.20000-2) operating in 2 buildings, connected with a VPN tunnel, and Cisco endpoint phones working for a while now. We are in the process of moving some of our users in our main building to Skype for Business, and all of our users in our 2^nd building, which is where we are having the problem I am describing.

We have had our teams exhaustively troubleshot this, with help from Cisco, and are a bit stuck. If anyone has any suggestions, it would be greatly appreciated! I’m going to paste some of the Cisco 2811 current-config at the end of this post.

Thanks in advance for your time if you read this!

The main issue right now:

- Cisco 2811 Router FXO port no longer activates a Valcom V-2006a overhead paging system, when dialed through CUCM, both systems do not have official vendor support.

More details:

We use an overhead paging system for announcements (mainly fire drills/emergency notifications) in our 2^nd building. On our legacy CUCM phones, we could dial a ## DTMF code, which dialed a gateway, to access this system. It’s now unusable, after making no changes, except implementation of an upgrade to SfB softphones for our users in that building and changes made to the Cisco 2811 router that activates that system which was backed out.

The CUCM system is tied in via a network connection to the Cisco 2811 router, in turn connected to an older Valcom V-2006a paging system, which requires a DTMF activation code of ##0 to activate the overhead paging speakers (tested, works with a test set phone hooked directly into it and dialing the DTMF code, but the 2811 router no longer activates it.)

We have had a lot of experts in the field working on this and they have come to some conclusions. Up until changes were made to our Cisco 2811 router recently for this project, this system had been tested within weeks and was working perfectly. Those changes were made by contracted experts in the field, with help from Cisco, but something went wrong and either there is a misconfiguration, or it was functioning as-is with an unknown problem for a while.

Up until now, the normal call flow of a call that activates our Cisco 2811 router’s FXO ports to dial the analog page system has looked like this:

[Phone/Endpoint] --> [CUCM SERVER] --> [MGCP GATEWAY] --> [VPN TUNNEL] --> Cisco 2811 Router FXO--> ##0 DTMF Dial which activates analog Valcom overhead paging system Our experts on this project, along with help from Cisco support, have come to this conclusion:

The reason for why this system was working up until changes to our Cisco 2811 router’s configuration were made is unknown, but since it is now out of support, we are looking at needing to replace the equipment since it’s no longer working. Cisco support has also apparently notated this is a bug in version CUCM 8.6 which we currently run.

They suggested the following 3 options:

- Restart our CUCM server, which we are going to do, tomorrow,
- Restart MGCP Gateways, which was done but did not help,
Or Upgrade to newer equipment compatible with a later version of CUCM, or an entirely different system that has support.

Here are their findings as to the problem with the call flow:

17:55:02.203 |RouteListCdrc::terminateCall - No more Routes in RouteListName = 434 Analog. Rejecting the call|1,100,56,1.67932216^10.1.135.138^SEP002584A1B30F17:55:02.203 |RouteListCdrc::terminateCall - Sending CcRejInd, with the cause code (34), to RouteListControl because all devices are busy/stopped.|1,100,56,1.67932216^10.1.135.138^SEP002584A1B30F17:55:02.204 |RouteListCdrc::terminateCall - precedenceBlocked == 0, cause = 34|1,100,56,1.67932216^10.1.135.138^SEP002584A1B30F17:55:02.204 |RouteListCdrc::terminateCall - Sending CcRejInd, with cause code (34), to Cc because it has not sent CcRegisterPartyB to Cc.|1,100,56,1.67932216^10.1.135.138^SEP002584A1B30F

Current configuration : 9154 bytes

!

version 12.4

service timestamps debug datetime localtime show-timezone

service timestamps log datetime msec localtime show-timezone

service password-encryption

!

hostname 2811MDF-434ELLIS

!

boot-start-marker

boot-end-marker

!

logging message-counter syslog

logging buffered 51200 warnings

[ REDACTED ]

aaa session-id common

clock timezone PST -8

clock summer-time PDT recurring

!

dot11 syslog

ip source-route

!

ip cef

!

no ip domain lookup

ip domain name glide.org

ip multicast-routing

no ipv6 cef

!

multilink bundle-name authenticated

!

trunk group ANALOG

hunt-scheme sequential

!

voice-card 0

!

application

global

service alternate default

!

[Redacted]

!

ip ssh version 2

!

class-map match-any AutoQoS-VoIP-RTP-Trust

match ip dscp ef

class-map match-any AutoQoS-VoIP-Control-Trust

match ip dscp cs3

match ip dscp af31

!

policy-map AutoQoS-Policy-Trust

class AutoQoS-VoIP-RTP-Trust

priority percent 70

class AutoQoS-VoIP-Control-Trust

bandwidth percent 5

class class-default

fair-queue

!

interface Tunnel0

ip address 1.1.2.2 255.255.255.0

ip pim sparse-dense-mode

no ip mroute-cache

tunnel source FastEthernet0/1

tunnel destination 10.1.50.1

!

interface FastEthernet0/0

description ### Uplink to LAN ###

ip address 10.2.105.5 255.255.255.0

ip flow ingress

ip flow egress

ip pim sparse-dense-mode

ip igmp join-group 230.20.20.20

duplex auto

speed auto

auto qos voip trust

service-policy output AutoQoS-Policy-Trust

!

interface FastEthernet0/1

description ### Uplink to MPLS router ###

ip address 10.2.60.2 255.255.255.252

ip pim sparse-dense-mode

no ip mroute-cache

duplex auto

speed auto

auto qos voip trust

service-policy output AutoQoS-Policy-Trust

!

interface Serial0/0/0

description $$ P2P T1 Connected to 330 ELLIS ST. $$

ip address 10.1.254.2 255.255.255.252

ip pim sparse-dense-mode

encapsulation ppp

shutdown

!

router eigrp 17218

network 10.2.20.0 0.0.0.255

network 10.2.60.0 0.0.0.3

network 10.2.80.0 0.0.0.255

network 10.2.94.0 0.0.0.255

network 10.2.95.0 0.0.0.255

network 10.2.100.0 0.0.0.255

network 10.2.105.0 0.0.0.255

network 172.16.20.0 0.0.0.255

network 192.168.30.0

no auto-summary

!

ip forward-protocol nd

ip route 0.0.0.0 0.0.0.0 10.2.60.1

ip route 10.2.20.0 255.255.255.0 FastEthernet0/0

ip route 10.2.80.0 255.255.255.0 FastEthernet0/0

ip route 10.2.94.0 255.255.255.0 FastEthernet0/0

ip route 10.2.95.0 255.255.255.0 FastEthernet0/0

ip route 10.2.100.0 255.255.255.0 FastEthernet0/0

ip route 172.16.20.0 255.255.255.0 FastEthernet0/0

no ip http server

ip http authentication aaa login-authentication http-auth

ip http authentication aaa exec-authorization http-auth

ip http secure-server

ip http timeout-policy idle 60 life 86400 requests 10000

!

ip flow-cache timeout active 5

ip flow-export source FastEthernet0/0

ip flow-export version 5

ip flow-export destination 10.1.20.60 9996

!

ip pim rp-address 10.1.50.1

ip pim spt-threshold infinity

ip mroute 10.1.135.20 255.255.255.255 Tunnel0

ip mroute 10.1.50.1 255.255.255.255 Tunnel0

!

logging trap warnings

logging facility local3

logging 10.1.20.60

!

snmp-server community Unk0d1t10n@lly RW

snmp-server ifindex persist

snmp-server location Glide

radius-server host 10.1.20.17 auth-port 1645 acct-port 1646 key 7 122B25131B1E1F23066A007B

!

control-plane

!

rmon event 33333 log trap AutoQoS description "AutoQoS SNMP traps for Voice Drops" owner AutoQoS

rmon alarm 33333 cbQosCMDropBitRate.50.14618161 30 absolute rising-threshold 1 33333 falling-threshold 0 owner AutoQoS

rmon alarm 33334 cbQosCMDropBitRate.34.5364641 30 absolute rising-threshold 1 33333 falling-threshold 0 owner AutoQoS

!

voice-port 0/1/0

trunk-group ANALOG 2

echo-cancel coverage 64

timing hookflash-out 50

connection plar 6200

description $$ 415-400-3796 $$

!

voice-port 0/1/1

trunk-group ANALOG 1

echo-cancel coverage 64

timing hookflash-out 50

connection plar 6200

description $$ 415-400-3798 $$

!

voice-port 0/1/2

echo-cancel coverage 64

timing hookflash-out 50

description $$ FXO $$

!

voice-port 0/1/3

echo-cancel coverage 64

timing hookflash-out 50

description $$ FXO $$

!

voice-port 0/3/0

!

voice-port 0/3/1

!

voice-port 0/3/2

!

voice-port 0/3/3

!

ccm-manager redundant-host 10.4.20.13

ccm-manager mgcp

no ccm-manager fax protocol cisco

ccm-manager music-on-hold

ccm-manager config server 10.1.135.12 10.4.20.13

ccm-manager config

!

mgcp

mgcp call-agent 10.1.135.12 2427 service-type mgcp version 0.1

mgcp rtp unreachable timeout 1000 action notify

mgcp modem passthrough voip mode nse

mgcp package-capability rtp-package

mgcp package-capability sst-package

mgcp package-capability pre-package

no mgcp package-capability res-package

no mgcp timer receive-rtcp

mgcp sdp simple

mgcp fax t38 ecm

mgcp fax t38 inhibit

mgcp bind control source-interface FastEthernet0/0

mgcp bind media source-interface FastEthernet0/0

mgcp behavior g729-variants static-pt

!

mgcp profile default

!

dial-peer voice 999010 pots

service mgcpapp

port 0/1/0

!

dial-peer voice 999011 pots

service mgcpapp

port 0/1/1

!

dial-peer voice 9007 pots

trunkgroup ANALOG

description $$ For Local Calling to 415 Area Code Only $$

destination-pattern 9[2-9]......

forward-digits 7

!

dial-peer voice 9011 pots

trunkgroup ANALOG

description $$ LD Calling Only $$

destination-pattern 91[2-9]..[2-9]......

forward-digits 11

!

dial-peer voice 911 pots

trunkgroup ANALOG

description $$ For EMERGENCY Calling Only $$

destination-pattern 911

forward-digits 3

!

dial-peer voice 9911 pots

trunkgroup ANALOG

description $$ For EMERGENCY Calling Only $$

destination-pattern 9911

forward-digits 3

!

dial-peer voice 90011 pots

trunkgroup ANALOG

description $$ For International Calling Only $$

destination-pattern 9011T

prefix 011

!

dial-peer voice 9411 pots

description $$ Forward 411 to GOOGLE 411 Service $$

destination-pattern 9411

prefix 18004664411

!

dial-peer voice 9611 pots

trunkgroup ANALOG

description $$ Service Calling Only $$

destination-pattern 9[235678]11

forward-digits 3

!

dial-peer voice 999033 pots

service mgcpapp

port 0/3/3

!

dial-peer voice 999030 pots

service mgcpapp

port 0/3/0

!

dial-peer voice 999031 pots

service mgcpapp

port 0/3/1

!

dial-peer voice 999013 pots

service mgcpapp

port 0/1/3

!

dial-peer voice 999012 pots

service mgcpapp

port 0/1/2

!

call-manager-fallback

secondary-dialtone 9

max-conferences 16 gain -6

transfer-system full-consult

ip source-address 10.2.105.5 port 2000 strict-match

max-ephones 42

max-dn 144 dual-line

system message primary GLIDE FAILOVER MODE

time-zone 5

!

line con 0

line aux 0

line vty 0 4

privilege level 15

transport input telnet ssh

line vty 5 15

privilege level 15

transport input telnet ssh

!

scheduler allocate 20000 1000

end

Jonathan Schulenberg · ‎09-15-2018

That SDI trace snippet suggests the MGCP port is either offhook or unregistered. What port is the Valcom connected to, what’s the port state (show voice port), and does it show registered in CUCM?

Also, did anyone touch the Valcom itself (eg connect to a different port or change DIP switch settings)?

Scott Pedersen · ‎09-21-2018

@Jonathan Schulenberg wrote:
"
Also, did anyone touch the Valcom itself (eg connect to a different port or change DIP switch settings)?"

This is very important. The valcom uses a switch setting to change between Loop Start and ground start. There are 3 switches in total for this and any of them being flipped the wrong way will stop the port from working correctly.

Also be sure that the FXO port isn't stuck in an off-hook state after your page finishes. A simple shut/no shut of the port resets this. You may have to adjust disconnect signal detection on the FXO or configure ground start vs. loop start if this is a consistent issue.

GLIDE · ‎09-21-2018

Thank you for the great suggestions. We rebooted our CUCM server but it didn't resolve this.

We are looking into how the Gateways in CUCM are configured currently, as there is in fact, evidence they are not configured correctly. We are still not sure if that is the root of the problem because up until recently it was working properly and to our knowledge, these were not changed.

In CUCM we found:
1. The FXO port does show as registered, but it is registered to the Call Manager Subscriber in another location (datacenter) instead of the Publisher at our main building.
2. The FXO port shows as “Device is not trusted” in Call Manager

Nothing on the Valcom such as DIP switches were ever touched, the only thing we have ever changed was the volume, and that change was made prior to it still working via adjustment of the little board mount potentiometer. Have always been extremely careful not to disturb it for the aforementioned concern.

Thanks again for any advice.

Cisco 2811 Router will not activate paging system via FXO Port - No more Routes in RouteListName, Cause Code 34