Re: ISR4300 SIP Service Disabled

joeharb · ‎02-04-2022

We have an ISR/CUBE running and it has been in production for several years, but we it was reported that inbound/outbound calls were failing. In troubleshooting it appears the SIP service is disabled, and I believe it has something to do with Licensing.

Please see the following output:

Feb 4 15:44:08.694: //CUBE-SL/Function/is_smart_agent_ack_not_received:
Feb 4 15:44:08.694: //CUBE-SL/Function/is_smart_agent_ack_not_received:
Feb 4 15:44:08.695: //-1/xxxxxxxxxxxx/SIP/Msg/ccsipDisplayMsg:
Sent:
SIP/2.0 503 Service Unavailable
Via: SIP/2.0/TCP 10.X.X.X:5060;branch=z9hG4bK97b1cd378767b4
From: <sip:10.X.X.X>;tag=2045899188
To: <sip:10.X.X.X>;tag=D7325812-25AF
Call-ID: 48e95980-1fd149c8-852e5c-b11020a@10.2.17.11
Warning: 399 10.246.100.1 SIP service disabled
CSeq: 101 OPTIONS
Reason: Q.850;cause=38
Content-Length: 0

We are in the process of trying to engage TAC but wanted to reach out to this group to see if anyone has any suggestions.

Thanks,

Joe

Maren Mahoney · ‎02-04-2022

What is at 10.246.100.1? A 503/Cause 38 (and Warning 399) error generally means that SIP signaling could not be established to the indicated target. The problem is likely not on the ISR4300 (especially if nothing there has changed). The problem is more likely to be at the target. Did an IP or FQDN change there? Did authentication/encryption change between these systems? These are the kinds of things to look at.

Maren

Scott Leport · ‎02-04-2022

Hi,

You’re right. This is the reason for this message and you need to get your CUBE checked in with a Smart Licensing server. Once you do that, the SIP service will start on the CUBE and all will be well again.

Leo Laohoo · ‎02-04-2022

@joeharb wrote:

Warning: 399 10.246.100.1 SIP service disabled

Please post the complete output to the router command of "sh license"? It smells like the evaluation license has expired.

joeharb · ‎02-05-2022

CSIFTC4351#show license al
Smart Licensing Status
======================

Smart Licensing is ENABLED

License Conversion:
Automatic Conversion Enabled: False
Status: Not started

Export Authorization Key:
Features Authorized:
<none>

Utility:
Status: DISABLED

Smart Licensing Using Policy:
Status: ENABLED

Data Privacy:
Sending Hostname: yes
Callhome hostname privacy: DISABLED
Smart Licensing hostname privacy: DISABLED
Version privacy: DISABLED

Transport:
Type: cslu
Cslu address: <empty>
Proxy:
Not Configured

Miscellaneous:
Custom Id: <empty>

Policy:
Policy in use: Merged from multiple sources.
Reporting ACK required: yes (CISCO default)
Unenforced/Non-Export Perpetual Attributes:
First report requirement (days): 365 (CISCO default)
Reporting frequency (days): 0 (CISCO default)
Report on change (days): 90 (CISCO default)
Unenforced/Non-Export Subscription Attributes:
First report requirement (days): 90 (CISCO default)
Reporting frequency (days): 90 (CISCO default)
Report on change (days): 90 (CISCO default)
Enforced (Perpetual/Subscription) License Attributes:
First report requirement (days): 0 (CISCO default)
Reporting frequency (days): 0 (CISCO default)
Report on change (days): 0 (CISCO default)
Export (Perpetual/Subscription) License Attributes:
First report requirement (days): 0 (CISCO default)
Reporting frequency (days): 0 (CISCO default)
Report on change (days): 0 (CISCO default)

Usage Reporting:
Last ACK received: <none>
Next ACK deadline: Feb 03 08:00:01 2022 MST
Reporting push interval: 30 days
Next ACK push check: <none>
Next report push: Feb 02 21:00:30 2022 MST
Last report push: <none>
Last report file write: <none>

Trust Code Installed: <none>

License Usage
=============

securityk9 (ISR_4351_Security):
Description: securityk9
Count: 1
Version: 1.0
Status: IN USE
Export status: NOT RESTRICTED
Feature Name: securityk9
Feature Description: securityk9
Enforcement type: NOT ENFORCED
License type: Perpetual

AdvUCSuiteK9 (ISR_4351_AdvancedUCSuite):
Description: AdvUCSuiteK9
Count: 1
Version: 1.0
Status: IN USE
Export status: NOT RESTRICTED
Feature Name: AdvUCSuiteK9
Feature Description: AdvUCSuiteK9
Enforcement type: NOT ENFORCED
License type: Perpetual

uck9 (ISR_4351_UnifiedCommunication):
Description: uck9
Count: 1
Version: 1.0
Status: IN USE
Export status: NOT RESTRICTED
Feature Name: uck9
Feature Description: uck9
Enforcement type: NOT ENFORCED
License type: Perpetual

Product Information
===================
UDI: PID:ISR4351/K9,SN:FDO211216FP

Agent Version
=============
Smart Agent for Licensing: 5.0.9_rel/68

License Authorizations
======================
Overall status:
Active: PID:ISR4351/K9,SN:FDO211216FP
Status: NOT INSTALLED
Status:PAK

Legacy License Info:
regid.2014-12.com.cisco.ISR_4351_UnifiedCommunication,1.0_a04fec0e-e944-4096-bcf8-05d6e9a0a6d3:
DisplayName: uck9
Description: uck9
Total available count: 1
Term information:
Active: PID:ISR4351/K9,SN:FDO211216FP
License type: PERPETUAL
Term Count: 1

regid.2014-12.com.cisco.ISR_4351_AdvancedUCSuite,1.0_9333dad1-59df-4b3d-8a65-d7a4b9df7296:
DisplayName: AdvUCSuiteK9
Description: AdvUCSuiteK9
Total available count: 1
Term information:
Active: PID:ISR4351/K9,SN:FDO211216FP
License type: PERPETUAL
Term Count: 1

Roger Kallberg · ‎02-05-2022

You have not received a RUM ACK from SSM.
Usage Reporting:
Last ACK received: <none>
Next ACK deadline: Feb 03 08:00:01 2022 MST

As you are past the deadline it means that SIP services will be shutdown.

Are you using CSSM, cloud, or on-prem SSM?

joeharb · ‎02-07-2022

I guess that is the issue, we have not converted to smart license on most of our equipment other than ACI/CUCM. From what I can tell the enforcement of disabling the service was changed in later versions. Would a reboot restart the timer, we are working with our Account team to determine the contract status of this device as well. All the license features appear to be perpetual, not sure why this specific service would be different. I am going to review our smart license portal to see what is availalble.

Any suggestions on resetting the timer or reboot on the device in the meantime?

Thanks,

Joe

Roger Kallberg · ‎02-07-2022

The only thing that would restart the timer is AFAIK to get an ACK on the RUM. This happens when the on-prem SSM or cloud SSM has processed the RUM sent by the endpoint, that is known as a PI, Product Instance.

The router is set to use Smart Licensing.

CSIFTC4351#show license al
Smart Licensing Status
======================

Smart Licensing is ENABLED

cornerstonesd · ‎02-24-2022

This looks like a smart license issue, the CSLU address is not configured in your output. If your CSSM registration has lapsed it will shutdown your SIP service.

View RUM reports... (there are probably dozens of unacknowledged reports)

(config)# service internal

# show license tech support | include RUM reports

# test license smart rum id-list

Option 1: Register Direct to CSSM online (Must generate idtoken in your Smart account)

*Change the transport policy to "smart"*

# license smart trust idtoken [ generated token ] local force

(config)# ip http client source-interface [ interface ]

(config)# license smart transport smart
(config)# license smart url default

Read more... https://www.cisco.com/c/en/us/td/docs/routers/sl_using_policy/b-sl-using-policy/how_to_configure_workflows.html#Cisco_Generic_Topic.dita_536a7edd-4cb2-458e-9171-6c7705e685c8

Option 2: Register to CSSM via CSLU host (no idtoken required but you still need a smart account)

Download the CSLU software here...https://software.cisco.com/download/home/286285506/type

(config)# ip http client source-interface GigabitEthernet0/0/0
(config)# license smart url cslu http://cslu host ip:8182/cslu/v1/pi
(config)# license smart transport cslu

# license smart sync all

Read more... https://www.cisco.com/c/en/us/td/docs/routers/sl_using_policy/b-sl-using-policy/how_to_configure_workflows.html#Cisco_Concept.dita_ee6ccac1-38e3-438b-bc00-4cacbbf57f64

You might need these too... (the Cisco IPs are real)

(config)# ip domain lookup source-interface [ interface ]
(config)# ip host tools.cisco.com 173.37.145.8 72.163.4.38

It could also be a CPU issue, there is a persistent CPU bug in v16.x and it is unconfirmed in later versions (I am running v17.6.x).

# show proc cpu history

FINAL RESORT!!! >>> USE AT YOUR OWN RISK <<<

This command should unlock the SIP service and it will delete all the RUM reports sitting on your router... you don't need them.

# license smart factory reset

(reload required)

I lost 3 weeks of my life to bring you this information... no one should have to suffer like this.

Leo Laohoo · ‎02-24-2022

@cornerstonesd wrote:

This looks like a smart license issue

There is a massive list of Field Notices involving Cisco Smart License.

FN - 72323 - Catalyst Switching Products - QuoVadis Root Certificate Decommission Might Affect Smart Licensing and Smart Call Home Functionality, or Display Other Symptoms

Cisco's "Smart" License -- Not so "smart", eh?

desmond.cassidy · ‎04-27-2022

I am losing weeks to months understanding Cisco licenses and now 12 months in, its come back to bite me. All SIP calls disabled

for how long I am not sure could have being 5 days.

Egg on my face.

Matheus Lopes · ‎07-05-2022

Thanks @cornerstonesd . Much appreciated!

I've performed the option 1, reset the SIP Trunk on CUCM and it fixed the issue.

Best Regards,

Matheus

emiliolara · ‎08-09-2022

CSCvz82168 is the bug “If the router does not receive an acknowledgment before the ACK deadline call processing is disabled.

Just happened to me

TAC applied the WorkAround

# license smart factory reset

(reload required)

desmond.cassidy · ‎08-09-2022

Very frustrating , what a strange feature to introduce. It completely knocks out the service. All the best.

Roger Kallberg · ‎08-09-2022

You can upgrade IOS on the gateway to a version that is not affected by this “feature”. We upgraded from 17.3.4 to 17.6.3 on all our voice gateways and have not seen this since.