UCCE 11.5: CVP call server status is partial

[mr_cisco_cisco]

I have a sprawler setup in simplex mode in a lab environment.  CVP, ICM and virtual voice browser 11.5

CVP call server status shows as partial.  

If I click on partial it shows:

ICM Service:  Stopped

IVR Service: Partial Service

SIP Service:  In Service

Infrastructure:  In Service

I'm trying to determine what is causing ICM Service to show as stopped. 

I have validated the IP addresses in CVP.  In ICM all the services (distributor, loggerA etc) are running.  I also rebooted CVP/ICM.  

What am I missing?  

Also what logs can I collect to troubleshoot?

[Sasikumar Divvela]

can you attach the logs available under the below location from CVP machine.

C:\cisco\CVP\logs 

[mr_cisco_cisco]

Hi Sasikumar,

I have attached the logs.

[Sasikumar Divvela · Accepted solution]

CVP License is expired. So it is shutting down the system. Please apply new license and check the server status.

4450: 192.168.138.12: May 29 2017 12:00:00.923 -0700: %CVP_11_5_Infrastructure-2-LICENSING: Evaluation Port license already expired on Mon May 29 00:00:00 PDT 2017 [id:9904]
4451: 192.168.138.12: May 29 2017 12:00:00.923 -0700: %CVP_11_5_Infrastructure-2-LICENSE_EXPIRATION: The port license for Evaluation has expired, gracefully shutting down the system... ; Please contact your Cisco partner or representative to obtain a new license. [id:1035]
141: 192.168.138.12: May 29 2017 12:00:01.048 -0700: %CVP_11_5_SIP-3-SIP_INFO: Shutting down B2BUA, forced=false [id:5000]
142: 192.168.138.12: May 29 2017 12:00:01.048 -0700: %CVP_11_5_SIP-3-SIP_INFO: B2BUA is shut down. [id:5000]
8: 192.168.138.12: May 29 2017 12:00:01.110 -0700: %CVP_11_5_ICM-3-LOGMSG_ICM_SS_EXCEPTION: com.cisco.cvp.icm.ICMSubsystem::gracefulShutdown: java.lang.NullPointerException
at com.cisco.cvp.icm.ICMSubsystem.gracefulShutdown(ICMSubsystem.java:455)
at com.cisco.cvp.subsystem.BaseSubsystem$1.run(BaseSubsystem.java:563)

[mr_cisco_cisco]

Sorry for the delay.  Thank you for your analysis.  I will get new licenses.  

[mr_cisco_cisco]

I finally got new licenses and applied them (using the steps mentioned here http://bit.ly/2s0DJWo) and rebooted CVP server, but I still have the same problem.  I see these errors in Error.2017-06-11.00.txt:  

37: 192.168.138.12: Jun 11 2017 13:40:31.922 -0700: %CVP_11_5_MSGBUS-1-PLUGIN_INITIALIZATION_FAILURE:  MSGBUS_ORM1 Plugin initialization failed  due to  Plugin : MSGBUS_ORM1 encountered a socket disconnect due to: Transport is down [id:9] 

52: 192.168.138.12: Jun 11 2017 13:43:08.365 -0700: %CVP_11_5_MSGBUS-1-PLUGIN_INITIALIZATION_FAILURE:  MSGBUS_VXML1 Plugin initialization failed  due to  Plugin : MSGBUS_VXML1 encountered a socket disconnect due to: Plugin was stopped by the application [id:9] 

I googled around but couldn't find much.

Is there an issue with license file?  

I'm attaching the license file and CVP logs.  Appreciate any help.

[Sasikumar Divvela · Accepted solution]

i see that license is applied , and services are in service. Can you check the status from CVP diag portal (http://cvpcallserverip:8000/cvp/diag) . what is the service status and license status showing there.

23: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: License file valid, examining... [id:1040]
24: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: License Feature: CVP_SOFTWARE [id:1040]
25: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: License Feature: CVPPorts [id:1040]
26: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: Feature: CVPPorts:11.0 [id:1040]
27: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: add port: CVPPorts:11.0 [id:1040]
28: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: createPortManager, name:CVPPorts,vendorString:PORTS=1,Version:11.0 [id:1040]
29: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: Creating port manager...and adding1 [id:1040]
30: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: License Feature: CVP_Software [id:1040]
31: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: Feature: CVP_Software:11.0 [id:1040]
32: 192.168.138.12: Jun 11 2017 13:43:55.086 -0700: %CVP_11_5_Infrastructure-5-LICENSE_NOTICE: Component CVP_Software license added. Expires on Tue Sep 05 00:00:00 PDT 2017

192.168.138.12: Jun 11 2017 13:44:00.822 -0700: %CVP_11_5_Infrastructure-5-RECEIVED_STATE_MSG:  StateManager: Subsystem [SIP1] reported change to state [In Service] due to [Dependency is at full service] [id:1014]

[mr_cisco_cisco]

The diagnostic frame view shows ICM is in partial service and IVR in full service (see diagnostic.png).  It also shows one licensed port.

But in opsconsole ICM shows as stopped, and IVR in partial service (see opsconsole.png)

I'm confused as to why there is a difference.  

[Sasikumar Divvela]

I always use diagnostic . Not sure how it can be different in opsconsole (could be bug also).

Regarding ICM service in partical -- Did you connect your VRU PG with CVP ? That should bring state to "In service"

Probably the license which you applied have 1 port capacity. i have checked in my system , it shows different count   

[mr_cisco_cisco]

In that case, I will stick to diagnostic.

What I'll do is revert to a snapshot where the demo license is active.  I think that should do it.  I will revert back.

[Sasikumar Divvela]

Even if you revert the snapshot , when system sync with NTP and get the latest time , you will get license error.

[mr_cisco_cisco]

Oh OK.  Then I will have to request for a new license.

[mr_cisco_cisco]

Once again I'm sorry for the delay.  I don't give up that easily :)

I ended up rebuilding CVP but I made the problem worse ha.  Call server status is down (before it was partial).  

I checked the CVP logs and this time there is no license error but I found this:

12: 192.168.138.12: Jul 26 2017 22:39:31.565 -0700: %CVP_11_5_Infrastructure-0-START:  No Subsystems have been started.  Have you configured your server yet? ; Make sure <CVP_HOME>\conf\propFileList.properties is loading "system" and that <CVP_HOME>\conf\system.properties lists at least one subsystem to load. [id:1033]

I believe for propFileList.properties it is referring to this line:

PropertiesFile.1 = system

In system.properites I see these lines which leads me to believe it should load the necessary subsystems:

CVPServlet.subsystem.1.Class = com.cisco.cvp.central.CVPController

CVPServlet.subsystem.1.ID = CVPCTL1

CVPServlet.subsystem.2.Class = com.cisco.cvp.ivr.IVRSubSystem

CVPServlet.subsystem.2.ID = IVR1

CVPServlet.subsystem.3.Class = com.cisco.cvp.sip.SipSubsystem

CVPServlet.subsystem.3.ID = SIP1

Does this look right?  

I also checked diag view and see this:

Licensed PortManager: NA
Licensed For Reporting: false

Correct me if I'm wrong but shouldn't it show an eval license?

I'm thinking I need to get a new license from TAC.  Do you agree?

Attaching the logs.  

[mr_cisco_cisco]

I rebooted CVP and now diagnostic view shows this:

SIP state: In Service Dump SIP State Machine
ICM state: Partial Service
IVR state: In Service
Licensed PortManager: TOTAL [30] AVAIL [30] IN USE [0]
Licensed For Reporting: false

I see the eval licenses now.  ICM stll in partial state.  I'm guessing something wrong with the VRU PG.  Will look into it further.

[Ritesh Desai]

Hi,

Can you check once the Default VRU Port in CVP OAMP?

Refer URL: http://docwiki.cisco.com/wiki/ICM_subsystem_is_in_OUT_OF_SERVICE_state_from_IVR_side_and_also_the_VRU_PIM_stays_in_IDLE_state_rather_than_ACTIVE_state_in_the_ICM_peripheral_gateway

regards,

Ritesh Desai