PG Connection to Central Controller Side A failed

[Ping Liu]

I recently installed ICM 8.5.4 in simplex mode (Side A only) on Cisco UCS B serial with VM 5. When we started all services on Rogger, PG , and AWD, we realise that pg and ctisvr was in IDLE, and PIMs were not active.

Here is basic detail:

1. Rogger, PG, and AWD are Pingable each other on both of public/private link

2. Rogger Public IP:    10.91.208.50 (v); 10.91.208.60 (vh)

3. PG Public IP:          10.91.208.51 (v); 10.91.208.61 (vh)

4. Rogger Private IP:    10.91.218.50 (p); 10.91.218.60 (ph)

5. PG Private IP:          10.91.218.51 (p); 10.91.218.61 (ph)

6. AWD IP:                  10.91.208.52

7. Binding order of two NICs in Rogger & PG is Public first, then follow by Private

ra - ccag log:

Events from January 30, 2013:

12:16:27:039 ra-ccag Initializing Event Management System (EMS) Library.

12:16:27:039 ra-ccag Trace: EMS Server pipe uicm\RouterA\ccagEMSPipe enabled for uicm\RouterA\ccag

12:16:27:041 ra-ccag Initializing Node Manager Library.

12:16:27:041 ra-ccag Trace: Monitor Server pipe uicm\RouterA\ccagCmdPipe enabled for uicm\RouterA\ccag

12:16:27:043 ra-ccag Trace: EMT I/O completion ports: max threads=8, concurent threads=0

12:16:27:051 ra-ccag DMP Agent process starting.  Release 8.5.4.0 , Build 275.

12:16:27:053 ra-ccag 2 Peripheral Gateways are configured.

12:16:27:057 ra-ccag Trace: MBufLimitMgr OpenMutex Global\MBUF Limit Table Mutex

12:16:27:555 ra-ccag Connection to MDS process established.

12:16:27:685 ra-ccag MDS is in service.

12:16:28:024 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40002-RIP10.91.208.51) succeeded

12:16:28:029 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40017-RIP10.91.208.61) succeeded

12:16:29:909 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40003-RIP10.91.208.61) succeeded

12:16:30:655 ra-ccag Central Controller service is available.

12:16:31:298 ra-ccag Device PG01 connected simultaneously from addresses 10.91.208.51 and 10.91.208.61.  Breaking device connection.

12:16:41:582 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40002-RIP10.91.208.51) succeeded

12:16:41:587 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40017-RIP10.91.208.61) succeeded

12:16:41:782 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40003-RIP10.91.208.61) succeeded

12:16:41:784 ra-ccag Device PG01 connected simultaneously from addresses 10.91.208.51 and 10.91.208.61.  Breaking device connection.

12:16:52:909 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40002-RIP10.91.208.51) succeeded

12:16:52:914 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40017-RIP10.91.208.61) succeeded

12:16:53:113 ra-ccag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-LPORT-40003-RIP10.91.208.61) succeeded

12:21:17:597 ra-ccag Device PG01 connected simultaneously from addresses 10.91.208.51 and 10.91.208.61.  Breaking device connection.

...................

12:21:27:589 ra-ccag The Network communications between ICM Router and Peripheral Gateway or NIC PG01 has been down for 5 minutes.

12:21:27:590 ra-ccag The Network communications between ICM Router and Peripheral Gateway or NIC PG02 has been down for 5 minutes.

pg1a - pgag log:

Events from January 30, 2013:

12:16:26:169 PG1A-pgag Initializing Event Management System (EMS) Library.

12:16:26:169 PG1A-pgag Trace: EMS Server pipe uicm\PG1A\pgagEMSPipe enabled for uicm\PG1A\pgag

12:16:26:169 PG1A-pgag Initializing Node Manager Library.

12:16:26:169 PG1A-pgag Trace: Monitor Server pipe uicm\PG1A\pgagCmdPipe enabled for uicm\PG1A\pgag

12:16:26:185 PG1A-pgag Trace: EMT I/O completion ports: max threads=4, concurent threads=0

12:16:26:216 PG1A-pgag DMP Agent process starting.  Release 8.5.4.0 , Build 275.

12:16:26:231 PG1A-pgag Trace: MBufLimitMgr OpenMutex Global\MBUF Limit Table Mutex

12:16:26:450 PG1A-pgag Connection to MDS process established.

12:16:26:715 PG1A-pgag MDS is in service.

12:16:27:963 PG1A-pgag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-ID-1-RPORT-40002-RIP10.91.208.50) succeeded

12:16:27:963 PG1A-pgag Connection to Central Controller side A established (low priority).

12:16:27:963 PG1A-pgag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-ID-1-RPORT-40017-RIP10.91.208.60) succeeded

12:16:27:963 PG1A-pgag Connection to Central Controller side A established (medium priority).

12:16:29:835 PG1A-pgag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-ID-1-RPORT-40003-RIP10.91.208.60) succeeded

12:16:29:835 PG1A-pgag Connection to Central Controller side A established (high priority).

12:16:31:317 PG1A-pgag DMP Agent process activated.

12:16:31:317 PG1A-pgag Path to Central Controller side A established.

12:16:31:317 PG1A-pgag Connection to Central Controller side A failed (low priority).

  Last EMT Error [64]: The specified network name is no longer available.

12:16:31:317 PG1A-pgag Connection to Central Controller side A failed (high priority).

  Last EMT Error [64]: The specified network name is no longer available.

12:16:31:317 PG1A-pgag Connection to Central Controller side A failed (medium priority).

  Last EMT Error [64]: The specified network name is no longer available.

12:16:31:317 PG1A-pgag Path to Central Controller side A disconnected.

12:16:41:707 PG1A-pgag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-ID-1-RPORT-40002-RIP10.91.208.50) succeeded

12:16:41:707 PG1A-pgag Connection to Central Controller side A established (low priority).

12:16:41:722 PG1A-pgag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-ID-1-RPORT-40017-RIP10.91.208.60) succeeded

12:16:41:722 PG1A-pgag Connection to Central Controller side A established (medium priority).

12:16:41:722 PG1A-pgag Trace: ICM_QoS_Policies::DeletePolicy(CISCO-ICM-QOS-ID-1-RPORT-40003-RIP10.91.208.60) succeeded

12:16:41:722 PG1A-pgag Connection to Central Controller side A established (high priority).

12:16:41:800 PG1A-pgag Path to Central Controller side A established.

12:16:41:800 PG1A-pgag Connection to Central Controller side A failed (low priority).

  Last EMT Error [64]: The specified network name is no longer available.

12:16:42:643 PG1A-pgag Connection to Central Controller side A failed (high priority).

  Last EMT Error [64]: The specified network name is no longer available.

12:16:42:643 PG1A-pgag Connection to Central Controller side A failed (medium priority).

  Last EMT Error [64]: The specified network name is no longer available.

12:16:42:643 PG1A-pgag Path to Central Controller side A disconnected.

......................................................

cg1a - ctisvr log:

Events from January 30, 2013:
12:16:44:624 cg1A-ctisvr Initializing Event Management System (EMS) Library.
12:16:44:624 cg1A-ctisvr Trace: EMS Server pipe uicm\CG1A\ctisvrEMSPipe enabled for uicm\CG1A\ctisvr
12:16:44:624 cg1A-ctisvr Trace: Monitor Server pipe uicm\CG1A\ctisvrCmdPipe enabled for uicm\CG1A\ctisvr
12:16:44:624 cg1A-ctisvr Initializing Node Manager Library.
12:16:44:624 cg1A-ctisvr CTIServer Release 8.5.4.0 , Build 275, Built 09/05/12 09:57:10
12:16:44:655 cg1A-ctisvr Trace: Perfmon Initialization for the instanceName = uicm CG1A
12:16:44:655 cg1A-ctisvr Trace: Initialize of Perfmon manager successful
12:16:44:655 cg1A-ctisvr Trace: EMT I/O completion ports: max threads=4, concurent threads=0
12:16:44:671 cg1A-ctisvr Trace: MBufLimitMgr OpenMutex Global\MBUF Limit Table Mutex
12:16:44:671 cg1A-ctisvr Connection to MDS process established.
12:16:44:671 cg1A-ctisvr MDS now in service.
12:16:44:858 cg1A-ctisvr ProcessSetIdleReq: Enterprise CTI Server going idle.
12:16:44:858 cg1A-ctisvr PG System Event: IdleCTIServer, PG Status: CENTRAL CONTROLLER OFFLINE
12:16:44:858 cg1A-ctisvr Enterprise CTI Server sending OPC CTI_OK_ACK acknowledgment for command CTI_SET_IDLE_REQ (TransID=1).
12:16:44:858 cg1A-ctisvr ICM Central Controller time adjusted by 0 second(s) to 01:16:44 UTC.
12:30:00:720 cg1A-ctisvr CTIServer IDLE, 0 CTI Client connection(s), 0 call(s) in progress, PG Status: CENTRAL CONTROLLER OFFLINE

..............................................

Can someone help please?

[Gergely Szabo · Accepted solution]

Hi,

try inserting the same IP addresses for "High" for both public and private interfaces. Does the error message still appear?

G.

[Ping Liu]

thanks for your reply.

i tried your suggestion, it works now.

i remember that we used one ip address for both interfaces at begining, it was not working. may other change i made in server also helps.

anyway, it is working now. thank you for your help.

[Ping Liu]

Even one IP for both of High/Normal works fine, but there is better configuration for UCCE 8.5:

1. Use separate IP address for Normal/High on both of Private/Visible link in Rogger;

2. Use separate IP address for Normal/High on Private link in PG, and ONLY one IP for Visible link in PG;

This configuration was working fine for my UCCE 8.5.4.

Thanks

[iptuser55]

With regards the single IP address in a PG, when was this changed to a single approach?

We have taken over a UCCE cluster 10.x with the two IP approach which we had the above issue. We have now removed the second IP and so hopefully it will resolve our issues going forward.

Question

What starts the issue in the first place, our cluster has been with two IP address but only recently have we had the issue so I`m wondering what starts it

thanks

[spaulin · Accepted solution]

Here is your problem:

"PG01 connected simultaneously from addresses 10.91.208.51 and 10.91.208.61.  Breaking device connection."

A PG should not have a "high priority" address.  Only one visible address is supported.

[Ping Liu]

Thanks,

You were right.

You can have two IP addresses for both of Private/Visible link on Rogger, plus two IP addresses for Private link on PG. But you only can have one IP address for Visible link on PG.