UCCE PG connect to CVP problem

[xiaosen weng]

Duplex side UCCX, version is 8.4

Side A Public IP : 192.168.1.46; Private IP: 172.16.12.6  , It runs routerA,loggerA,PG1A,PG2A
Side A Public IP : 192.168.1.47; Private IP: 172.16.12.7  , It runs routerB,loggerB,PG1B,PG2B

PG2 connect to 2 CVP call server, the ucce and cvp all run's good for 8 years.

and yesterday I found that PG2 reports some problem, it cann't connect with CVP very well, I don't konw why,

the problem and configure shows as attach, any one can help me?

[xiaosen weng]

another attach 

[xiaosen weng]

new logs as follows

pgag_log:

Events from June 18, 2020:
15:34:24:526 PG2A-pgag Initializing Event Management System (EMS) Library.
15:34:24:526 PG2A-pgag Trace: EMS Server pipe msh\PG2A\pgagEMSPipe enabled for msh\PG2A\pgag
15:34:24:526 PG2A-pgag Initializing Node Manager Library.
15:34:24:526 PG2A-pgag Trace: Monitor Server pipe msh\PG2A\pgagCmdPipe enabled for msh\PG2A\pgag
15:34:24:526 PG2A-pgag Trace: EMT I/O completion ports: max threads=4, concurent threads=0
15:34:24:542 PG2A-pgag DMP Agent process starting. Release 8.5.4.0 , Build 275.
15:34:24:542 PG2A-pgag Trace: MBufLimitMgr OpenMutex Global\MBUF Limit Table Mutex
15:34:24:964 PG2A-pgag Connection to MDS process established.
15:34:24:964 PG2A-pgag Connection to Central Controller side A established (low priority).
15:34:24:964 PG2A-pgag Connection to Central Controller side A established (high priority).
15:34:24:964 PG2A-pgag Connection to Central Controller side A established (medium priority).
15:34:25:308 PG2A-pgag Connection to Central Controller side B established (low priority).
15:34:25:308 PG2A-pgag Connection to Central Controller side B established (medium priority).
15:34:25:308 PG2A-pgag Connection to Central Controller side B established (high priority).
15:34:54:667 PG2A-pgag Trace: TOS requested but no TOS connection exists. Assuming unreachable.
15:34:54:667 PG2A-pgag Trace: TOS requested but no TOS connection exists. Assuming unreachable.
15:34:54:667 PG2A-pgag MDS is in service.
15:35:00:261 PG2A-pgag DMP Agent process activated.
15:35:00:261 PG2A-pgag Connection to Central Controller side A failed (low priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:00:261 PG2A-pgag Connection to Central Controller side A failed (high priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:00:261 PG2A-pgag Connection to Central Controller side B failed (low priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:00:261 PG2A-pgag Connection to Central Controller side A failed (medium priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:00:261 PG2A-pgag Connection to Central Controller side B failed (medium priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:00:261 PG2A-pgag Connection to Central Controller side B failed (high priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:10:370 PG2A-pgag Connection to Central Controller side A established (high priority).
15:35:10:370 PG2A-pgag Connection to Central Controller side A established (low priority).
15:35:10:370 PG2A-pgag Connection to Central Controller side A failed (high priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:10:370 PG2A-pgag Connection to Central Controller side A established (medium priority).
15:35:10:370 PG2A-pgag Connection to Central Controller side A failed (low priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:10:464 PG2A-pgag Connection to Central Controller side A failed (medium priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:10:714 PG2A-pgag Connection to Central Controller side B established (low priority).
15:35:10:714 PG2A-pgag Connection to Central Controller side B failed (low priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:10:714 PG2A-pgag Connection to Central Controller side B established (medium priority).
15:35:10:714 PG2A-pgag Connection to Central Controller side B established (high priority).
15:35:10:714 PG2A-pgag Connection to Central Controller side B failed (medium priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:10:714 PG2A-pgag Connection to Central Controller side B failed (high priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:20:464 PG2A-pgag Connection to Central Controller side A established (high priority).
15:35:20:464 PG2A-pgag Connection to Central Controller side A established (low priority).
15:35:20:464 PG2A-pgag Connection to Central Controller side A failed (high priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:20:464 PG2A-pgag Connection to Central Controller side A failed (low priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:20:573 PG2A-pgag Connection to Central Controller side A established (medium priority).
15:35:20:573 PG2A-pgag Connection to Central Controller side A failed (medium priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:21:058 PG2A-pgag Connection to Central Controller side B established (medium priority).
15:35:21:058 PG2A-pgag Connection to Central Controller side B established (low priority).
15:35:21:058 PG2A-pgag Connection to Central Controller side B established (high priority).
15:35:21:058 PG2A-pgag Connection to Central Controller side B failed (medium priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:21:073 PG2A-pgag Connection to Central Controller side B failed (low priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:21:073 PG2A-pgag Connection to Central Controller side B failed (high priority).
Last EMT Error [64]: The specified network name is no longer available.
15:35:30:573 PG2A-pgag Connection to Central Controller side A established (high priority).
15:35:30:573 PG2A-pgag Connection to Central Controller side A established (low priority).
15:35:30:573 PG2A-pgag Connection to Central Controller side A failed (high priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:30:573 PG2A-pgag Connection to Central Controller side A failed (low priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:30:667 PG2A-pgag Connection to Central Controller side A established (medium priority).
15:35:30:667 PG2A-pgag Connection to Central Controller side A failed (medium priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:31:417 PG2A-pgag Connection to Central Controller side B established (low priority).
15:35:31:417 PG2A-pgag Connection to Central Controller side B established (medium priority).
15:35:31:417 PG2A-pgag Connection to Central Controller side B established (high priority).

mds_log

Events from June 18, 2020:
15:34:24:573 PG2A-mds Initializing Event Management System (EMS) Library.
15:34:24:573 PG2A-mds Trace: EMS Server pipe msh\PG2A\mdsEMSPipe enabled for msh\PG2A\mds
15:34:24:573 PG2A-mds Initializing Node Manager Library.
15:34:24:573 PG2A-mds Trace: Monitor Server pipe msh\PG2A\mdsCmdPipe enabled for msh\PG2A\mds
15:34:24:573 PG2A-mds Trace: EMT I/O completion ports: max threads=8, concurent threads=0
15:34:24:604 PG2A-mds Trace: MBufLimitMgr OpenMutex Global\MBUF Limit Table Mutex
15:34:24:604 PG2A-mds MDS Process starting. Release 8.5.4.0 , Build 275.
15:34:24:604 PG2A-mds Trace: AppRTT set in WATCHING mode
15:34:24:604 PG2A-mds Trace: MDSPROC - CreateMutex - Global\MDS Side A Clock Master Mutex
15:34:24:604 PG2A-mds Synchronizer initializing for duplex operation.
15:34:24:964 PG2A-mds Client nm registered with handle 41.
15:34:24:964 PG2A-mds Client nm started.
15:34:24:964 PG2A-mds Client pgag registered with handle 48.
15:34:24:964 PG2A-mds Client pgag started.
15:34:24:964 PG2A-mds Client pim1 registered with handle 33.
15:34:24:964 PG2A-mds Client pim2 registered with handle 34.
15:34:24:964 PG2A-mds Client pim1 started.
15:34:24:964 PG2A-mds Client pim2 started.
15:34:24:964 PG2A-mds Client opc registered with handle 1.
15:34:24:964 PG2A-mds Client opc started.
15:34:26:558 PG2A-mds Client tsyp registered with handle 65.
15:34:26:558 PG2A-mds Client tsyp started.
15:34:54:667 PG2A-mds Initiating test of peer Synchronizer.
15:34:54:667 PG2A-mds Trace: Sending TOS request: sequence = 1.
15:34:54:667 PG2A-mds Trace: Received TOS response: sequence=1 status=UNREACHABLE.
15:34:54:667 PG2A-mds Trace: Sending TOS request: sequence = 2.
15:34:54:667 PG2A-mds Trace: Received TOS response: sequence=2 status=UNREACHABLE.
15:34:54:667 PG2A-mds Peer Synchronizer was found to be unreachable.
15:34:54:667 PG2A-mds Synchronizer switching to non-duplex operation.
15:34:54:667 PG2A-mds MDS now in service.
15:35:24:667 PG2A-mds Synchronizer timed out trying to establish connection to peer.
15:38:04:042 PG2A-mds Client nm stopping normally.
15:38:04:042 PG2A-mds Trace: Node Manager thread received shutdown message
15:38:04:042 PG2A-mds Client pgag stopping normally.
15:38:04:042 PG2A-mds Client tsyp stopping normally.
15:38:04:136 PG2A-mds MDS going out of service.
15:38:04:136 PG2A-mds Client pim1 stopping normally.
15:38:04:136 PG2A-mds Client pim2 stopping normally.
15:38:04:245 PG2A-mds Client opc stopping due to error.
15:38:09:745 PG2A-mds Trace: Node Manager thread sending NM_MSG_SHUTDOWN_REPLY_OK to Node Manager.
15:38:09:745 PG2A-mds Trace: Node Manager thread completed clean shutdown dialog with Node Manager. Exiting process.

OPC_log

Events from June 18, 2020:
15:34:24:573 PG2A-opc Initializing Event Management System (EMS) Library.
15:34:24:573 PG2A-opc Trace: EMS Server pipe msh\PG2A\opcEMSPipe enabled for msh\PG2A\opc
15:34:24:573 PG2A-opc Trace: Starting OPC with Oats disabled, g.geoTelBaseDir = C:\icm\msh\pg2a
15:34:24:573 PG2A-opc Initializing Node Manager Library.
15:34:24:573 PG2A-opc Trace: Monitor Server pipe msh\PG2A\opcCmdPipe enabled for msh\PG2A\opc
15:34:24:573 PG2A-opc OPC Software Version Release 8.5.4.0 , Build 275.
15:34:24:573 PG2A-opc Trace: EMT I/O completion ports: max threads=4, concurent threads=0
15:34:24:589 PG2A-opc Trace: MBufLimitMgr OpenMutex Global\MBUF Limit Table Mutex
15:34:24:964 PG2A-opc Connection to MDS process established.
15:34:24:964 PG2A-opc Trace: GetInSync: Serialization Disabled.
15:34:24:964 PG2A-opc Trace: GetInSync: Synchronization holdoff disabled.
15:34:54:667 PG2A-opc MDS is in service.
15:34:54:667 PG2A-opc Current time: 06/18/2020 07:34:54.659.
15:34:54:667 PG2A-opc Local time: 06/18/2020 15:34:54.659.
15:34:54:667 PG2A-opc Midnight: 06/17/2020 16:00:00.000.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ActivationPDDelayTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CallWrapupDataTimeout set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ConfigurePDTimeout set to 30.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ConfigurePGTimeout set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ConfigureRCTimeout set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value DebugControlFlags1 set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value DebugControlFlags2 set to 268435456.
15:34:54:667 PG2A-opc PG\\OPC Registry Value RoutingClientMonitorTimeout set to 60.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PendingPreRoutedCallsMonitorTimeout set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value FailureMask set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value GenerationCounter set to 5.
15:34:54:667 PG2A-opc PG\\OPC Registry Value GenerationCounter set to 4.
15:34:54:667 PG2A-opc PG\\OPC Registry Value InitializePGAgentTimeout set to 30.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PeripheralMonitorTimeout set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PIMActiveTimeout set to 20.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PIMConfiguredTimeout set to 30.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PIMIdleRetryCount set to 2.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PIMIdleRetryDelayTime set to 60.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PIMIdleTimeout set to 20.
15:34:54:667 PG2A-opc PG\\OPC Registry Value PIMNullTimeout set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ReportUnknownTargetAssignments set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ReportUnknownEstablishedCalls set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ReportUnknownHeldCalls set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ReportUnknownQueuedCalls set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ReportUnknownRetrievedCalls set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ReportPersistentStorageUpdates set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value TranslationRouteTimeout set to 60.
15:34:54:667 PG2A-opc PG\\OPC Registry Value NodeManagerPingTimeChecks set to 4.
15:34:54:667 PG2A-opc PG\\OPC Registry Value DeactivatePIMForInvalidHalfHour set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ConfigErrorsSuppressionMinutes set to 60.
15:34:54:667 PG2A-opc PG\\OPC Registry Value ErrorsSuppressDuplicates set to TRUE.
15:34:54:667 PG2A-opc PG\\OPC Registry Value DateTimeForHistoricalDataRecovery set to MM/DD/YYYY/HH/MM.
15:34:54:667 PG2A-opc PG\\OPC Registry Value DateTimeForHistoricalDataRecovery set to MM/DD/YYYY/HH/MM.
15:34:54:667 PG2A-opc PG\\OPC Registry Value LegacyPreCallMonitorTimeout set to 300.
15:34:54:667 PG2A-opc PG\\OPC Registry Value InitializationFlowControlThreshold set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value DisableForcedLogoutReasonCodeOnPIMOutofService set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value MDSMessageClassMetering set to 0.
15:34:54:667 PG2A-opc Trace: Registry MethodForMatchRouteID set to 0
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\MaxNumTCDBuffered set to 5.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\MaxNumAgentLogoutBuffered set to 100.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\MaxNumAgentSkillGroupLogoutBuffered set to 100.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\MaxNumAgentStateTracesBuffered set to 80.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\TCDFlushTimeout set to 3.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\AgentLogoutFlushTimeout set to 120.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\AgentSGLogoutFlushTimeout set to 120.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\AgentStateTraceFlushTimeout set to 120.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\TCDForInternalCalls set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\PassThruFlushTimeout set to 16.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\SkillGroupDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\ServiceDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\RouteDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\TrunkGroupDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\PeripheralDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\ControllerDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\NetworkTrunkGroupDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\RTAgentDataFlushTimeout set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\SimulationResetsCreateHHRecords set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\PeripheralTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\SkillGroupTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\ServiceTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\RouteTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\TrunkGroupTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\NetworkTrunkGroupTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\AgentSkillGroupTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\AgentTmpFileFrequency set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\EnableTCDHistFile set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\RTTmpFileFrequency set to 20.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\DiskQueueThreshold set to 60.
15:34:54:667 PG2A-opc Trace: !Setting CTIAgentEventFlushTimeoutInMilliSeconds to 0 disables agent-event buffering.
15:34:54:667 PG2A-opc Trace: ! This may have an adverse effect on the network bandwith between CTI Server and OPC.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\CTIAgentEventFlushTimeoutInMilliSeconds set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\AgentAvailableStateUpdateDelayInMilliSeconds set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\OPIData\PGHHDelay set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureDataFileMaxBytes set to 10000000.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureDataFileTotalMaxBytes set to 2000000000.
15:34:54:667 PG2A-opc Trace: Capture was enabled at 15:34:54
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureMessages set to 1.
15:34:54:667 PG2A-opc Trace: Output Capture was disabled at 15:34:54
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureOutput set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureMDSClockTick set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureCompression set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureCompressionCommand set to .
15:34:54:667 PG2A-opc PG\\OPC Registry Value EnableOATS set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OATSBlockID set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureZipCompressionEnabled set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureZipFormat set to gzip -f -q .
15:34:54:667 PG2A-opc PG\\OPC Registry Value CaptureZipExtension set to .gz.
15:34:54:667 PG2A-opc PG\\PIMS\\pim1\\GenericData Registry Value PeripheralID set to 5002.
15:34:54:667 PG2A-opc PG\\PIMS\\pim1\\GenericData Registry Value PeripheralEnabled set to 1.
15:34:54:667 PG2A-opc PIM1 PeripheralDisabled set to 1 for unknown peripheral 5002.
15:34:54:667 PG2A-opc PG\\PIMS\\pim1\\GenericData Registry Value ClientType set to 13.
15:34:54:667 PG2A-opc PG\\PIMS\\pim1\\GenericData Registry Value ParentPeripheralID set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\ARSPeripheral set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\MapPeripheralTargetsWithoutTrunkGroup set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\MaxQueueTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\MaxNullTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\MaxAlertingTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\MaxConnectedTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\MaxClearedCallTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\DeliveredEventsRestartsServiceLevelTimer set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\TransferEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\EstablishedEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\QueuedEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\ConferenceEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\CallsOfferedSumOfCompleted set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\CountOutboundAgentCallsHandled set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\DeliveredEventsRemoveQueuedSkillGroups set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\AlertingEventsRemoveQueuedSkillGroups set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\DeliveredEventsResetASACalculation set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\DontTranslationRouteConfiguredDNs set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\TerminateCallsOnPIMRestarts set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\TransferEventsUsesExistingNewCalls set to 0.
15:34:54:667 PG2A-opc Trace: OPC_CCMaxRouterCallKeyObjDuration:: Ignore setting OPC\CallControl\pim1\MaxRouterCallKeyObjDuration with 21600, value must be larger then MaxConnectedTimeAllowedForCall setting of 43200
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\DisableTrunkReuseCheck set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\NewCallOffersUpdateDNIS set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\TakeBacknXferDigitDelayMs set to 500.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\AgentDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\RNATCDHoldOffTimeoutInSeconds set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\IgnoreNotConfigSG set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\OptimizeAgentStatesToRouter set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\EnableACDTimeInterval set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\DumpStateXfer set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim1\HandleRNATimeInSeconds set to 10.
15:34:54:667 PG2A-opc PG\\PIMS\\pim2\\GenericData Registry Value PeripheralID set to 5003.
15:34:54:667 PG2A-opc PG\\PIMS\\pim2\\GenericData Registry Value PeripheralEnabled set to 1.
15:34:54:667 PG2A-opc PIM2 PeripheralDisabled set to 1 for unknown peripheral 5003.
15:34:54:667 PG2A-opc PG\\PIMS\\pim2\\GenericData Registry Value ClientType set to 13.
15:34:54:667 PG2A-opc PG\\PIMS\\pim2\\GenericData Registry Value ParentPeripheralID set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\ARSPeripheral set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\MapPeripheralTargetsWithoutTrunkGroup set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\MaxQueueTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\MaxNullTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\MaxAlertingTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\MaxConnectedTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\MaxClearedCallTimeAllowedForCall set to 43200.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\DeliveredEventsRestartsServiceLevelTimer set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\TransferEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\EstablishedEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\QueuedEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\ConferenceEventsAssignNewTarget set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\CallsOfferedSumOfCompleted set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\CountOutboundAgentCallsHandled set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\DeliveredEventsRemoveQueuedSkillGroups set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\AlertingEventsRemoveQueuedSkillGroups set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\DeliveredEventsResetASACalculation set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\DontTranslationRouteConfiguredDNs set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\TerminateCallsOnPIMRestarts set to 1.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\TransferEventsUsesExistingNewCalls set to 0.
15:34:54:667 PG2A-opc Trace: OPC_CCMaxRouterCallKeyObjDuration:: Ignore setting OPC\CallControl\pim2\MaxRouterCallKeyObjDuration with 21600, value must be larger then MaxConnectedTimeAllowedForCall setting of 43200
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\DisableTrunkReuseCheck set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\NewCallOffersUpdateDNIS set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\TakeBacknXferDigitDelayMs set to 500.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\AgentDataSyncTime set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\RNATCDHoldOffTimeoutInSeconds set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\IgnoreNotConfigSG set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\OptimizeAgentStatesToRouter set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\EnableACDTimeInterval set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\DumpStateXfer set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value OPC\CallControl\pim2\HandleRNATimeInSeconds set to 10.
15:34:54:667 PG2A-opc PG\\OPC Registry Value SOFTWARE\Cisco Systems, Inc.\ICM\msh\PG2A\MisEnabled set to 0.
15:34:54:667 PG2A-opc PG\\OPC Registry Value SOFTWARE\Cisco Systems, Inc.\ICM\msh\PG2A\MisEnabled\VRUServiceControlQueueReporting set to 0.
15:34:54:667 PG2A-opc Incorrect arguments to EMS wrapper function (EMSTrace, EMSFail). Message id was 0x21010003. Format string was "Capture Dir is %s"
15:34:54:667 PG2A-opc Trace: Capture: couldn't stat directory OPCcap\20200618_153454 in current directory. Attempting to create ...
Last C Runtime Error [2]: No such file or directory
15:34:54:667 PG2A-opc Trace: Capture: successfully created directory OPCcap\20200618_153454 in current directory.
15:34:54:808 PG2A-opc Trace: Saved PG registy .
15:34:54:808 PG2A-opc Trace: Perfmon Initialization for the instanceNumber = 2, msh PG2A
15:34:54:808 PG2A-opc Trace: Perfmon Counters initialised
15:34:55:167 PG2A-opc Trace: Capturing the first MDS Clock Tick
15:35:00:261 PG2A-opc Sending initialize PG Agent request - ActiveState=0 Present=1,0 Side=A.
15:35:00:261 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:35:00:261 PG2A-opc SideA pgag process is OK.
15:35:00:417 PG2A-opc SideA pim1 process is OK.
15:35:00:464 PG2A-opc SideA pim2 process is OK.
15:35:30:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:35:30:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:35:30:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:36:00:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:36:00:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:36:00:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:36:30:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:36:30:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:36:30:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:37:00:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:37:00:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:37:00:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:37:30:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:37:30:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:37:30:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:38:00:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:38:00:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:38:00:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:38:04:042 PG2A-opc Trace: Node Manager thread received shutdown message
15:38:04:245 PG2A-opc MDS is now OUT OF service.
15:38:04:245 PG2A-opc Trace: Reset of Perfmon Counters Succeeded
15:38:04:245 PG2A-opc Requesting MDS termination due to error.
15:38:04:745 PG2A-opc Connection to MDS process closed.

[Ritesh Desai]

Hi @xiaosen weng  Request and suggestion to avoid posting logs in message body. Better is to attach the files.

+++PGAG Logs+++

I see establish and fail error messages to Side A and Side B controller. verify the hostname / IP address configured in pgsetup.exe

15:35:00:261 PG2A-pgag Connection to Central Controller side A failed (low priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:00:261 PG2A-pgag Connection to Central Controller side A failed (high priority).
Last EMT Error [-519897068]: The connection has been closed by the remote process.
15:35:00:261 PG2A-pgag Connection to Central Controller side B failed (low priority).
Last EMT Error [64]: The specified network name is no longer available.

++ MDS LOGS++

from the logs the private network is unreachable. 

15:34:54:667 PG2A-mds Initiating test of peer Synchronizer.
15:34:54:667 PG2A-mds Trace: Sending TOS request: sequence = 1.
15:34:54:667 PG2A-mds Trace: Received TOS response: sequence=1 status=UNREACHABLE.
15:34:54:667 PG2A-mds Trace: Sending TOS request: sequence = 2.
15:34:54:667 PG2A-mds Trace: Received TOS response: sequence=2 status=UNREACHABLE.
15:34:54:667 PG2A-mds Peer Synchronizer was found to be unreachable.

Check this URL  The same error "The specified network name is no longer available." the issue was resolved after changes in network. This also could be the reason if the PG-A Setup configuration is same as PG-B.

+++ OPC Logs +++

Highlighted in black, i assume changes has happened. May be incorrect configuration led to MDS out of sync.

15:34:54:667 PG2A-opc PG\\OPC Registry Value SOFTWARE\Cisco Systems, Inc.\ICM\msh\PG2A\MisEnabled\VRUServiceControlQueueReporting set to 0.
15:34:54:667 PG2A-opc Incorrect arguments to EMS wrapper function (EMSTrace, EMSFail). Message id was 0x21010003. Format string was "Capture Dir is %s"
15:34:54:667 PG2A-opc Trace: Capture: couldn't stat directory OPCcap\20200618_153454 in current directory. Attempting to create ...
Last C Runtime Error [2]: No such file or directory
15:34:54:667 PG2A-opc Trace: Capture: successfully created directory OPCcap\20200618_153454 in current directory.
15:34:54:808 PG2A-opc Trace: Saved PG registy .
15:34:54:808 PG2A-opc Trace: Perfmon Initialization for the instanceNumber = 2, msh PG2A
15:34:54:808 PG2A-opc Trace: Perfmon Counters initialised
15:34:55:167 PG2A-opc Trace: Capturing the first MDS Clock Tick
15:35:00:261 PG2A-opc Sending initialize PG Agent request - ActiveState=0 Present=1,0 Side=A.
15:35:00:261 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:35:00:261 PG2A-opc SideA pgag process is OK.
15:35:00:417 PG2A-opc SideA pim1 process is OK.
15:35:00:464 PG2A-opc SideA pim2 process is OK.
15:35:30:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:35:30:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:35:30:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:36:00:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:36:00:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:36:00:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:36:30:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:36:30:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:36:30:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:37:00:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:37:00:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:37:00:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:37:30:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:37:30:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:37:30:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:38:00:167 PG2A-opc Timeout initializing PG Agent, Activate=3.
15:38:00:167 PG2A-opc Sending initialize PG Agent request - ActiveState=3 Present=1,0 Side=A.
15:38:00:167 PG2A-opc Trace: OPCCtrlBlock::CalculateWeightOnBothSides - Updated weights SideA = 0, SideB = 0.
15:38:04:042 PG2A-opc Trace: Node Manager thread received shutdown message
15:38:04:245 PG2A-opc MDS is now OUT OF service.
15:38:04:245 PG2A-opc Trace: Reset of Perfmon Counters Succeeded
15:38:04:245 PG2A-opc Requesting MDS termination due to error.
15:38:04:745 PG2A-opc Connection to MDS process closed.

1. I assume there is change in configuration.

2. there is possibility of wrong Logical ID in PG Setup. Check URL here the possibility.

3. Can you ping the private interfaces of both end with ip address and host names as well. verify the RTT response it should be <80 as per v12.0. Your version is too old not sure about RTT. Check SRND here Chapter 11 BW and latency requirements.

Additional recommendation is to upgrade the solution.

thanks ad regards,

Ritesh Desai

Please rate helpful post.

[xiaosen weng]

Thank you  Ritesh Desai

    I guess the problem is in pg setup, that I forgot to have "duplex mode" enabled a few days ago when I reconfigure the PG2, I will go to have a try. by the way,  if  I  complete reconfigure the PG2, Will  I need to reboot  the Rogger service ? 

[Ritesh Desai]

Hi @xiaosen weng

Good you identified the issue and it worked for you. You first make the change of SIMPLEX to DUPLEX and observe the PG2A PIM process shows you ACTIVE on SideA. Also while making the change enable the traces for PGAG, PIM, MDS, OPC. If the process does not come up you can fetch the debugs and attach here to look into it.

[xiaosen weng]

hi Ritesh Desai

   After I reconfigure and reboot the PG2A/2B, now PG2A seems working fine, thank you for your help. 

and now I find some warnings and  errors from the windows application, see attach, I don't know why, could you help me？

[Ritesh Desai]

Hi @xiaosen weng 

Monitor for few minutes and see if still the events are generated.

regards,

[xiaosen weng]

hi @Ritesh Desai 

    I am checking  it  now, still has errors and warnings,  see attach , I don't know what's the problem with UCCE, but the system seems run good.

[Ritesh Desai]

Hi @xiaosen weng 

Seems still the MDS is having issues. Go to icmdba utility, space used summary. Compare the Side A and Side B the configuration tables is same or different?

If you can attached pgag, mds, pim, opc logs from 1.46 and 1.47. If you can attach event viewer logs would be helpful.

[Mortaza Rohani]

Dear @Ritesh Desai,
I'm facing the same error (The specified network name is no longer available) and You mentioned that "This also could be the reason if the PG-A Setup configuration is same as PG-B."
Does it mean each CVP should be defined in only one PG?

From the below image, I understand that CVP must be connected to both PGs, is it wrong?

[xiaosen weng]

each PG must setup to connect both CVP. but only one PG is in active , another in standby.