PG server get shutdown when PG services are started

[navingolani]

Hi,

PFA screen shot for the error. We have Side A/ B Sprawler's. Now as soon as we start the PG service for one of the VRU PG it throws ICM NM shutdown message. I have seen the configurations, ran PG setup & other IP settings(pub & private) as well all look good. In my past I had seen this message but after correcting the configurations it use to resolve the issue.

Also, when compared the registry settings for mds process under NM, I found that Shutdowndirty value is set to 1. I tried with 0 for that, but no help. After starting PGservice with 0 It again throws the same error. Once I initiate the stopshut command, it will stop the PG service & will make Shutdowndirty value back to 1 from 0.

This is the behavior on both A & B side with mdmp file created. Development team can be helpful to read the crashed code, but if any one with similar experience?

Can you kindly advise if any thing that's missing & need to be validated.

Regards,

Navin Golani

[Bill Mungaven]

A couple of weeks ago I rebooted both my primary and backup PGs after installing Microsoft patches and got this message on the backup PG. I had to go to a command prompt and issue the stopshut command. TAC couldn't tell me why the PG is shutting down. Without the stopshut utility, the backup server continues rebooting.

I can't provide any further assistance but I'm also having this problem.

Bill

[navingolani]

I am sure some thing some where some value is conflicting.

Fail: State Transfer Overseer EMTListen failed.
  Last API Error [10048]: Only one usage of each socket address (protocol/network address/port) is normally permitted.
12:36:20:616 PG3B-mds Fail: MDS Overseer EMTListen failed.

  Last MDS Error [-519815419]: Client is already registered.
12:36:38:268 PG3B-nm Trace: Mutex released ...
12:36:38:268 PG3B-nm Trace:   Wait returned for process mds pid 2d74 handle 300 wait handle 300
12:36:38:268 PG3B-nm Process mds on ICM\XYZ\PG3B went down for unknown reason. Exit code 0x80000003. It will be automatically restarted.

[david.macias]

Can you post the complete startup log leading up to the crash?

david

[david.macias]

Can you re-run setup and see if the shutdown on error is set?

david

[navingolani]

Did you meant PG setup or Websetup ?

PG setup won't have such options.

However, in websetup I have 'Reboot Machine on Error ' as disabled which is under Router options and logger additional opt.

[Omar Deen]

Have you added persistent routes? If so, have you doubled checked that they are absolutely correct? You can tell they're good by viewing packets sent and received.

[navingolani]

Hello Omar,

Yes, persistent routes are there & correct. Also Packet Sent/receive look good.

One thing I wanted to know here is : do we have limitation on Sprawler for not having more than 2 PG per server? So if 3rd PG is configured on the server it will not allow ? 

Per Cisco Doc, I noticed that below thing. But I am not sure if it say specifically for PG types ( max2 per server , e.g CUCM , VRU,MR) or counts of PG ( max 2)

Maximum number of PG types per server platform  ::  Up to two PG types are permitted per server, provided that any given server is limited to the maximum agent

[piyush aghera]

Hi All,

i am also facing this issue on sprawler, though i just have one PG per server.  So i think this is not related to any limitation but something else is causing this issue.

[navingolani]

Check for peripheral id/Logical Id value on PGsetup, also persistent route as part of diff VLAN,  Verify NIC values.

I am having 3 pg on the sprawler and becoz of limitation, it had the port conflict on either side pg, hence PG's were getting shutdown.  But as you are having only one pg per server, then check for above config.