Cisco ISE 3.0 Application Server stuck in 'Initializing' state

[laurathaqi]

Dear community, 

I deployed 2x Cisco ISE 3.0 Small VMs. During the process of adding them in a High availability state, ISE2 was found in the deployment tab in the ISE1 Window screen. However, when the process continued to the restart stage as per the usual registration process, ISE1 application server is stuck in the "Initializing" state, meanwhile ISE2 restarted successfully. 

Information for the current setup:

1. Application stop, start, reload, VM restart did not change the problem state.

2. Hardware Resources are based on best practices of: 32GB RAM, 16CPU and 1200GB Disk. 

3. Solution in VMware based. 

4. DNS, NTP were synchronized and working correctly. 

5. ISE 1 has ping to ISE 2 and vice versa. Both of them do have access to the Internet, as I synchronized with the CSSM and registered the licenses successfully for both of the VMs.

Do you guys have any idea why this is happening? 

Looking forward to hearing your thoughts on this issue and what might be causing the issue.

Your support is highly welcome and appreciated. 

Thank you,

Laura  

[Marcelo Morais · Accepted solution]

Hi @laurathaqi 

 I have no idea what causes this issue, but try to:

1. Deregister ISE2 (ISE1 and ISE2 > Standalone)
2. On ISE1 (the ISE that has the initializing issue)
application reset-config
3. Check the Application Server state of ISE1
4. Register ISE1 with ISE2
5. Check the Application Server state again

Hope this helps !!!

[laurathaqi]

Hi @Marcelo Morais 

I will try this tomorrow early and let you know about the progress. 

I have one question though. The command of application reset-config will delete all the configurations done so far in the ISE node with the noted issue right? So far I have imported the Root and Subordinate Certificates and have done some base configurations. Would like to know if that will delete all, so I can take it into consideration to schedule time for redoing these tasks also!

ISE knows how to be weird sometimes  

Thank you for your highly valuable support. 

Best,

Laura  

[Marcelo Morais]

Hi @laurathaqi ,

 yes, the command will delete the configurations ... please export your ISE1's Certificate.

Note: as soon as you register the ISE1 with ISE2, ISE2 will push the configuration (sync) to ISE1.

Regards,

[laurathaqi]

Hi @Marcelo Morais 

Hope you are doing well. 

The information shared is highly appreciated by my side. 

I restarted the whole VM from the ESXI console and when checked today the Application Server was in the running status. I am not sure what caused the issue but the only damage done was that ISE2 was not registered anymore as a Secondary role even though it was showing in the Admin panel of ISE1. Meanwhile when login in ISE2 admin console, it was showing as Primary itself.

Now as I imported the VM Smart Licenses before the issue happened, there were two licenses in use status. However, since the registration of ISE2 was not correct, I deregistered the node, and registered it again. And the registration went all fine. But the issue was the VM license, since it was showing that I was using three licensees now instead of two. Meanwhile I only have two ISE instances running, and only two VM licenses in the Smart Account Manager. Smart Account Manager shows alert that I am overusing VM licenses.

I am getting an error of OutofCompliance warning in ISE with a meaning that I am overusing VM licenses, but in real I am not as I only have two VM instances deployed. I think this happened because of the issues during ISE2 node deregistration and re registration. It could be that ISE 2 is being counted as two VMs as I registered is twice in the Admin console of ISE1. 

I tried to troubleshoot from CSSM side, and also from ISE side but no success. I opened a TAC case, and hopefully will be getting information in regards. If you have had such a case, or know about this issue, sharing would be highly appreciated.

NOTE: ISE1 was meant to be Primary having Active Admin panel and PSN as active services. Meanwhile ISE2 was meant to be Secondary in Admin Panel service, Primary in Monitoring Service and Active in PSN. 

I will share information as soon as I get progress on the issue.

Best wishes and thank you, 

Laura 

[Marcelo Morais · Accepted solution]

Hi @laurathaqi 

 I had the same issue a couple of months ago:

"... I am getting an error of OutofCompliance warning in ISE with a meaning that I am overusing VM licenses, but in real I am not as I only have two VM instances deployed. I think this happened because of the issues during ISE2 node deregistration and re registration..."

 TAC had to remove the "stuck license" on CSSM and we had to generate a New Token for a New Registration.

Hope this helps !!!

[laurathaqi]

Hi @Marcelo Morais

This information just made my day :). Many many thanks. 

Best wishes, 

Laura