Cisco ISE 3.1 After Patch Update Issue

[azman.mansor]

Dear Cisco Support,

We have 2 Cisco ISE 3.1 appliance.. Recently we have update using patch 7 update.

Once device ISE #1 has been updated we unable to access/view login page but able to ping that IP.

trying to ssh but looks like password does not work anymore to ssh.

We able to access ISE #2 device via browser but the configuration were difference and all setting looks was not there such a policies, configuration and etc..  trying to ssh to ISE #2 devices but same scenario occurred as device ISE #1.

Please help how can we access back our ISE

Thank You

Azman

[marce1000]

            >...trying to ssh but looks like password does not work anymore to ssh.
 - Can you clarify this observation to become exact , meaning do you get an authorization failure or something else ?

 M.

[azman.mansor]

Hi Marce,

Now i'm able to ssh to the #1 ISE using very very old password i've set long time ago. probably at the initial setup.

But i'm still not able to login using webpage browser. Below were patching i've been updated

Version information of installed applications
---------------------------------------------

Cisco Identity Services Engine
---------------------------------------------
Version : 3.1.0.518
Build Date : Tue Aug 10 04:28:55 2021
Install Date : Tue Jun 28 12:04:42 2022

Cisco Identity Services Engine Patch
---------------------------------------------
Version : 3
Install Date : Tue Jun 28 14:01:42 2022

Cisco Identity Services Engine Patch
---------------------------------------------
Version : 7
Install Date : Sun Jun 18 15:44:49 2023

[azman.mansor]

below is the application status running

ISE PROCESS NAME STATE PROCESS ID
--------------------------------------------------------------------
Database Listener running 8669
Database Server running 125 PROCESSES
Application Server running 23513
Profiler Database running 15353
ISE Indexing Engine not running
AD Connector running 25082
M&T Session Database running 4404
M&T Log Processor running 23760
Certificate Authority Service running 24912
EST Service running 42501
SXP Engine Service disabled
TC-NAC Service disabled
PassiveID WMI Service disabled
PassiveID Syslog Service disabled
PassiveID API Service disabled
PassiveID Agent Service disabled
PassiveID Endpoint Service disabled
PassiveID SPAN Service disabled
DHCP Server (dhcpd) disabled
DNS Server (named) disabled
ISE Messaging Service running 11468
ISE API Gateway Database Service running 14084
ISE API Gateway Service running 20317
Segmentation Policy Service disabled
REST Auth Service disabled
SSE Connector disabled
Hermes (pxGrid Cloud Agent) disabled

[marce1000]

            - Reboot the involved nodes , 

 M.

[azman.mansor]

Reboot the node won't effect the ISE indexing started again. application start stop didn't work as well. After open TAC with cisco found the ntp server not synchronize with the ISE. ISE engine were not running at all effected the GUI unable to access but still able to ping. We swing to the secondary ISE to allow the operation running again until we fix issue on the primary.

[MaErre21325]

I have a similar problem, after patching to patch 7 i'm no more able neither to ping, ssh or https my standalone Pan.

Do you have any tpis?

Thank you

[marce1000]

 - @MaErre21325 Rest assured the only thing you can do is reboot the standalone PAN , and check it's (health) state afterwards , the more important question (then) becomes : did you take a configuration backup before applying patch 7 (e.g), 

 M.

[MaErre21325]

Hi @marce1000,

no, i didn't take a backup...and i'm really worried about recreate every single policy

[marce1000]

 - @MaErre21325 , I am sorry but you always must make a backup before upgrades , 

 M.

[Aref Alsouqi]

Is this a VM or a hardware appliance? could you please check the console screen and share what it says?

[MaErre21325]

it's a vm and luckly from console all the services were up, just reloaded it and worked

thank you

[marce1000]

- Remember to take backups in the future and also take a (configuration)
backup >now< !

M.

[MaErre21325]

Yes, lesson learned!

[azman.mansor]

Hi Cisco support,

just want to share what we have done on the issue after patch update.

I've raise to TAC engineer support and and we troubleshoot the problem together. We found one of the symptom is ntp server unable to reach from the ISE itself. For temporary we swing to the secondary ISE to allow production run until we solve again issue on ntp server configuration. They tried to restart the application service and reboot the appliance. Even they have done it from the root but engine indexing still unable to start because of the ntp server communication.

Today we have a call session with Cisco again to solve issue on the primary. Will share the outcome then