Admin Certificate Renewal: Impact

[Tymofii Dmytrenko]

At the moment we have individual certificates deployed to each ISE node for Admin purpose. We also have wildcard certificate which we use for EAP Authentication and sponsors portal. The problem is that due to HSTS ISE PSN presents its own Admin certificate when redirection is being performed and hence portal cannot be accessed using portal's FQDN.

So, I decided to use Wildcard certificate for Admin function too, but when I Edited Wildcard certificate to set Admin as one of its functions, I've got the following warning

In my understanding, this means that I will not be able to manage or monitor ISE nodes while this activity takes place. However, is there anyone from Cisco who can confirm the exact impact? Is Authentication going to be affected? Does this happen simultaneously on all nodes in the deployment?

[Rob Ingram]

Hi,

It depends on how many nodes you are running in your environment and what services are running on those nodes.

For example, when the services restart on the PAN assuming the PSN service is running on another ISE node then you can still authenticate users, but you do lose the ability to process sessions that would require writing to the database. This link indicates exactly what is and is not available.

HTH

[Jason Kunst]

I checked on this and yes services will restart when replacing a cert

[Tymofii Dmytrenko]

@Jason Kunst are you referring to ALL services, or just Application Service (so, only GUI-based services will not be available, such as Management GUI, CWA, sponsors portals, BYOD portals, so on)? Thanks

[Jason Kunst]

All services should be affected because your cert applies to admin

[Tymofii Dmytrenko]

Thanks @Jason Kunst. How this activity may possibly affect Radius if that one is using different certificate (endpoint authentication)? Thanks

[hslai]

The restart of ISE services includes the session services (RADIUS and T+) regardless the EAP server using a different certificate. On the other hand, no ISE restart if only the EAP server certificate updated.

Please note that it will rolling restart ISE services on all the other ISE nodes, if we change the admin certificate on the primary ISE node.

[Tymofii Dmytrenko]

Oh... so this is a MAJOR change with a MASSIVE impact then? Oh my.

Thanks for confirming. At least I know what to expect now.

[hslai · Accepted solution]

Correct, on the impact of updating the admin certificate on the primary ISE node.

CSCut10928 is an enhancement and not yet implemented. If needed, you may use the workaround for CSCut10928 to reduce the impact.

[Damien Miller]

You assumed correct.  When you hit ok on that warning, the application server service will restart.  The entire server does not restart though, just one service.

You will not be able to access the gui while the the application server service restarts, and any feature leveraging the gui, such as sponsor guest account portal would be unavailable. 

Something to keep in mind is that the warning indicates all nodes including PSNs.  This means that if you are using the CWA portal for guest authentication on any other PSNs, that will also be unavailable for around 10-20 minutes.  

[Tymofii Dmytrenko]

@Damien Miller thanks! This is what I thought. I will raise a change to implement. At the moment we don't use ISE v2.3 for CWA, but this is ongoing project, so I'd better update certificate before it becomes even more critical.

I'll make sure the change happens close to end of day over the weekend just in case.

[Tymofii Dmytrenko]

I just thought I will give an update to everyone. So, we've swapped Admin certificate in our Cisco ISE v2.3 environment and I have observed the following.

• Only Application Service restarted, other services were not affected (inc Radius authentication)
• Service restarted on PAN, and on all other nodes ONE BY ONE as they are listed on the deployment page
• Impact was very low (we don't use ISE portals, only RADIUS/TACACS)

This contradicts with the information provided here. I had to go through a pain of raising this as a Major change and going through CAB :D

Anyway, happy there was no issues at all. Might be useful to anyone who wants to do the same

To summarize,

We had wildcard certificate used for EAP and Portals (sponsors), but NODE-specific certificates for ADMIN (each node had its own crt). This resulted in HSTS issues with Sponsors portal, where initially node presents its OWN ADMIN certificate, followed by WILDCARD PORTAL after redirection. Portal access was broken. We had to promote Wildcard crt to ADMIN function. It was successful, with minum impact and resolved HSTS problem we faced.

Thanks all

[Abid Abdul Latif]

I second what Tymofii Dmytrenko posted there was no impact on Radius / TACACS requests during admin cert renewal process. 

[dperera]

Thank you Tymofii Dmytrenko. I have done the same recently. This thread was useful as it is not clear in the cisco documentation.  Sharing my experience in brief, I renewed admin certificate on 1+1 HA deployment running ISE 3.1 Patch 04. There is no service impact during the application restart because secondary server was working. Once the primary is up, secondary server restarted the application service. In short, certificate renewal process is smooth. The only problem I faced is ”admin certificate” on secondary node is not getting replicated. Below is the error message. Certificate Replication Failed: Admin=****; Server=ISE-01; Message=Failed to replicate certificate **** to node ISE-02 because Cannot delete Admin certificate.

So I had to deregister secondary node from the primary. then it becomes standalone. Import admin certificate to the secondary server and register it again to the primary server. overall  process is smooth without any impact on radius and tacacs+ authentication/authorization services. Please note while deregister/reregister process, services of secondary server is getting restarted. 

Thanks all