topic Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices in Network Access Control

Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Sun, 31 May 2020 14:55:33 GMT

We have an automation (using python) project where we have to update shared secrets on network devices (Cisco IOS/IOS XR and also other non-Cisco platforms, via netmiko). Of course, we also have to update (via ERS REST API) the Cisco ISE server with the same shared secrets.

What is the best approach to updating, with minimum connectivity outage? One device at a time, ie, updating the ISE, then updating the device, check for connectivity, then move on to another device? Or bulk update, ie, updating the shared secrets on the ISE for a small group of devices, then updating the secrets for the same group of devices?

I assume it would be one at a time, but like hear additional feedbacks.

Also, is there a solution whereby we can dictate/direct the Cisco devices (or any network device platform) and Cisco ISE to try to check the authenticate using the new shared secret. If check is good, then flip over to the new shared secret. That way, we can get a minimum connectivity disruption. Is that possible?

Thanks,

Peter

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

balaji.bandi — Sun, 31 May 2020 17:21:14 GMT

Do you have a fall back Local Account? if yes

I will change all the Secret at end Device First and later on ISE Side. and test

best practice, test 1 or 2 devices all working as expected, then deploy mass device config change. (even it fails you have fallen back to Local Account to change as required)

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Sun, 31 May 2020 22:07:24 GMT

Thanks BB.

We have quite a few devices, so likely that some won't have a local account to fall back. Especially, a number of them aren't Cisco platforms.

So, I assume the safest plan is to do one by one?

BTW, is this how to check if Cisco IOS routers/switches are configured with local tacacs/radius authentication fallback when communication ISE/ACS server is down?

aaa authentication login default group tacacs+ line

aaa authentication login console group radius local

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

balaji.bandi — Mon, 01 Jun 2020 05:27:47 GMT

The device does not have control like Cisco devices if not many suggest changing manually and test it.

as per the cisco devices - you should ( as per my interested) have local account some point you may need if any disaster of ISE or network connection loss.

below command fall back to Local - only if you have a local username and enable password enabled.

aaa authentication login console group radius local

Note: if not your device is locked and you need to go with password recovery.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

Mike.Cifelli — Mon, 01 Jun 2020 14:26:20 GMT

Agree with @balaji.bandi . Another thought to ensure clients connected to certain NADs that onboard via 8021x or mab is to change the reauth timer to a greater time setting than you have configured now. Essentially what you could do is bump the reauth timers to 8-12 hours in your authz profiles to buy you time to conduct your changes, and avoid having client onboarding issues. Always test on one or two devices first before mass rollout. Good luck & HTH!

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

howon — Mon, 01 Jun 2020 17:16:45 GMT

Aside from the tips provided, you can also leverage second shared secret feature on ISE. This allows two shared secret to be active at the same time for migration (Only available for RADIUS):

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Mon, 01 Jun 2020 20:27:30 GMT

Thanks everyone for your feedbacks!

I will look more into this.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

balaji.bandi — Mon, 01 Jun 2020 22:00:56 GMT

@Mike.Cifelli yes that would be a nice idea, original post does not mention any BYOD feature using with ISE, But your point needs to consider one another aspect of dependency, good point.

let us know how it goes.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Mon, 01 Jun 2020 22:20:29 GMT

Hi Mike,

Your idea sounds good, but I am not well tracking. If you can further expand to help me understand that would be great.

Specifically, on these:

- NADs: what do you mean by NADs?

- that onboard via 8021x or mab

- reauth timer: where do I change that? On the devices, ISE server, or both?

Thanks.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

balaji.bandi — Mon, 01 Jun 2020 22:34:26 GMT

Do you use ISE for dot1.X authentication? or BYOD in your environment?

check other screenshot posted on this post - you can have seconds shared key, (not tried myself)

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

Mike.Cifelli — Tue, 02 Jun 2020 17:29:19 GMT

Specifically, on these:
- NADs: what do you mean by NADs?
Network Access Devices. Edge/Access switches that clients connect to.
- that onboard via 8021x or mab
How you authenticate and authorize hosts onto their respective network.
- reauth timer: where do I change that? On the devices, ISE server, or both?
If you are utilizing ISE already to push authz policy I would recommend configuring it in the authz profiles. This can be done under 'Common Tasks' section under 'Reauthentication'.
HTH!

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Tue, 02 Jun 2020 22:11:58 GMT

We are not using ISE for dot1.X authentication, no BYOD.

The TACACS doesn't have the 2nd shared secret, but it has the retired secret feature. I like to explore that feature more. If anyone has good experiences with the retired secret feature in production or in lab, please share. Details on how that feature works would be great.

Thanks in advance!

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Tue, 02 Jun 2020 22:10:32 GMT

Thanks Mike. I will explore the auth timer more

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

hslai — Thu, 04 Jun 2020 02:02:14 GMT

...

The TACACS doesn't have the 2nd shared secret, but it has the retired secret feature. I like to explore that feature more. If anyone has good experiences with the retired secret feature in production or in lab, please share. Details on how that feature works would be great.

...

See Change your shared secret without network disruption

This can only be configured in ISE admin web UI.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Thu, 04 Jun 2020 02:05:46 GMT

Thanks hslai. So the retire secret feature can't be configured via ERS REST API?

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

hslai — Sat, 06 Jun 2020 17:54:26 GMT

@pn2020 wrote:

... So the retire secret feature can't be configured via ERS REST API?

That is correct. Please voice your feedback through New Features and Feedback

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and network devices

pn2020 — Sat, 06 Jun 2020 18:03:57 GMT

That's definitely disappointing and a major deficiency, especially RADIUS supports the secondary shared secret.

I did leave a request on that forum channel.

Thanks.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and n

Arne Bier — Fri, 14 Mar 2025 00:41:43 GMT

In case anyone stumbles on this discussion in future, I made some further revelations about the usage of RADIUS second shared secret. Here are some limitations/caveats regarding RADIUS:

Even when second RADIUS shared secret is configured, ISE will still use the primary shared secret when it sends CoA. If the network device has been updated to the new (second) shared secret, then the CoA will not be acknowledged by the device
The second RADIUS shared secret will only be honoured by ISE if the Access-request contains the attribute 80 (Message-Authenticator) - the absence of this attribute will cause a reject. Most PAP requests and things like IOS "test aaa" commands do not include Message-Authenticator. EAP requests always include the Message-Authenticator.

Therefore, the message is, your mileage may vary. The safest option is probably to update one device at a time, taking care of the CoA shared secret on the device as well. That is the cleanest and safest approach - and also the hardest.

As for TACACS, it's 2025 and ISE 3.4 still doesn't support rotation in bulk. It's a case of doing those one-by-one too, Not fun.

Re: Best approach to updating TACACS/RADIUS share secrets on ISE and n

Arne Bier — Fri, 16 May 2025 03:20:26 GMT

An update on the TACACS+ key rotation using REST API topic - great news! The Cisco Developer ISE SDK is listing this feature as being available in ISE 3.5 if I read that correctly. ISE 3.5 is not yet publicly available.