Radius Server DEAD and ALIVE Problem

[Mehmet Can]

Hello All,
I want to ask about after ISE integration. I provisioned our 2 Border Switch but i'm getting a lot of error messages about ISE. What's the problem? In version 16.9.3 I don't get this error. How can I solve this problem?
I don't configure aaa and radius configuration manualy. All configuration created by DNA Center.

Versions:

SW: 9500-16X - 16.11.1c

DNA: 1.3.1.4

Jan 16 2020 04:37:05.642 UTC: %RADIUS-6-SERVERALIVE: Group dnac-client-radius-group: Radius server X.X.X.X:1812,1813 is responding again (previously dead).
Jan 16 2020 04:37:05.642 UTC: %RADIUS-4-RADIUS_ALIVE: RADIUS server X.X.X.X:1812,1813 is being marked alive.
Jan 16 2020 04:37:06.510 UTC: %RADIUS-4-RADIUS_DEAD: RADIUS server X.X.X.X:1812,1813 is not responding.
Jan 16 2020 04:37:22.250 UTC: %CTS-3-AAA_NO_RADIUS_SERVER: No RADIUS servers available for CTS AAA request for CTS env-data SM
Jan 16 2020 04:37:26.654 UTC: %RADIUS-3-ALLDEADSERVER: Group dnac-network-radius-group: No active radius servers found. Id 177.
Jan 16 2020 04:38:49.996 UTC: %RADIUS-6-SERVERALIVE: Group dnac-network-radius-group: Radius server Y.Y.Y.Y:1812,1813 is responding again (previously dead).
Jan 16 2020 04:38:49.996 UTC: %RADIUS-4-RADIUS_ALIVE: RADIUS server Y.Y.Y.Y:1812,1813 is being marked alive.
Jan 16 2020 04:39:34.352 UTC: %RADIUS-4-RADIUS_DEAD: RADIUS server Y.Y.Y.Y:1812,1813 is not responding.
Jan 16 2020 04:40:06.509 UTC: %RADIUS-4-RADIUS_ALIVE: RADIUS server X.X.X.X:1812,1813 is being marked alive.
Jan 16 2020 04:41:18.708 UTC: %RADIUS-4-RADIUS_DEAD: RADIUS server X.X.X.X:1812,1813 is not responding.
Jan 16 2020 04:41:50.966 UTC: %RADIUS-3-ALLDEADSERVER: Group dnac-client-radius-group: No active radius servers found. Id 146.
Jan 16 2020 04:42:34.352 UTC: %RADIUS-6-SERVERALIVE: Group dnac-client-radius-group: Radius server Y.Y.Y.Y:1812,1813 is responding again (previously dead).
Jan 16 2020 04:42:34.352 UTC: %RADIUS-4-RADIUS_ALIVE: RADIUS server Y.Y.Y.Y:1812,1813 is being marked alive.
Jan 16 2020 04:42:45.520 UTC: %RADIUS-4-RADIUS_DEAD: RADIUS server Y.Y.Y.Y:1812,1813 is not responding.
Jan 16 2020 04:42:45.520 UTC: %RADIUS-3-ALLDEADSERVER: Group dnac-network-radius-group: No active radius servers found. Id 188.
Jan 16 2020 04:42:50.967 UTC: %CTS-3-AAA_NO_RADIUS_SERVER: No RADIUS servers available for CTS AAA request for CTS env-data SM
Jan 16 2020 04:44:18.708 UTC: %RADIUS-6-SERVERALIVE: Group dnac-network-radius-group: Radius server X.X.X.X:1812,1813 is responding again (previously dead).
Jan 16 2020 04:44:18.708 UTC: %RADIUS-4-RADIUS_ALIVE: RADIUS server X.X.X.X:1812,1813 is being marked alive.

Radius Conf.

aaa group server radius dnac-client-radius-group
 server name dnac-radius_X.X.X.X
 server name dnac-radius_Y.Y.Y.Y
 ip radius source-interface Loopback0
aaa group server radius dnac-network-radius-group
 server name dnac-radius_X.X.X.X
 server name dnac-radius_Y.Y.Y.Y
 ip radius source-interface Loopback0
aaa authentication login dnac-cts-list group dnac-client-radius-group local
aaa authentication login VTY_authen group dnac-network-radius-group local
aaa authentication dot1x default group dnac-client-radius-group
aaa authorization exec VTY_author group dnac-network-radius-group local if-authenticated 
aaa authorization network default group dnac-client-radius-group 
aaa authorization network dnac-cts-list group dnac-client-radius-group 
aaa accounting identity default start-stop group dnac-client-radius-group
aaa accounting exec default start-stop group dnac-network-radius-group
aaa server radius dynamic-author
 client X.X.X.X server-key 7 PRE-SHARED-KEY
 client Y.Y.Y.Y server-key 7 PRE-SHARED-KEY
ip radius source-interface Loopback0 
radius-server attribute 6 on-for-login-auth
radius-server attribute 6 support-multiple
radius-server attribute 8 include-in-access-req
radius-server attribute 25 access-request include
radius-server attribute 31 mac format ietf upper-case
radius-server attribute 31 send nas-port-detail mac-only
radius-server dead-criteria time 5 tries 3
radius-server deadtime 3
radius server dnac-radius_X.X.X.X
 address ipv4 X.X.X.X auth-port 1812 acct-port 1813
 timeout 4
 retransmit 3
 pac key 7 PRE-SHARED-KEY
radius server dnac-radius_Y.Y.Y.Y
 address ipv4 Y.Y.Y.Y auth-port 1812 acct-port 1813
 timeout 4
 retransmit 3
 pac key 7 PRE-SHARED-KE

[soporte.sona]

Hello

I have the same issue, i have 3 edge devices with the same problem. did you find a solution?

regards,

[Cristian Matei]

Hi,

   Is there UDP 1812/1813, 1645/1646 between the NAD and ISE, and is also ISE configured? This seems to be the problem from those logs. What exact problem re you experiencing? 

Regards,

Cristian Matei.

[Mehmet Can]

Hi,

I have an open Case on this. The ISE team is continuing to work on the issue. I will share information when I get an answer.

[hugo.girard]

hello,
I have the same behaviour with Catalyst 9300.
OK with version 16.9.3
Pb with version 16.9.4

I don't have the solution but I have isolated the command which involves this issue :
cts authorization list <cts-list>
Because I don't use trustsec now, I have removed this command in order to avoid polluting logs.

[MALi-786]

Hi, you got any update from Cisco on this case?

[Mehmet Can]

you will not believe it, but the case has been open for 3 months. There is still no further improvement.

[Emre Ozel]

hi,
I shared a few things about the issue solution. I recommend you to look.

https://community.cisco.com/t5/switching/detect-up-down-radius-server/m-p/4145622/highlight/true#M492353

[alvarteearu]

Had a same issue, even when I completely blocked RADIUS access with Firewall, it kept popping up as alive.

The solution or maybe we can also call it a workaround, was “automate-tester” and “probe-on” function that is available from IOS 15.2(2)E / XE 03.04.00E. With this addition “dead” server will be marked “up” only when a response is received from the RADIUS server, hence as I actually did not get responses back from the server, it was kept “dead”.

[Mehmet Can · Accepted solution]

I resolved this issue. My problem is our internal firewall doing NAT our RLOC IP from in to out way. When i delete our RLOC IP Block from NAT Pool everything was fine How could i see? I got packet captures from edge switch uplink ports.

[Hamada Ahmed]

@Mehmet Can  How did discover the RLOC IP in NAT ?