Every 6 hours RADIUS_DEAD - not responding

[Barry Landon]

Every 6 hours on some devices we see in logs RADIUS_DEAD not responding with all three radius servers.  Other parts of the network it happenes even more frequently.  From debug it an accouting session initiates but soon after a tcp reset occurs due to NAS error (auth failed).  Between these six hours the radius servers comms are ok. No issues. We can access the device no problem but during RADIUS_DEAD were not able to access the host.

Im trying to determine if the issue is with our local host that it has a parameters that the radius sever rejects or the issue is with the a misconfiguration at the radius server with local config is ok.

This particular device is C9300 but im seeing the same problem with 3650 and Nexus 9k all running their Cisco suggested IOS.

Below is the full configuration and attached debug.  Appreciate people assistance in this. 

aaa group server radius X-RADIUS
server name X-RADIUS-1
server name X-RADIUS-2
server name X-RADIUS-3
ip radius source-interface VLANX
exit

aaa authentication login default group X-RADIUS local
aaa authentication login vty group X-RADIUS local
aaa authentication login con group X-RADIUS local
aaa authentication enable default line group X-RADIUS enable
aaa accounting send stop-record authentication failure
aaa accounting exec net-supp start-stop group X-RADIUS
aaa accounting connection net-supp start-stop group X-RADIUS
aaa authorization exec net-supp group X-RADIUS local
aaa authorization commands 1 net-supp group X-RADIUS local
aaa authorization commands 15 net-supp group X-RADIUS local

radius server X-RADIUS-1
address ipv4 10.X.X.X auth-port 1812 acct-port 1813
key 1234abcd
radius server X-RADIUS-2
address ipv4 10.X.X.X auth-port 1812 acct-port 1813
key 1234abcd
radius server X-RADIUS-3
address ipv4 10.X.X.X auth-port 1812 acct-port 1813
key 1234abcd
!
exit

line vty 0 15
password JGHGGS1
login authentication vty
accounting connection net-supp
accounting exec net-supp
transport input ssh
transport output none
exec-timeout 9 0

[marce1000]

  >...

  >This particular device is C9300 but I m seeing the same problem with 3650 and Nexus 9k all running their Cisco suggested IOS.

    - Since different device types are involved I also suggest in analyzing network behavior and or traffic. Look for bursts, networking spikes , or  just traffic data from/to those devices at that time but also during normal operation. Also use                             syslogging to a central syslog server, and the same for snmp-traps to a trap receiver.

 M.

[MHM Cisco World]

automate-tester username dummy

Try this which make NAS test aaa reach ability 

[Barry Landon]

Does this test the connection to the radius server only at set intervals or sends probe once the radius is marked as dead?

[MHM Cisco World]

[MHM Cisco World]

at interval the client send test to RADIUS and check if the RADIUS response or not.

[hslai]

If such events always associated with the accounting stop requests following auth failures, then you might consider to remove this line

aaa accounting send stop-record authentication failure

Regarding the configuration command automate-tester, see 

• Demystifying RADIUS Server Configurations > RADIUS Server Failure Handling
• ISE Secure Wired Access Prescriptive Deployment Guide > Preparing a Switch for Identity-Based Network Access > Best Practice Global Settings for Switch > RADIUS Server Failure Detection