The problem can be that the xauth times out. Increase the timeout value for AAA server in order to resolve this issue .
For example:
Hostname(config)#aaa-server test protocol radius
hostname(config-aaa-server-group)#aaa-server test host 10.2.3.4
hostname(config-aaa-server-host)#timeout 10