ISE Alarms: COA failed due to "No valid Session"

[PSM]

We are seeing alarms for COA failure on Cisco ISE dashboard. When open the details of the log on ISE in the Result section Reply-Message is "No valid Session" but interestingly when we see some of the sessions using the session id mentioned in the log there are valid sessions on wireless controllers. This is happening intermittently and not always. We are sure COA in general is working. 

 Here is the snapshot from one of the log message.

Wireless controller is a Cisco 9800-40 running on 17.12.4 and we see on many WLCs.

[Arne Bier]

I see this on various types of Catalyst devices too (switches and WLCs) - in theory this should not happen, because if a session terminates, then ISE should get the Accounting Stop, and that means ISE has no reason to send any subsequent CoA Requests to that device for that Session. Either the Accounting Stop messages are getting lost (or not processed correctly) or there is some other issue.  I have a TAC case open at the moment to investigate something similar where ISE doesn't seem to process the Accounting requests properly, and indicates in Context Visibility that the session id 'Disconnected', when it's alive and well. Only when I delete the Endpoint in ISE, and repeat the test, then all is well.

[PSM]

 @Arne Bier thanks for sharing the information. We have verified multiple times that session still exist on WLC even after 2 hours of COA failed message. Had opened a TAC case as well but TAC asked to run radioactive trace. But they don't understand that issue is so random that you don't know on which endpoint it will happen and when it is going to happen. Also can't run radio active trace for thousands of endpoints. To save our time we asked to close the case and didn't proceed further.

[MHM Cisco World]

Do you add key for CoA server in wlc 9800

MHM

[PSM]

@MHM Cisco World yes, key is added and in general COA is working as expected. This issue is happening intermittently and not every time. 

[Arne Bier]

I have one theory. In Meraki for sure, CoA combined with Fast Transition is problematic because when a client is first authenticated by ISE by WAP#1, but then the client roams to WAP#2, ISE will not know that this has happened, because Fast Transition spares the client from having to reauth again. Good for the client , but a disaster for CoA. ISE will send the CoA to a WAP that is no longer responsible for that session. This is why you will see COA failures in such a constellation.

Maybe your situation is similar? Is this a Cisco FlexConnect scenario?

ISE will only send a CoA to a NAD if ISE believes the session is still active. So either the Accounting Stop was not sent or processed, or more likely is the theory I just explained

[PSM]

Interesting. But in our case it is Cisco 9800 and all APs are running in local mode. 

[PSM]

@Arne Bier  and all do we need NAC state to be enabled in Policy Profile even though we are using Dot1x as authentication method and dynamic-author/COA is already configured for PSNs ? In my understanding NAC state is needed in case of CWA. Couldn't found any good Cisco documentation which explain 'NAC state' and in which scenarios except CWA it is needed.   

[Arne Bier]

Maybe this link helps - regarding NAC State (relating to a feature in the C9800) - I can tell you that in my networks, NAC State is not enabled in the C9800 Policy Profile for 802.1X WLANs.  Probably because in cases where there is no AAA override, the RADIUS server is just sending Accept or Reject, and that's sufficient for the C9800.  I don't know if it harms enabling NAC State, even though the RADIUS server doesn't return any attributes to be overwritten - probably not. 

In the case of Guest SSID, NAC State and AAA override are used (in my networks) because we want to overwrite the Session-Timeout value.