ISE endpoint purge question

[eglinsky2012]

We use MAC address bypass on our guest network as a solution for non-802.1x devices to get on WiFi and bypass the captive portal. It is currently performed with IP address reservation, but we will be switching to MAB via ISE. Registration is performed on a house-developed Web portal that adds the endpoint to a group via API. We would like to migrate all existing registrations to ISE, including some that may currently be inactive and never come online again. Then have ISE purge a device after it has been inactive for x amount of time.

We questioned whether an endpoint that has never been online would get purged. I tested this by setting a purge rule to purge devices out of the group that haven't been active in 1 day. I then registered a dummy MAC address Wednesday afternoon to see if it got purged. It's now Thursday late morning and it has not been purged. Purge is set to run daily at 3am, so I'd have thought that it should be purged by now.

Is there something I'm missing, or is this expected behavior to not purge devices that have never been online that cannot be changed?

ISE version is 3.2 P4.

The purge rule I created is the RegisteredEndPointsPurgeRule.

[Arne Bier]

Hello @eglinsky2012 

The granularity is not very good - Elapsed Days > 1 means that, at least 48 hours must pass, from the time the endpoint was first seen,  before the flag is set to TRUE. And only then, when the 3AM purge job runs, will this endpoint be deleted.

So here's the worst case scenario.

Monday endpoint is first seen at 03:01 AM 

Tuesday purge job runs at 03:00 (Endpoint Elapsed Days = 0 - no purge ) - one minute later Elapsed Days = 1

Wednesday purge job runs at 03:00 (Endpoint Elapsed Days = 1 - no purge ) - one minute later Elapsed Days = 2

Thursday purge job runs at 03:00 - Endpoint Elapsed Days = 2 - Endpoint purged. 

The best you can do is to make the Elapsed Days GREATER THAN 0

[eglinsky2012]

Thanks for the info, @Arne Bier , that makes sense. Indeed, the endpoint did get purged at some point after I last checked Friday afternoon. That makes sense; endpoint created Wednesday afternoon, >1 (i.e. 2 days) must pass (Friday afternoon), and the next purge cycle after that was Saturday morning. The concern we had was that the endpoint was never seen; it was a dummy MAC address that never came online. But it did get purged, which is good.

I have a couple followup questions. Is there a limit to how many endpoints can be in a group, or in ISE as a whole? We might end up with several thousand endpoints in this group before they get purged due to inactivity. Plus, we have around 500,000 endpoints in the "Unknown" group. However, the dashboard indicates there are only 252,027 endpoints authenticated (27% connected, 72% disconnected) and 303,600 total endpoints. I'm having trouble wrapping my brain around what's what, so if there's any documentation you could link to that explains this, I'd appreciate it.

Also, should we have a purging rule to purge inactive endpoints from all (or most of) our identity groups? (GuestEndpoints that are not registered to bypass the portal, Unknown, Profiled, etc.) If so, what's the best practice for a timeframe, maybe inactive after 0 days?

[Arne Bier]

This topic of ISE Endpoint housekeeping has also consumed my attention for some time. I don't recall any documentation on it, other than the purge tools that ISE has available - what we do with them is just a product of our imagination.

As for the maximum/limit of endpoints supported, I have not seen a document that tells me whether the Oracle Database has any specific limits, or the Context Visibility (Elastic Search database that gets its data from Oracle DB) - the max number quoted in Cisco Performance and Scale document is 2 million concurrent endpoints when using the largest PAN node. That's been a limit for quite some time now.  I don't know if having more than 2 million endpoints in a single ISE deployment is sensible.

As for the question of what to purge and when to purge it, I'd say that endpoints that have been inactive for more than 180 deserve to be deleted. Unless, any of those endpoints have been assigned to a specific Endpoint Group (static assignment).  Unless you are 100% sure that such an endpoint is no longer required, then it's better to not delete it. Of course, if there are endpoints in ISE that have no Policy Set logic relating to them (e.g. very old endpoints from many years ago that someone forgot to delete) then clean those up. You must have a clear understanding of your Policy Set before you delete those endpoints.

If you want to analyse the endpoints, from various perspectives (lifetime, vendor MAC OUI, etc.) you can export all endpoints from the PAN CLI (application configure ise) and then import that into a spreadsheet for analysis. However, if you find that there are many endpoints you want to delete as a result of that analysis, then the only sensible way I know of, is to use REST API to delete those endpoints - a small python script that reads the endpoint_id from a file, and then makes an API call to ISE to delete it.  I don't think there is a way to delete them from Context Visibility via some .csv import - but that would be a little more user friendly.

[eglinsky2012]

Thanks, Arne. I did see that 2 million limit as concurrent active endpoints. What I'm wondering is if there's a limit to the total endpoints that are in ISE (whether it be in a custom group, Unknown, Profiled, etc.). Our virtual nodes I believe are the SNS 3655, with 96 GB RAM and 24 CPUs. It's a "large" deployment with 4 PSNs.

At this time, we strictly use ISE for 802.1x authentication for wireless clients on our vanity SSID and eduroam, and shortly, we will move our guest network captive portal to ISE as well (using MAB for IoT devices that cannot do 802.1x authentication and are in the "RegisteredDevices" endpoint group for captive portal bypass). We don't use our policies to determine who gets access to what. It's more of an accountability thing, just to authenticate, and everyone gets the same level of access from there, and access is restricted upstream by other means.

In other words, all the endpoints currently in ISE are there because they authenticated to WiFi with 802.1x. There's no need to keep them in ISE for any length of time after they go offline that I can think of. If they get purged, they'll just get added back to the Unknown or Profiled group next time they connect. Prime (and later DNA) will keep track of who was logged in to what device and when, plus ISE's authentication logs go to Splunk for longer-term archiving. The only endpoints to keep long term will be in the RegisteredDevices, which are the ones that will bypass the guest network captive portal. I'll have them purge after say 9 months of inactivity.

Does that help narrow down what we should be doing for purging?

[Marcelo Morais]

Hi @eglinsky2012 ,

 beyond what @Arne Bier already said ...

1st a Large Deployment consist of 2x PAN, 2 MnT and PSNs on Dedicated Nodes and more than 150,000 concurrent Active Sessions.

2nd each SNS 3655 Dedicated PSN Node has a maximum of 50,000 Concurrent Active Session (each Dedicated PSN Node)

Note: remember that " ... The maximum Concurrent Active Session values given above for each Deployment are applicable for Connected Devices that are generating Dot1x Authentications up to 4 times a day ... "

3rd " ... It is highly recommended to purge Guest and Inactive Endpoints at regular intervals to avoid latency in ISE operations ... "

4th at ISE > Context Visibility > Endpoints, check the Inactive Endpoints dashboard for a "macro view" of "Last Activity Date"

5th use the CLI bellow for a "detail view" of the InactiveDays attribute of the Endpoints:

ise/admin# application configure ise
...
[16] Get All Endpoints

6th since the Inactive Endpoints dashboard shows up to the last 30 days, then a ENDPOINTPURGE for InactiveDays GREATHERTHAN 30 is a good first step (at ISE > Administration > Identity Management > Settings > Endpoint Purge)

7th at ISE > Administration > System > Maintenance > Operational Data Purge check your Data Retention Period, in your case (maybe) you would like to match the Operation Data Retention Period with the Inactive Endpoints Period.

Hope this helps !!!

[Amro]

For Guest endpoints, what would be the appropriate elapsed number of days that is tried and tested, because our organization thinks that 1 day is appropriate, but i ve seen guests come back to get another login since they dont keep the user/pass and because of the purging every 2 days it keeps asking them for login.

Thanks

[Arne Bier]

I would reckon that a weekly pass makes sense for the self-registered Guest Type - 5 days (to cover Mon-Fri).  it's not like you're giving them access to an organisation's intranet - it's just internet access. I think internet access should be given out more freely now that bandwidth is not as expensive as it used to be. Mark all the traffic with the lowest DSCP and let them at it. 
The Sponsor Portal can contain a longer list of Guest Types - e.g. if you have a specialist contractor who needs 3 weeks onsite Free Wifi to connect to the VPN or whatever, then the Sponsor can select the "monthly" guest type. So that is on a need to have basis.

Sometime, organisations make strange rules like "you only get access for 1 day and then you must reapply again" because perhaps they had a bad experience in the past, and they need tighter control to weed out those users who might potentially abuse the system. Who knows why. I'd rather give someone 1 week, and if they only needed 1 day, then I'd go into the Sponsor Portal and disable their account.  To be fair, this should all be automated with some kind of front desk ipad where you register at reception, get your sticker and wifi token. When you leave the site and logout on the iPad, then it should deactivate your guest account too. I have seen this integration with Aruba Clearpass (it was very slick).