topic Re: ISE 2.4 Base License inflation overrun with continous increase in Network Access Control

ISE 2.4 Base License inflation overrun with continous increase

derobbacher — Mon, 02 Dec 2019 10:30:56 GMT

Hello ISE-Specialists,

we are running 2 ISEs on 2.4.0.357 SW-Level with Patch 5,6 on Virtual Machines to authenticate WLAN users via Radius for several months.

There are around 3000 users authenticating on daily Basis.

Because we run into performance issues with Radius Accounting on the old Cisco ACS System, that were

replaced by ISEs in the past, we did not activate Radius-Accounting on our Cisco WiSM2-WLCs running Code 8.3.141.

Since several weeks, we face a License warning stating that our 4000 Base licences are exhausted.

(We have only Base licenses installed )

It is interesting to see, that the consumed license value is increasing with every week.

Today we have reached already a value of 15.000 consumed licenses !!!

To me it looks like ISE is no longer able to phase out old sessions.

What can be done to cool down the ISE platform to normal values ?

Is the activation of Radius Accounting a solution ?

That You for every hint.

Kind regards from Frankonia

Wini

Re: ISE 2.4 Base License inflation overrun with continous increase

Arne Bier — Mon, 02 Dec 2019 23:49:57 GMT

By design, ISE will consume a base license for a successful auth, and then maintain that license for up to 5 days. If you use RADIUS Accounting, then you can manage the license release dynamically. It was an issue in the past that the WLC would send too many RADIUS Accounting requests, and thus, be a nuisance to RADIUS servers. But 8.3 should be fine. Enable Interim Updates for 0 (zero) seconds. This tells the WLC to only send Interim Updates when the IP address of the client changes (or if there is a DHCP event). Other than that, you should see a RADIUS Accounting Start when the session starts, and a Stop when the session ends (normally, or abnormally). These RADIUS events inform ISE about license usage.

If none of this helps then you may have a TAC case on your hands.

Re: ISE 2.4 Base License inflation overrun with continous increase

derobbacher — Tue, 03 Dec 2019 13:02:12 GMT

Thank You Arne for Your help.

For testing, I have activated RADIUS Accounting on one of our WLCs to see if it improves the Situation.

Is is also possible to free the eaten up Base-licenses by simply deleting the WLAN-Clients on the ISE ?

When i look into Work Centers->Network Access ->Identities -> Endpoints I have a list of more than

60000 entries of MAC addresses.

What happens if I would delete everthing or big parts of these entries ?

Would this reduce also the wrong number of Base licences used by the clients ?

Please check and advise

Kind regards

Wini

Re: ISE 2.4 Base License inflation overrun with continous increase

Damien Miller — Tue, 03 Dec 2019 13:31:00 GMT

Just to be clear, total known endpoints does not equal license usage. The context visibility endpoint database you referred to just keeps track of endpoints and their attributes.

Base licenses are consumed by active endpoint sessions. If you don't receive an accounting stop or an interim accounting update, then a session will remain active for 5 days. In an environment with a lot of guest users, this could cause high license usage. In an environment where the same endpoints are on the network daily, it wouldn't have a huge impact.

So brass tacks here, how many base licenses are you actually using right now, what are you total active sessions, and what is your expected active sessions?

Re: ISE 2.4 Base License inflation overrun with continous increase

derobbacher — Thu, 05 Dec 2019 12:06:50 GMT

Hello Damien,

thank You for Your advice.

Right now, ISE ist showing that we are consuming 15500 Base Licenses.

But that is nonsense.

The number of active sessions right now are around 3500.

As already said, we have installed 4000 Base licenses.

The primary ISE knows around 68000 Endpoints today.

Comparing with our two central WiSM-2-WLCs:

Actually around 2500 users in two SSIDs where i have activated Radius Accounting.

You can see a sudden increase in licens usage within only 2 days.

This is impossible in our hospital.

Our consultant recommended to reboot the machines, but it did not cure the problem.

Is is possible to reinitialize the license usage database ?

Or can I immediatelly manually start the cleanup process which runs every 5 days in background ?

What consequences do we face here in our hospital, if the problem is not solved until the end

of the warning period ? Will ISE stop working ?

By the way, we also send out an SSID for Guests. But here we do not want to make use of the ISE.

It is a open and free WLAN-access used within Bavaria with open security.

Radius Accounting is marked, but no Radius-Server-entry is configured.

Or does this mean, that the globally defined Radius-Servers are used instead ?

Can You advice what to do please ?

Kind reagards

Wini

Re: ISE 2.4 Base License inflation overrun with continous increase

Arne Bier — Thu, 05 Dec 2019 22:24:11 GMT

Hi @derobbacher

I have not seen any mechanisms that allow us to reset/control the licensing usage processing in ISE. You might have some luck if you toggled Smart Licensing on/off to see if reverting from Smart to Traditional Licensing does something to help you.

ISE won't fall over if you violate the license. But. If you violate it more than 45 days in a 60 days time frame, then ISE will constrain your Web Admin access to the Licensing page (to force you to upgrade the license).

Regarding the Guest SSID, if you have ticked those RADIUS Server boxes (but there are no servers listed) then no RADIUS will be used for that SSID. You might as well uncheck those boxes to avoid confusion.

Have you checked whether your ISE deployment is receiving RADIUS Accounting? e.g. you could run an Operational Report and see what devices are sending you RADIUS Accounting. You should see a bunch of Start and Stop requests at least. And Interim Updates are a bonus too.

Re: ISE 2.4 Base License inflation overrun with continous increase

hslai — Sun, 08 Dec 2019 04:36:30 GMT

Removing Stale Sessions should help cleaning up stale sessions.

Also, please check whether ISE receiving the accounting STOP requests and whether it cleans the sessions up as a result.

Re: ISE 2.4 Base License inflation overrun with continous increase

derobbacher — Mon, 09 Dec 2019 12:22:24 GMT

Hello hslai,

thank You for Your hint.

I habe downloaded curl and used it to remove stale sessions:

C:\>curl -k insecure -u admin:password https://ise1/admin/API/mnt/Session/ActiveCount

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><sessionCount><count>2079</count></sessionCount>

C:\>

C:\>curl -X DELETE -k insecure -u admin:password https://ise1/admin/API/mnt/Session/Delete/All

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><mnt-rest-result><http-code>200</http-code><cpm-code>52092</cpm-code><descripti

n>The software update was completed successfully</description><module-name>MnT</module-name><internal-error-info></internal-error-info

<requested-operation>Not Available</requested-operation><resource-id>2356</resource-id><resource-name>MNT_RAD_SESS</resource-name><sta

us>SUCCESSFUL</status></mnt-rest-result>

C:\>curl -k insecure -u admin:password https://ise1/admin/API/mnt/Session/ActiveCount

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><sessionCount><count>12</count></sessionCount>

C:\>

It looks like the number of sessions was reduced sucessfully.

But the ISE is still telling me that the number of licenses in use is 14648(>1255%) of our entitlement of 4000 licenses.

How can i get rid of this wrong Information in the ISE database ?

I have only some days left. until ISE will stop working obviously.

Please check and come back with Information.

Kind regards

Wini

Re: ISE 2.4 Base License inflation overrun with continous increase

derobbacher — Tue, 10 Dec 2019 08:01:58 GMT

Hello Cisco ISE Experts,

I have investigated a little and it looks to me that due to a power outage testing, apparently a

Health-problem between the two ISEs has caused an error in the licences usage database some weeks ago.

Eventhough, we only have around 2000-3000 users per day, the number of consumed licenses is not reducing.

To be honest, I'm not happy with this situation caused by a bug in the Software obviously.

Is it possible to start over the license usage from the beginning to clean out this mistake ?

Who can help in this tricky situation ?

Kind regards

Wini

Re: ISE 2.4 Base License inflation overrun with continous increase

hslai — Sun, 15 Dec 2019 01:08:40 GMT

No, a power-outage testing should not have caused it. Please engage TAC to investigate.