Cisco ISE - MAB authorization behaviour

[Jagermeister]

Hi all,

I'm trying to setup ISE to perform a simple MAB profiling for a printer. For some reason ISE rejects the printer the first time I connect it:

Event	5400 Authentication failed
Failure Reason	15039 Rejected per authorization profile

I am using an authorization profile that contains a Logical Profile for printers. Within this logical profile I've added the a 'Ricoh-Device' Profiling Profile, which has a profiling condition to look if the Mac address OUI contains 'ricoh'. 

When i look up the device in the authentication endpoints I do see the printer with the 'Ricoh-Device' endpoint profile and the authentication failure reason, which is Rejected per authorization profile. Once i cycle the switchport manually, the device is getting accepted and placed into my printer VLAN.  After that the endpoint store is listing that it is using the correct authorization policy. 

I am hoping that someone can point me into the right direction. manually bouncing the port is not feasible, Can anyone help me finding out why this is happening?

Setup: 

- 2x ISE Node with all personas ( 3.4.0.608 ), hosted in Azure. 

- NAD: Various Cisco Meraki switches, my test lab is a MS120-8FP

Access policy:

Authorization policy:

1th attempt log:

Step ID	Description	Latency (ms)
	11001	Received RADIUS Access-Request
	11017	RADIUS created a new session	0
	11027	Detected Host Lookup UseCase (Service-Type = Call Check (10))	1
	15049	Evaluating Policy Group	0
	15008	Evaluating Service Selection Policy	0
	15041	Evaluating Identity Policy	1
	15048	Queried PIP - Normalised Radius.RadiusFlowType	1
	22072	Selected identity source sequence - anonymized_AD_Sequence	0
	15013	Selected Identity Source - anonymized-AD	1
	24432	Looking up user in Active Directory - anonymized-AD	0
	24325	Resolving identity - <mac address anonymized>	20
	24313	Search for matching accounts at join point - anonymized.local	1
	24318	No matching account found in forest - anonymized.local	0
	24367	Skipping unusable domain - anonymized trust is one-way	0
	24367	Skipping unusable domain - anonymized trust is one-way	0
	24367	Skipping unusable domain - anonymized trust is one-way	0
	24322	Identity resolution detected no matching account	0
	24352	Identity resolution failed - ERROR_NO_SUCH_USER	0
	24412	User not found in Active Directory - anonymized-AD	0
	15013	Selected Identity Source - Internal Endpoints	0
	24209	Looking up Endpoint in Internal Endpoints IDStore - <mac address, anonymized>	0
	24217	The host is not found in the internal endpoints identity store	2
	22016	Identity sequence completed iterating the IDStores	0
	22056	Subject not found in the applicable identity store(s)	1
	22058	The advanced option that is configured for an unknown user is used	0
	22060	The 'Continue' advanced option is configured in case of a failed authentication request	0
	15036	Evaluating Authorization Policy	0
	24209	Looking up Endpoint in Internal Endpoints IDStore - <mac address anonymized>	0
	24217	The host is not found in the internal endpoints identity store	2
	15048	Queried PIP - EndPoints.EndPointPolicy	6
	15048	Queried PIP - EndPoints.LogicalProfile	1
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	15048	Queried PIP - EndPoints.LogicalProfile	1
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	24432	Looking up user in Active Directory - anonymized-AD	1
	24325	Resolving identity - <mac address, anonymized>	4
	24313	Search for matching accounts at join point - anonymized.domain	0
	24318	No matching account found in forest - anonymized.domain	0
	24367	Skipping unusable domain - anonymized,Domain trust is one-way	0
	24367	Skipping unusable domain - anonymizedDomain trust is one-way	0
	24367	Skipping unusable domain - anonymized,Domain trust is one-way	0
	24322	Identity resolution detected no matching account	0
	24352	Identity resolution failed - ERROR_NO_SUCH_USER	0
	24412	User not found in Active Directory - anonymized-AD	0
	15048	Queried PIP - anonymized-AD.ExternalGroups	4
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	15048	Queried PIP - EndPoints.LogicalProfile	1
	15048	Queried PIP - EndPoints.LogicalProfile	1
	15048	Queried PIP - EndPoints.LogicalProfile	0
	15048	Queried PIP - EndPoints.LogicalProfile	1
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	15048	Queried PIP - EndPoints.EndPointPolicy	2
	15048	Queried PIP - EndPoints.EndPointPolicy	1
	15048	Queried PIP - EndPoints.LogicalProfile	1
	15016	Selected Authorization Profile - DenyTest	0
	15039	Rejected per authorization profile	0
	11003	Returned RADIUS Access-Reject	1

2th log after port bounce:

Steps

	Step ID	Description	Latency (ms)
	11001	Received RADIUS Access-Request
	11017	RADIUS created a new session	0
	11027	Detected Host Lookup UseCase (Service-Type = Call Check (10))	1
	15049	Evaluating Policy Group	0
	15008	Evaluating Service Selection Policy	0
	15041	Evaluating Identity Policy	1
	15048	Queried PIP	0
	22072	Selected identity source sequence	1
	15013	Selected Identity Source - Internal Endpoints	0
	24432	Looking up user in Active Directory - <mac address, anonymized>	0
	24325	Resolving identity	13
	24313	Search for matching accounts at join point	0
	24318	No matching account found in forest	0
	24367	Skipping unusable domain	0
	24367	Skipping unusable domain	0
	24367	Skipping unusable domain	0
	24322	Identity resolution detected no matching account	0
	24352	Identity resolution failed	0
	24412	User not found in Active Directory	1
	15013	Selected Identity Source - Internal Endpoints	0
	24209	Looking up Endpoint in Internal Endpoints IDStore - <mac address, anonymized>	0
	24211	Found Endpoint in Internal Endpoints IDStore	1
	22037	Authentication Passed	1
	15036	Evaluating Authorization Policy	0
	15048	Queried PIP	9
	15048	Queried PIP	0
	15016	Selected Authorization Profile - anonymized-VLAN45-Printers	1
	24209	Looking up Endpoint in Internal Endpoints IDStore - <mac address>	1
	24211	Found Endpoint in Internal Endpoints IDStore	1
	11002	Returned RADIUS Access-Accept	0
	5238	Endpoint authentication problem was fixed	0

[MHM Cisco World]

In ISE

OUI is take from first three numbers of MAC.

MHM

[Jagermeister]

Thanks,

I don't think that is unique to ISE, its well documented in IEEE 802 related RFC's that the first 24 bits from the 48 bits MAC address is the OUI. 

[MHM Cisco World]

ricoh <<- this make me think that OUI is wrong are you sure mac address start with this ? 

Also MAB for printer dont use CoA' CoA mainly use when you have guest.

MAB is straight process' send call-back ISE use wired-mab policy and then send access-accept with vlan/dacl authz.

Why you have CoA for what ? 

MHM

[Arne Bier]

@MHM Cisco World you're way off target again in your responses. CoA has many use cases, not only "mainly used when you have guest". Profiling relies heavily on CoA and that's what we're discussing here. When ISE performs a MAB auth it might require multiple iterations of the ISE Policy Set to achieve the desired outcome - and CoA is the mechanism to reauthorize the endpoint until it matches the desired Authorization Rule 

[Jagermeister]

I think I understand what you mean with the OUI part. 

ISE has multiple default profiler conditions like this;

So the profiler condition looks at the 1th 24 bits of the MAC, looks it up in some kind of database to resolve the organization of this OUI number, and then compares the return value with this attribute value that is listed in the Profiler Condition.  So no, the MAC itself doesn't start with ricoh but the result of the OUI lookup does contain this.

Why CoA ? 

I can't clearly verify what I am thinking yet...  But i think the 1th attempt my printer tries to authenticate fails because ISE didn't profile it yet.  After being correctly profiled, the authorization changed and a reauthentication CoA request is being used for that (?).  I'm not completely sure about this, so please correct me if I'm wrong :). 

[MHM Cisco World]

Thanks for reply' check my comment below for using CoA with profiling 

MHM

[Arne Bier]

@Jagermeister what you're describing is a text book example of what happens when CoA (Change of Authorization) is not working. CoA allows the RADIUS server to send a message to the NAD to re-authorize the session it can do other things too but for profiling ISE will send a CoA Reauth). Therefore check that your Network Device in ISE has CoA/Dynamic Authorization checkbox ticked - Cisco uses UDP/1700 for this.  On the NAD, check that dynamic authorization is enabled - and also, ensure that the RADIUS shared secret matches that of the device define in ISE. And ... this caught me recently, if your RADIUS traffic run inside a VRF on the switch, then ensure that the dynamic authorization mentions the vrf name. This of course applies to all other RADIUS commands.

Once you have working CoA, you will see this in the ISE Live Logs, and then profiling becomes a plug-and-play experience.

[Jagermeister]

@Arne Bier ,

Hi Arne, thanks for replying. 

I expected issues with CoA indeed so I already made sure that all the settings were correct. I checked again, and my NAD configuration seems OK. 

I currently am reproducing the situation in the following manner: 

1. I delete the endpoint from the endpoint store, this should trigger a CoA 

2. supplicant on switchport is deauthenticated 

3. Supplicant is appearing again in endpoint store now, hitting on my default deny authorization policy

4. After waiting quite some time (~15 minutes), the supplicant is granted access to the network and the Meraki switch event logs show "event type; RADIUS dynamic VLAN assignment"

The take a closer look i've made a pcap and I have let it run until the client got authenticated again (hiding first 2 octets for privacy):

It seems that it does send the CoA that's triggered by the default 'Endpoint Delete' profiler exception action.  As you can see it gets acknowledged as well. After that I see a bunch of new CoA requests, all containing VSA: t=Cisco-AVPair(1) l=35 val=subscriber:command=reauthenticate, but the switch never sends an ACK on those.   Even though it doesn't acknowledge the last, the supplicant still gains access after waiting for ~15 minutes. Manually triggering CoA's from the end point store also seems to work (getting ack's on those) . 

Any idea?

[Arne Bier]

The Wireshark decode is quite revealing. I have had many issues with Meraki and its RADIUS implementation (esp the MS390 - what an awful product) - they often lack features that Cisco switches have had for over 10 years, and then still contain many bugs. I'd open a ticket with Meraki. Not responding to a CoA is a bug. If the NAD believes the CoA was sent in error, it must respond with a CoA NAK. I see those sent by Catalyst switches. I assume you've checked the Meraki release notes or tried to update the switch as well?

Do you know why you have so many duplicate packets shown in your Wireshark? 

[Jagermeister]

Hi Arne, Thanks for replying again. 

I've did some more research today and it seems that Meraki expects the following for a CoA reauth request:

- Calling-Station-ID

- Cisco AV Pair: 

• subscriber:command=reauthenticate
• audit-session-id

After I delete the supplicant from the endpoint store, it is issueing a CoA which contains both the subscriber command and the audit session ID. Shortly after this, ISE is sending CoA reauthentication requests again, but they do not contain the audit-session-id.  So I think the switch doesn't know for which client the CoA is destined for, according to the RFC the Meraki switch should reply with a NAK, bit Meraki seems to just ignore it...  According to the Meraki documentation, the Session ID is learned from the original Radius access accept message, the thing is that my device gets a rejected the 1th time (Acces-Reject), so that might be the reason it doesn't supply the CoA with a audit-session-id (?). 

How I reproduce it:

1. Delete supplicant from endpoint store, this is triggering a CoA reauthentication request, which is getting acknowledged by the Meraki switch. Switchport reauthentication is triggered and the supplicant is performing an Access-request again.

2. Endpoint shows up again in endpoint store, gets rejected by the default deny authorization policy. In the PCAP an access-request and access-reject is seen

3. After that the supplicant stays unauthenticated for quite some time. CoA requests are seen at this point, they do not contain a audit-session-ID. These packets do arrive at the Meraki switch but an ACK/NAK is never given.

4. After manually cycling the port, or waiting for the supplicant to reauthenticate on its own again, a new access-request is being made, responded with Access-Accept, which contains the correct profile. The client works now. 

I'm still looking into the duplicated packets, not sure what is causing that. 

Do you think it is correct to assume that the supplicant didn't get profiled yet on the 1th try, causing an access-reject. After ISE profiles it, ISE sends a CoA to trigger the client to perform a reauthentication request again? 

[Arne Bier · Accepted solution]

@Jagermeister - good point about the Access-Reject. I should have picked up on that sooner - with Monitor Mode and Low Impact Mode we should never have a default Access-Reject at the end of the Authorization Profile

My MAB Monitor Mode and Low Impact Authorization Policy Sets always end like this:

The final Default will never get matched and will always have zero hits.

[Jagermeister]

Hi Arne,

Yes sir, that is it !

After thinking this through it makes sense. The Meraki switch learns the CoA session ID from the access-accept messages but putting a default deny in it prevents CoA to work properly.  I've created a catchall policy with limited access, which will provide the unprofiled client an access-accept with a session ID, now the Session ID can be learned and is present in the CoA after ISE profiles the client. Result; works perfect, yay!

Thanks a lot for your help, I've accepted your response as the solution. 

[MHM Cisco World]

there are three option for profiling 
no CoA 

port bounce 

ReAuth

since as I understand you want to use profiling for printer try use port bounce instead of CoA, again no need CoA 

[Jagermeister]

Hi,

Port bounce is a type of CoA. There is a valid need for CoA since the authorization changes after the supplicant gets profiled by ISE. If I wouldn't use CoA, then I will have to wait until the device itself decides to do a reauthentication request or I would have to cycle the switchport, which is not desired behavior.