topic Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION in Network Access Control

Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Wed, 09 Oct 2024 05:41:32 GMT

Hi Expert,

Recently, one port on our 9200 switch went to err-disable state and the status LEDs are all off.

The port configuration is as below: it is connected to a desktop PC

The logs is as below, it looks before it goes to err-disable state, the port up and down for many times. seems the user was rebooting the machine

during error, checked the controller:

The machine authentication in ISE is not responded:

I just found this link seems the same issue, but not sure why the pc's MAC changed?

https://github.com/inverse-inc/packetfence/issues/1588

It occurred the second time, and can be resolved by shutdown/no shutdown to reset the port.

But we still need to find out the root cause. Any ideas about it?

Thanks

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

Leo Laohoo — Wed, 09 Oct 2024 01:13:10 GMT

Remove all the config off that interface and see how many MAC addresses are coming out of that port.

A PC hosting VM is one suspect for having two (or more) MAC addresses. Another is laptop dock because the dock itself has one MAC address and the laptop has another.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Wed, 09 Oct 2024 02:49:54 GMT

Thanks very much for your reply, the user is using a desktop and doesn't need a dock, I think he won't touch the network cable when it is working. It seems occured when he shut down/reboot the desktop.

I set up a remote PS session to his desktop and only 2 adaptors found, no such MAC 0848.8905.3e52 found

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Wed, 09 Oct 2024 05:46:58 GMT

Just found another port the same error at several days ago ( we moved the same desktop to the new port), but the MAC address is not the same

Also I checked the port-security, it seems not enabled,

but the MAC address learnt from this port is showing "STATIC"

I assume the "sticky" is still on? or it is because below: authentication violation default is "shutdown"?

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

MHM Cisco World — Wed, 09 Oct 2024 08:40:27 GMT

I need to see log detail

Share it here

MHM

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Wed, 09 Oct 2024 11:36:19 GMT

"Date","Time","Facility","Level","Host Name","Message Text"
"2024-10-09","17:22:43","Local7","Notice","10.2.9.5","37464: Oct 9 17:22:43.162: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-09","17:22:43","Local7","Error","10.2.9.5","37463: Oct 9 17:22:42.162: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-09","17:22:39","Local7","Error","10.2.9.5","37462: Oct 9 17:22:38.936: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-09","17:22:38","Local7","Notice","10.2.9.5","37461: Oct 9 17:22:37.932: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-09","17:22:33","Local7","Notice","10.2.9.5","37460: Oct 9 17:22:33.894: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-09","17:22:33","Local7","Error","10.2.9.5","37459: Oct 9 17:22:32.893: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-09","17:22:28","Local7","Error","10.2.9.5","37458: Oct 9 17:22:27.552: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-09","17:22:27","Local7","Notice","10.2.9.5","37457: Oct 9 17:22:26.557: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-09","09:55:45","Local7","Notice","10.2.9.5","36587: Oct 9 09:55:44.726: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-09","09:55:44","Local7","Error","10.2.9.5","36586: Oct 9 09:55:43.724: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-09","09:55:36","Local7","Error","10.2.9.5","36583: Oct 9 09:55:35.900: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-09","09:55:33","Local7","Notice","10.2.9.5","36581: Oct 9 09:55:32.588: %LINK-5-CHANGED: Interface GigabitEthernet8/0/41, changed state to administratively down"
"2024-10-09","09:55:28","Local7","Notice","10.2.9.5","36579: Oct 9 09:55:27.970: %PARSER-5-CFGLOG_LOGGEDCMD: User:admin logged command:interface GigabitEthernet8/0/41 "
"2024-10-08","16:46:47","Local7","Notice","10.2.9.5","35580: Oct 8 16:46:46.048: %AUTHMGR-5-SECURITY_VIOLATION: Security violation on the interface GigabitEthernet8/0/41, new MAC address (0848.8905.3e52) is seen.AuditSessionID Unassigned"
"2024-10-08","16:46:46","Local7","Warning","10.2.9.5","35579: Oct 8 16:46:46.031: %PM-4-ERR_DISABLE: security-violation error detected on Gi8/0/41, putting Gi8/0/41 in err-disable state"
"2024-10-08","16:46:43","Local7","Error","10.2.9.5","35578: Oct 8 16:46:42.796: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","16:46:42","Local7","Notice","10.2.9.5","35577: Oct 8 16:46:41.794: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","16:46:38","Local7","Notice","10.2.9.5","35576: Oct 8 16:46:38.830: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-08","16:46:38","Local7","Error","10.2.9.5","35575: Oct 8 16:46:37.829: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-08","16:46:31","Local7","Error","10.2.9.5","35574: Oct 8 16:46:31.625: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","16:46:31","Local7","Notice","10.2.9.5","35573: Oct 8 16:46:30.627: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","09:39:51","Local7","Notice","10.2.9.5","35107: Oct 8 09:39:51.062: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-08","09:39:51","Local7","Error","10.2.9.5","35106: Oct 8 09:39:50.060: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-08","09:39:47","Local7","Error","10.2.9.5","35105: Oct 8 09:39:46.725: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","09:39:46","Local7","Notice","10.2.9.5","35104: Oct 8 09:39:45.721: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","09:39:38","Local7","Notice","10.2.9.5","35103: Oct 8 09:39:38.727: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-08","09:39:38","Local7","Error","10.2.9.5","35102: Oct 8 09:39:37.725: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-08","09:39:32","Local7","Error","10.2.9.5","35101: Oct 8 09:39:31.529: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-08","09:39:31","Local7","Notice","10.2.9.5","35100: Oct 8 09:39:30.572: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","17:11:08","Local7","Notice","10.2.9.5","32778: Oct 4 17:11:07.805: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","17:11:07","Local7","Error","10.2.9.5","32777: Oct 4 17:11:06.808: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","17:11:04","Local7","Error","10.2.9.5","32776: Oct 4 17:11:03.592: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","17:11:03","Local7","Notice","10.2.9.5","32775: Oct 4 17:11:02.589: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","17:10:58","Local7","Notice","10.2.9.5","32773: Oct 4 17:10:58.714: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","17:10:58","Local7","Error","10.2.9.5","32772: Oct 4 17:10:57.714: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","17:10:52","Local7","Error","10.2.9.5","32771: Oct 4 17:10:52.396: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","17:10:52","Local7","Notice","10.2.9.5","32770: Oct 4 17:10:51.397: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","14:31:50","Local7","Notice","10.2.9.5","32693: Oct 4 14:31:50.942: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","14:31:50","Local7","Error","10.2.9.5","32692: Oct 4 14:31:49.940: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","14:31:47","Local7","Error","10.2.9.5","32691: Oct 4 14:31:46.717: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","14:31:46","Local7","Notice","10.2.9.5","32690: Oct 4 14:31:45.712: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","14:31:41","Local7","Notice","10.2.9.5","32689: Oct 4 14:31:41.986: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","14:31:41","Local7","Error","10.2.9.5","32688: Oct 4 14:31:40.985: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-04","14:31:36","Local7","Error","10.2.9.5","32687: Oct 4 14:31:35.513: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-04","14:31:35","Local7","Notice","10.2.9.5","32686: Oct 4 14:31:34.509: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-03","16:49:23","Local7","Notice","10.2.9.5","31747: Oct 3 16:49:22.984: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-03","16:49:23","Local7","Error","10.2.9.5","31746: Oct 3 16:49:21.981: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to up"
"2024-10-03","16:49:19","Local7","Error","10.2.9.5","31745: Oct 3 16:49:18.744: %LINK-3-UPDOWN: Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-03","16:49:18","Local7","Notice","10.2.9.5","31744: Oct 3 16:49:17.740: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to down"
"2024-10-03","16:49:10","Local7","Notice","10.2.9.5","31743: Oct 3 16:49:10.859: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/0/41, changed state to up"

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Wed, 09 Oct 2024 11:37:49 GMT

it stuck at 16:46 after the user left the office and then next day the admin run shutdown/no shutdown to resolve the issue.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

MHM Cisco World — Wed, 09 Oct 2024 14:35:04 GMT

there Steps in detail explain the error in ISE share it

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Wed, 09 Oct 2024 22:53:48 GMT

Steps
11001 Received RADIUS Access-Request
11017 RADIUS created a new session
15049 Evaluating Policy Group
15008 Evaluating Service Selection Policy
15048 Queried PIP - Normalised Radius.RadiusFlowType
15048 Queried PIP - DEVICE.Device Type
11507 Extracted EAP-Response/Identity
12500 Prepared EAP-Request proposing EAP-TLS with challenge
12625 Valid EAP-Key-Name attribute received
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12502 Extracted EAP-Response containing EAP-TLS challenge-response and accepting EAP-TLS as negotiated
12800 Extracted first TLS record; TLS handshake started
12545 Client requested EAP-TLS session ticket
12542 The EAP-TLS session ticket received from supplicant while the stateless session resume is disabled. Performing full authentication
12805 Extracted TLS ClientHello message
12806 Prepared TLS ServerHello message
12807 Prepared TLS Certificate message
12808 Prepared TLS ServerKeyExchange message
12809 Prepared TLS CertificateRequest message
12810 Prepared TLS ServerDone message
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
11001 Received RADIUS Access-Request
11018 RADIUS is re-using an existing session
12504 Extracted EAP-Response containing EAP-TLS challenge-response
12505 Prepared EAP-Request with another EAP-TLS challenge
11006 Returned RADIUS Access-Challenge
12935 Supplicant stopped responding to ISE during EAP-TLS certificate exchange (step latency=120000 ms Step latency=120000 ms)
61025 Open secure connection with TLS peer
5411 Supplicant stopped responding to ISE

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

Arne Bier — Thu, 10 Oct 2024 05:15:33 GMT

The port security err-disable should only have kicked in because there was a violation in the number of MAC addresses seen in the DATA domain - and this restriction only applies to multi-domain
I am fairly sure that multi-auth is the default (hidden from show run) in IOS - but you could try to confiure it manually

conf t interface gig 8/0/41 access-session host-mode multi-auth end

In multi-auth mode the switch will allow multiple MAC addresses in the DATA domain, and every MAC address is subject to RADIUS auth. It doesn't explain WHY you get spurious MAC addresses on the switch, but I have never found multi-domain to be a reliable config method, simply because some phones can have the nasty ide effect of first landing their MAC in the DATA domain , before the flip over to the VOICE domain. If, at this time a PC is connected to the back of the phone, then you have 2 MAC addresses in the DATA domain, albeit for a split second - but this is enough to err-disable the interface.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

MHM Cisco World — Thu, 10 Oct 2024 10:20:39 GMT

&&Windows GPO

Your windows GPO for wireless and wired network is also important. Open your 802.1x GPO and navigate to Security Settings -> Wired Network (802.3) Policies -> Network Profile -> IEEE 802.1X Settings.

Check the following settings and set them accordingly:

Setting	Value
Maximum Authentication Failures	5
Maximum EAPOL-Start Messages Sent	5
Held Period (seconds)	1
Start Period (seconds)	5
Authentication Period (seconds)	30

The values are suggestions to start with. They may need to be adjusted to fit your environment.

&&Client Firewall and Antivirus

Your clients firewall or antivirus could be the problem, especially if you are using authentication timeouts in your ISE policies. For testing purposes you can temporarily disable your firewall or antivirus and check if the problem persists.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Fri, 11 Oct 2024 00:56:47 GMT

Thanks very much for your reply. Only this desktop PC has the issue and the others which are in the same domain with same settings don't have the issue. Also, it seems another new MAC triggered the security violation, which means not the failures. I've checked no other devices like ip phones, the cable connected to the PC directly and the other end is connected to data point under the desk. I've changed all the cables and changed another data point, to see if it happened again.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Fri, 11 Oct 2024 05:26:05 GMT

Thanks very much for your reply.

It happened again today, with another new MAC address been seen. That's very strange, I can't find the new MAC coming from where...

We don't have IP phones or HUB or deck for the PC.

Maybe we can set it to authentication violation restrict or replace, because we use Windows 10 machine certificate and user certificate for authentication, we don't mind what MAC address it is.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Fri, 11 Oct 2024 02:10:12 GMT

I tested again, the port doesn't have any "sticky" feature, as I used 2 different usb network adaptor to connect to the same port. They have different MAC addresses and both got DHCP IP and working properly.

Just suspect where is the new MAC coming from, only 2 adaptors on the PC, one is the normal one, another is the Zscaler adaptor, none of them have the above detected "new MAC is seen.." that's very odd.

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Fri, 11 Oct 2024 05:22:49 GMT

Today morning the issue happened again. The same issue same desktop, and we can see that there are some up and downs yesterday afternoon, the last state at yesterday is UP, and suddenly it went to err-disable at today morning, other new MAC address appeared. The user is working from home and RDP to the desktop. no other devices at all.. where is the MAC address from..

I found this bug: https://bst.cisco.com/quickview/bug/CSCus17694 but our version is:

Cisco IOS XE Software, Version 17.09.04a
Cisco IOS Software [Cupertino], Catalyst L3 Switch Software (CAT9K_LITE_IOSXE), Version 17.9.4a, RELEASE SOFTWARE (fc3)

is it possible to verify whether the memory is leaky or not , when it happened or after I fixed it?

A work around might be setup the err-disable recovery to 5 seconds?

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

Arne Bier — Fri, 11 Oct 2024 05:39:35 GMT

Wow - I hope it's not a bug - I guess one way to find out is to reboot the switch and then observe - if it's a slow memory leak then you might not see this issue for a while. To be honest, I have never seen an err-disable on any of my NAC deployments, because I only use host mode "multi-auth". The only time I have seen err-disable in NAC deployments, was in the lab when I configured host mode multi-domain and then provoked the switch to cause the issue.

If you're 100% sure you have configured multi-auth, then you should NOT ever encounter err-disable. Unless there is some other funky thing I didn't know about port security. Have you opened a TAC case?

Re: Cisco 9200 err-disable AUTHMGR-5-SECURITY_VIOLATION

117222400 — Fri, 18 Oct 2024 05:40:44 GMT

Thanks for your reply, we've opened a case to TAC, and they collected some packets on the port.

We also changed the data port on the user's desk and the network cable that connected to the user's PC, the issue didn't occur for a week now. not sure if it is a cable or patching panel issue.