DHCP Sooping Binding Table doesnot clear

xishan · ‎12-01-2023

Hi,

I am having C2960X Switch, causing "shut/not shut" and "%SW_DAI-4-DHCP_SNOOPING_DENY:" logs everyday:

%LINK-3-UPDOWN: Interface GigabitEthernet3/0/9, changed state to up
%LINK-3-UPDOWN: Interface GigabitEthernet3/0/9, changed state to down
%LINK-3-UPDOWN: Interface GigabitEthernet3/0/9, changed state to up
%SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi3/0/9, vlan 20.([1628.9ad8.44df/169.254.174.73/0000.0000.0000/169.254.174.73/10:22:25 Fri Dec 1 2023])

What I have found out is that, the DHCP Snooping Binding Table is keeping the old binding even there was no user connected to port for last 12 hours. In that case, when a new user connects to a port the ARP checks the Binding table and finds that there is already an entry. So the ports goes shut.

My current Config is:

ip dhcp snooping vlan 20
no ip dhcp snooping information option
ip dhcp snooping

ip arp inspection vlan 20
ip arp inspection vlan 20 logging acl-match matchlog
ip arp inspection vlan 20 logging dhcp-bindings all

interface GigabitEthernet3/0/9
switchport access vlan 20
switchport mode access
ip arp inspection limit rate 120
ip access-group ACL-GROUP1 in
authentication control-direction in
authentication event server dead action authorize vlan 20
authentication host-mode multi-auth
authentication open
authentication order mab dot1x
authentication port-control auto
authentication timer reauthenticate server
mab
mls qos trust dscp
dot1x pae authenticator
service-policy input access
end

What else should be configured that the DHCP Snooping Binding table clear the entries so I don't get the DAI Logs?

Thanks

MHM Cisco World · ‎12-01-2023

Can I see

Show dhcp snooping table

MHM

xishan · ‎12-04-2023

Hi @MHM Cisco World

Here is an example of what is currently in DHCP Snooping table for one VLAN and interfaces including /42.

MacAddress IpAddress Lease(sec) Type VLAN Interface
------------------ --------------- ---------- ------------- ---- --------------------
XX:F9 10.93.20.64 627844 dhcp-snooping 22 GigabitEthernet1/0/42
XX:28 10.93.20.117 863330 dhcp-snooping 22 GigabitEthernet3/0/42
XX:1E 10.93.20.144 863509 dhcp-snooping 22 GigabitEthernet1/0/42
XX:56 10.93.20.164 525922 dhcp-snooping 22 GigabitEthernet1/0/42
XX:F9 10.93.20.75 552464 dhcp-snooping 22 GigabitEthernet2/0/42
XX:C2 10.93.20.58 374202 dhcp-snooping 22 GigabitEthernet1/0/42
XX:10 10.93.20.86 465126 dhcp-snooping 22 GigabitEthernet2/0/42

Whole table is quite big as its a stack of 5.

balaji.bandi · ‎12-01-2023

what version of IOS code running on this device ?

Cat 2960 default arp timeout 4 hours - check that. ?

how is your DHCP leased time setup ?

can you try enable device tracking ?

https://www.cisco.com/c/en/us/support/docs/lan-switching/8021x/119374-technote-dacl-00.html

i have seen that you have first authentication MAB and 802.1x is that intention ?

also refer below guide for reference :

https://community.cisco.com/t5/networking-knowledge-base/the-quot-sw-dai-4-dhcp-snooping-deny-quot-error-message-is/ta-p/3132652

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

xishan · ‎12-04-2023

Hi @balaji.bandi

The IOS version ist 15.2(7)E6
ARP type: ARPA, ARP Timeout 04:00:00
Lease Time: 10 Days (Could this be the problem)

No information comes in DHCP Snooping debugging.

#sh run all | i dhcp
service dhcp
psp dhcp pps 0
ip arp inspection vlan 20 logging dhcp-bindings all
ip dhcp relay information policy replace
ip dhcp relay information check
ip dhcp relay override giaddr link-selection
ip dhcp use class
ip dhcp use vrf connected
ip dhcp binding cleanup interval 120
ip dhcp compatibility suboption link-selection cisco
ip dhcp conflict logging
ip dhcp ping packets 2
ip dhcp ping timeout 500
ip dhcp snooping vlan 20
no ip dhcp snooping information option allow-untrusted
no ip dhcp snooping information option unicast
no ip dhcp snooping information option
no ip dhcp snooping database
ip dhcp snooping database write-delay 300
ip dhcp snooping database timeout 300
ip dhcp snooping verify mac-address
ip dhcp snooping verify no-relay-agent-address
no ip dhcp snooping wireless bootp-broadcast enable
ip dhcp snooping

#show logging
%SW_DAI-4-SPECIAL_LOG_ENTRY: 10 Invalid ARP packets [09:23:00 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 12 Invalid ARP packets [09:25:35 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 3 Invalid ARP packets [09:28:38 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 15 Invalid ARP packets [09:28:47 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 7 Invalid ARP packets [09:28:47 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 9 Invalid ARP packets [09:29:38 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 2 Invalid ARP packets [09:31:09 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 9 Invalid ARP packets [09:32:11 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 16 Invalid ARP packets [09:34:05 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 15 Invalid ARP packets [09:35:17 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 14 Invalid ARP packets [09:36:31 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 10 Invalid ARP packets [09:38:45 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 16 Invalid ARP packets [09:42:23 CET Mon Dec 4 2023]
%SW_DAI-4-SPECIAL_LOG_ENTRY: 15 Invalid ARP packets [09:43:01 CET Mon Dec 4 2023]

#show ip device tracking all
Global IP Device Tracking for clients = Enabled
Global IP Device Tracking Probe Count = 3
Global IP Device Tracking Probe Interval = 120
Global IP Device Tracking Probe Delay Interval = 60
-----------------------------------------------------------------------------------------------
IP Address MAC Address Vlan Interface Probe-Timeout State Source
-----------------------------------------------------------------------------------------------
10.93.20.52 xxxx.xxxx.xx25 22 GigabitEthernet2/0/44 120 INACTIVE ARP
10.93.20.53 xxxx.xxxx.xxa1 22 GigabitEthernet1/0/19 120 ACTIVE ARP
192.168.4.110 xxxx.xxxx.xx38 4 GigabitEthernet2/0/9 120 ACTIVE ARP
10.93.20.54 xxxx.xxxx.xx41 22 GigabitEthernet2/0/17 120 ACTIVE ARP
10.93.10.42 xxxx.xxxx.xx63 10 GigabitEthernet1/0/14 120 ACTIVE ARP
10.93.10.43 xxxx.xxxx.xxc3 10 GigabitEthernet2/0/9 120 ACTIVE ARP
10.93.20.55 xxxx.xxxx.xxfe 22 GigabitEthernet2/0/8 120 INACTIVE ARP
10.93.20.48 xxxx.xxxx.xx58 22 GigabitEthernet1/0/6 120 ACTIVE ARP
192.168.5.104 xxxx.xxxx.xx62 4 GigabitEthernet2/0/4 120 ACTIVE ARP
10.93.10.45 xxxx.xxxx.xx5e 10 GigabitEthernet2/0/3 120 ACTIVE ARP
10.93.20.49 xxxx.xxxx.xxd3 22 GigabitEthernet2/0/29 120 ACTIVE ARP
192.168.5.107 xxxx.xxxx.xxcf 4 GigabitEthernet5/0/37 120 ACTIVE ARP
10.93.10.46 xxxx.xxxx.xx8d 10 GigabitEthernet2/0/9 120 ACTIVE ARP
10.93.20.50 xxxx.xxxx.xx90 22 GigabitEthernet3/0/45 120 INACTIVE ARP
192.168.7.104 xxxx.xxxx.xxb6 4 GigabitEthernet1/0/8 120 ACTIVE ARP
10.93.20.61 xxxx.xxxx.xx25 22 GigabitEthernet2/0/26 120 INACTIVE ARP
10.93.20.62 xxxx.xxxx.xx89 22 GigabitEthernet1/0/9 120 ACTIVE ARP
10.93.20.59 xxxx.xxxx.xx0e 22 GigabitEthernet4/0/12 120 INACTIVE ARP
10.93.10.57 xxxx.xxxx.xxa5 10 GigabitEthernet2/0/4 120 ACTIVE ARP

MAB and DOT1X was already configured by someone else. What I can see on ISE that device first try MAB and fail, then they try DOT1X and get authenticated. I only know that some device carry no auth certificate. Should the recommended way be DOT1X and then MAB?

Thanks

balaji.bandi · ‎12-04-2023

%SW_DAI-4-SPECIAL_LOG_ENTRY: 10 Invalid ARP packets [09:23:00 CET Mon Dec 4 2023]

this is ok for indication. as below :

Error Message    SW_DAI-4-SPECIAL_LOG_ENTRY: [dec] Invalid ARP packets [[time-of-day]].

Explanation This message means that the switch has received ARP packets considered invalid by ARP inspection. The packets are erroneous, and their presence might show attempted man-in-the-middle attacks in the network. This message differs from other SW_DAI messages in that this message captures all messages when the rate of incoming packets exceeds the dynamic ARP inspection logging rate. [dec] is the number of invalid ARP packets, and [time-of-day] is the time of day.

==========

Do you have any network performance issue or users complaining about DHCP issue ?

DOT1X and then MAB - this is industry suggestions (MAB is not secure as 802.1x for some device there is no suplicant so they go use MAB as alternative option).

You can find more logs here and what action need to be done :

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3560/software/release/12-2_25_sed/system/message/smg/msg_desc.html#wp361480

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

xishan · ‎12-05-2023

Hi @balaji.bandi

Thanks for the link, one of my problems %SW_DAI-4-DHCP_SNOOPING_DENY is answered:

This error comes when the user connects with a port, ARP checks DHCP Snooping binding table, no binding is found, we get this error. DHCP does its work, we get IP, binding table will be updated and things work, no action is required.

- Now the second problem, random ports are going down and coming back up and I can only see the ports up/down and %SW_DAI-4-SPECIAL_LOG_ENTRY logs on Switch. Even though there are no connectivity on the port since last couple of days

For Example Port Gi3/0/39:

#show logging

Dec 5 2023 08:20:20.328 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to up
Dec 5 2023 08:21:00.735 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to down
Dec 5 2023 08:38:49.460 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to up
Dec 5 2023 08:39:00.100 CET: %SW_DAI-4-SPECIAL_LOG_ENTRY: 27 Invalid ARP packets [08:38:53 CET Tue Dec 5 2023]
Dec 5 2023 10:17:10.146 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to down

#show int GigabitEthernet3/0/39
Last input 3d19h, output 01:18:13, output hang never

There is no user connected on the port and the question is: Is DAI causing the port go down/up? Because most of the ports showing up/down logs are where I can see a user connected.

Why are the random ports going up/down?

Your Questions:

Do I have any network performance issue or users complaining about DHCP issue?
Yes, the users sometimes say that the Internet Connection is slow but no DHCP issues.

DOT1X and then MAB
There are some devices i.e. Printers, etc. which do not use Certificates so we have configues it like mab and then dot1x. We are considering to give a try using dot1x and then mab. But we don't know what error will we face by doing so.

Thanks

balaji.bandi · ‎12-05-2023

Do I have any network performance issue or users complaining about DHCP issue?
Yes, the users sometimes say that the Internet Connection is slow but no DHCP issues.

- This need to investigate where the bottleneck issue in the network , need to test each area.

DOT1X and then MAB
There are some devices i.e. Printers, etc. which do not use Certificates so we have configues it like mab and then dot1x. We are considering to give a try using dot1x and then mab. But we don't know what error will we face by doing so.

- i understand printer need MAB, they go 802.1x fails and go to MAB with time.

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

MHM Cisco World · ‎12-04-2023

We start from dot1x because I think it issue here'

You use multi-auth host mode which meaning that multi host connect to same port'

The success auth host add as static to port

Now if the host not more connect and it not send eap logoff the host still see as connect' why because the SW un authz the port when link down or eap logoff receive.

I see you config timer reauth via server but you dont config port with dot1x reauth

Try add it to one port and see.

MHM

xishan · ‎12-05-2023

Hi @MHM Cisco World

I am considering to change the auth order from mab dot1x to first dot1x and then mab. We don't know what problems will the clinet face by doing so, I am trying to implement the same scenario in LAB and will be changing the config.

Right now the big question is, why is DHCP Snooping binding table not clearing when there is no user connected to ports. It shows the binding lease for like 5-6 or even more days?

#sh ip dhcp snooping binding | i 3/0/39
MacAddress IpAddress Lease(sec) Type VLAN Interface
------------------ --------------- ---------- ------------- ---- ------
XX:XX:XX:E2 10.42.20.56 848262 dhcp-snooping 22 GigabitEthernet3/0/39
XX:XX:XX:53 10.42.20.206 528391 dhcp-snooping 22 GigabitEthernet3/0/39

#sh int GigabitEthernet3/0/39
Last input 3d21h, output 02:44:46, output hang never

Dec 5 2023 08:20:20.328 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to up
Dec 5 2023 08:21:00.735 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to down
Dec 5 2023 08:38:49.460 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to up
08:39:00.100 CET: %SW_DAI-4-SPECIAL_LOG_ENTRY: 27 Invalid ARP packets [08:38:53 CET Tue Dec 5 2023]
Dec 5 2023 10:17:10.146 CET: %LINK-3-UPDOWN: Interface GigabitEthernet3/0/39, changed state to down

Is DAI causing the up/down? If yes, why when there is even no user connected on port since a couple of days?

MHM Cisco World · ‎12-05-2023

One by one

First as I mention you use mutli auth' can you check

Show auth session

Check if the user appears in dhcp snooping binding appear also in auth session.

MHM

xishan · ‎12-05-2023

Yes the user appears in the Auth Session as well as the DHCP snooping binding table.

Multi-Auth is configured becasue some users use also use IP Phones for voice.

MHM Cisco World · ‎12-05-2023

As I mention you need to make SW use reauth' I see only timer command.

This way the SW remove host (not connect) from auth session.

And DAI and dhcp snooping update it entry according to it.

Just do test in one port

Add

Dot1x reauth

Connect pc

Then disconnect pc

Check all table see if pc is remove or not.

MHM

David Ruess · ‎12-04-2023

Hello,

DAI uses the DHCP Snooping database to allow/dis-allow ARP packets. If the DHCP IP/PC MAC combination is in the DHCP snooping database then it allows it. The fact you are getting an error that is not allowing the ARP message on the newly added PC is likely due to the fact that its information hasn't been populated in to the DHCP Snooping binding table. The host needs to go through the DHCP process so the switch can look through the packets and add the needed information. A couple of questions:

1. Is the PC able to get a DHCP address? You note that the debug of the DHCP snooping yields no results which makes me think it snot completing that processes first, which is required to populate the snooping Database.

2. Does the PC have a static IP? If this is the case you need to configure an IP source binding ACL to put a static entry in the DHCP snooping database.

You can also try clearing just that port of the binding (DO NOT CLEAR THE WHOLE DATABASE)

clear ip dhcp snooping binding interface <interface>

Hope this information helps

-David

xishan · ‎12-05-2023

Hi @David Ruess

Yes, hosts are getting the IPs from DHCP Server even there appeared nothing while debugging ip dhcp snooping, and they don't have static IP addresses. Currently, logging is full of random port up/down logs even there is no user connected to ports but ports go down and come back up.

As far as I have checked,ARP and DHCP Snooping config on Switch is right. I can't do much in Production so I am preparing a Switch in Lab with same config, IOS version and will check all the possible options there. I will change the DHCP Lease time even I don't think that DHCP is causing a problem here but I will still check it.

I am wondering why the DHCP Snooping table is not flushing all the bindings when the ip dhcp snooping binding leanup interval 120 is configured. Will also try removing the bindings on a port myself and will share the result.

Thanks