01-11-2013 05:35 AM - edited 03-10-2019 12:20 PM
We have a 3750 running IOS ver 12.2 (44) SE, it has performed great and we have never had a problem with it. However we have noticed that when we had an outage some of our Wireless APs didn't come up as they get DHCP from the 3750. The DHCP scope said IP was depleted although there were IPs to give. We had to delete and recreate the DHCP Pool. However two days later we got the same problem and then had to do the same thing over again. Any Ideas?
Solved! Go to Solution.
01-11-2013 07:02 AM
You are getting conflicts in at least two different subnets on this device I see them in both the 192.168.12.X and the 192.168.4.X. The impact these conflicts will have on the subnets will vary greatly depending on: the size of the subnets, the number of clients attempting to obtain addresses, and the order in witch the clients attempt to renew their addresses.
Basically the problem goes as follows:
-The switch hands out addresses and gives: address 1 to client a, address 2 to client b, address 3 to client c, ect...
-The switch reloads and loeses its database, it now has no idea that it has given any address to any client.
-Client d attempts to obtain an IP address, the switch says "lets give him address 1" but then it finds that address 1 is in use (by client a) and says "I will mark that as a bad address and not give that out" it then repeats this process until it finds an address that is not in use, marking everyone it can ping as bad along the way.
-Client a then attempts to renew its IP address, but the switch says " no you can't have that, it's a badd address" and gives it the first good address it has.
Depending on the utilization of the subnet, the switch can end up filling up a whole pool with bad addresses and not have any good ones to hand out. If the utilization is low in the subnet, it may have enough to give everyone a new address on top of the bad addresses and you would not know you have a problem.
01-11-2013 06:00 AM
Hi,
Any specific log entires or errors to go by? What was the output of the following commands?
sh ip dhcp server statistics
sh ip dhcp pool
sh ip dhcp binding
sh ip dhcp conflict
You may also want to run a
deb ip dhcp server events when this issue happens for more verbose logging.
Regards
Stephen
==========================
http://www.rConfig.com
A free, open source network device configuration management tool, customizable to your needs!
- Always vote on an answer if you found it helpful
01-11-2013 06:11 AM
Library_3750_L3#sho ip dhcp server statistics
Memory usage 9419
Address pools 2
Database agents 0
Automatic bindings 18
Manual bindings 0
Expired bindings 355
Malformed messages 30
Message Received
BOOTREQUEST 162818
DHCPDISCOVER 679538
DHCPREQUEST 717559
DHCPDECLINE 204
DHCPRELEASE 7015
DHCPINFORM 791935
Message Sent
BOOTREPLY 0
DHCPOFFER 16669
DHCPACK 97891
DHCPNAK 887
Library_3750_L3#sh ip dhcp binding
IP address Client-ID/ Lease expiration Type
Hardware address
192.168.4.101 0100.1fca.2843.d8 Jan 12 2013 01:23 PM Automatic
192.168.4.102 0100.1fca.2844.42 Jan 12 2013 01:23 PM Automatic
192.168.4.104 0100.1819.bdbb.4a Jan 12 2013 01:23 PM Automatic
192.168.4.105 0100.1ae2.531a.82 Jan 12 2013 01:23 PM Automatic
192.168.4.109 0100.2414.d23f.e8 Jan 12 2013 01:23 PM Automatic
192.168.4.110 0100.1fca.2844.1c Jan 12 2013 01:23 PM Automatic
192.168.4.114 0100.1ae2.5315.de Jan 12 2013 01:23 PM Automatic
192.168.4.119 01d4.8cb5.b005.96 Jan 12 2013 01:23 PM Automatic
192.168.4.120 0100.1aa2.3354.4a Jan 12 2013 01:23 PM Automatic
192.168.4.122 01c8.4c75.a989.74 Jan 12 2013 01:23 PM Automatic
192.168.4.125 0100.1aa2.3350.2c Jan 12 2013 01:23 PM Automatic
192.168.4.127 0100.1fca.27e5.0c Jan 12 2013 01:23 PM Automatic
192.168.4.128 0100.1fca.2843.78 Jan 12 2013 01:24 PM Automatic
192.168.4.131 0100.1aa2.3350.38 Jan 12 2013 01:24 PM Automatic
192.168.4.133 0130.f70d.e65f.d9 Jan 12 2013 01:24 PM Automatic
192.168.14.95 0100.175a.df7e.81 Jan 12 2013 01:24 PM Automatic
192.168.14.96 0100.2414.b446.27 Jan 12 2013 01:27 PM Automatic
192.168.14.97 0100.1759.3f49.87 Jan 12 2013 01:29 PM Automatic
Library_3750_L3#sh ip dhcp conflict
IP address Detection method Detection time
192.168.12.29 Gratuitous ARP Mar 01 1993 12:11 AM
192.168.12.26 Gratuitous ARP Mar 01 1993 12:11 AM
192.168.12.75 Ping Mar 01 1993 12:12 AM
192.168.12.27 Gratuitous ARP Mar 01 1993 12:12 AM
192.168.12.130 Ping Mar 01 1993 12:53 AM
192.168.12.131 Ping Mar 01 1993 12:53 AM
192.168.12.138 Ping Mar 01 1993 12:54 AM
192.168.12.168 Ping Mar 01 1993 01:30 AM
192.168.12.187 Ping Mar 01 1993 03:14 AM
192.168.12.188 Ping Mar 01 1993 03:14 AM
192.168.12.189 Ping Mar 01 1993 03:14 AM
192.168.12.190 Ping Mar 01 1993 03:14 AM
192.168.12.191 Ping Mar 01 1993 03:14 AM
192.168.12.192 Ping Mar 01 1993 03:14 AM
192.168.12.193 Ping Mar 01 1993 03:14 AM
192.168.12.194 Ping Mar 01 1993 03:14 AM
192.168.12.195 Ping Mar 01 1993 03:14 AM
192.168.12.196 Ping Mar 01 1993 03:14 AM
192.168.4.100 Ping Jan 11 2013 01:23 PM
192.168.4.103 Ping Jan 11 2013 01:23 PM
192.168.4.106 Ping Jan 11 2013 01:23 PM
192.168.4.107 Ping Jan 11 2013 01:23 PM
192.168.4.108 Ping Jan 11 2013 01:23 PM
192.168.4.111 Ping Jan 11 2013 01:23 PM
192.168.4.112 Ping Jan 11 2013 01:23 PM
192.168.4.113 Ping Jan 11 2013 01:23 PM
192.168.4.115 Ping Jan 11 2013 01:23 PM
192.168.4.116 Ping Jan 11 2013 01:23 PM
192.168.4.117 Ping Jan 11 2013 01:23 PM
192.168.4.118 Ping Jan 11 2013 01:23 PM
192.168.4.121 Ping Jan 11 2013 01:23 PM
192.168.4.123 Ping Jan 11 2013 01:23 PM
192.168.4.124 Ping Jan 11 2013 01:23 PM
192.168.4.126 Ping Jan 11 2013 01:23 PM
192.168.4.129 Ping Jan 11 2013 01:23 PM
192.168.4.130 Ping Jan 11 2013 01:23 PM
192.168.4.132 Ping Jan 11 2013 01:23 PM
Library_3750_L3#sh ip dhcp ?
binding DHCP address bindings
conflict DHCP address conflicts
database DHCP database agents
import Show Imported Parameters
relay Miscellaneous DHCP relay information
server Miscellaneous DHCP server information
snooping DHCP snooping
01-11-2013 06:02 AM
Are you storing the DHCP file in persistent memory? The defualt location for a Cisco device to store its DHCP database is in RAM. When the device reloads it loses the database and you wind up with conflicts as the device trys to hand out IPs that are already in use.
You can move the file with "dhcp database Flash:dhcp.txt"
01-11-2013 06:12 AM
I am unsure where to look for where the database is stored, but I do know our other DHCP Pool has no problem.
01-11-2013 06:18 AM
From your output, you definitely have a bunch of conflicts. This strongly supports my theory of what your issue is. I would recommend moving the DHCP file to persistent memory and clearing out the conflicts that have been created with the "clear ip dhcp conflict * " command repeatedly until you stop seeing them.
01-11-2013 06:38 AM
Can you explain why it is happening to one DHCP Pool and not the other?
01-11-2013 07:02 AM
You are getting conflicts in at least two different subnets on this device I see them in both the 192.168.12.X and the 192.168.4.X. The impact these conflicts will have on the subnets will vary greatly depending on: the size of the subnets, the number of clients attempting to obtain addresses, and the order in witch the clients attempt to renew their addresses.
Basically the problem goes as follows:
-The switch hands out addresses and gives: address 1 to client a, address 2 to client b, address 3 to client c, ect...
-The switch reloads and loeses its database, it now has no idea that it has given any address to any client.
-Client d attempts to obtain an IP address, the switch says "lets give him address 1" but then it finds that address 1 is in use (by client a) and says "I will mark that as a bad address and not give that out" it then repeats this process until it finds an address that is not in use, marking everyone it can ping as bad along the way.
-Client a then attempts to renew its IP address, but the switch says " no you can't have that, it's a badd address" and gives it the first good address it has.
Depending on the utilization of the subnet, the switch can end up filling up a whole pool with bad addresses and not have any good ones to hand out. If the utilization is low in the subnet, it may have enough to give everyone a new address on top of the bad addresses and you would not know you have a problem.
01-14-2013 07:47 AM
This was from TAC
Thanks for your e-mail.
This problems seems to be related to DHCP conflict issue. Let me explain what it is:
Before allocating an ip address to the device requesting for it the router sends an echo request packet to check if the ip address is available. If the router receives ICMP Echo Reply message the address is in use. If the DHCP conflict logging is enabled (default), the router will log the conflict with a syslog message (not in a separate log file) and put the address on the list of conflicts. The addresses on that list (displayed with show ip dhcp conflict) are not used in the future (similar to the addresses configured with the ip dhcp excluded-addresses command). To reuse a conflicting address, we need to remove it from the list with the clear ip dhcp conflict address (or * for all addresses) command.
Cisco recommends disabling the conflict logging by executing the command 'no ip dhcp conflict logging' on the device. This does not affect the DHCP functionality in any way since the server will always ping before allocating an IP address. The only difference would be that if it receives a reply then it will not put that IP in the conflict table and will move on to the next available IP address in the pool and
allocate that if there is no conflict.
As we can see you already have a lot of IP addresses in conflict list:
I would highly recommend you to disable HDCP conflict logging using 'no ip dhcp conflict logging' command. This should fix the issue you are facing. Feel free to revert in case you have any questions for me.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide