Hi guys,
I have the 2x WS-C3560X-48P running 12.2(55)SE5, RELEASE SOFTWARE (fc1)
Both are configured with VRRP:
3560X-cr01#show standby brief
P indicates configured to preempt.
|
Interface Grp Pri P State Active Standby Virtual IP
Vl10 10 105 P Active local 172.17.224.3 172.17.224.1
Vl60 60 105 P Active local 172.17.226.3 172.17.226.1
Now I did all tests and troubleshoot on the current active: cr01 which holds the following DHCP configuration (standby switch currently has no DHPC config):
show run i | ip dhcp
ip dhcp excluded-address 172.17.224.0 172.17.224.10
ip dhcp excluded-address 172.17.224.240 172.17.224.250
ip dhcp excluded-address 172.17.226.0 172.17.226.10
ip dhcp pool CORPORATE224_DATA
ip dhcp pool CORPORATE226_DATA
Trouble started when I tried migrating DHCP service from the switch to one of our HQ AD servers.
In order to avoid full site outage I did the following:
1. One day in advance I switched default lease time:
ip dhcp pool CORPORATE226_DATA
lease 0 2
2. Next day I have configured AD server with the pool options required, then went on switch
no ip dhcp pool CORPORATE226_DATA
interface vlan 60
ip helper-address 172.17.120.71
3. Went on a workstation, shut/no shut the port on switch and then reviewed the ipconfig /all output
Trick was that it was showing the switch as DHCP server:
DHCP Server . . . . . . . . . . . : 172.17.226.2
Still on my AD server I could see some leases including my test workstation. About 2h later entire segment (vlan 60) had no connectivity as no device connected there was receiving any DHCP IP lease.
As an workaround I had to copy/paste and enable old DHCP pool on the switch as it seemed Microsoft AD setup (Windows 2008) was not working.
After adding everyone back online I redid steps 2&3 and ran a firewall sniffer capture.
Put that capture on Wireshark and here's what I could see (on the attachments).
Basically capture shows DHCP Discover and DHCP Offer but NO DHCP Request or DHCP Ack. Since these 4 messages are using same sniffing ports I assume the latter two messages never reach the firewall. By now you figured it out that DHCP server is reachable through an IPSEC tunnel. Still that server holds IP pools for other remote sites, which clears my concerns about DHCP servers' configuration. Another step I did was to use another DHCP server on another location (so another IPSEC tunnel):
ip helper-address 172.17.140.71
and to my surprise FW sniffer file looked the same: it traced just DHCP Discover and Offer messages for most of the time and some scarce DHCP Ack.
Wireshark file was captured using the following filter: tcpdump -nn host 172.17.120.71 port 67 or port 68 and was ran on each of the DHCP server location firewall. It didn't cross my mind at that time to run it also on the location where this issue sits. Also I checked on all FWs for any dropped packet and couldn't see any.
One more step I did I attempted to catch all DHCP traffic and PBR it through our MPLS line between site and HQ. Now I could not do a sniffer session here as I would have had to mirror one port there, but the result was the same which brings me to the point of suspecting as culprits either DHCP server (although I tried two different machines) or the switch itself.
Now please share your thoughts or things to check for this. My only idea sits on the thread's title: this happens because I did NOT completely disable DHCP service on the switch. You can also consul the live debug dhcp I ran on the switch.
Thanks in advance,
Florin.
L.E. here's SDM config if it matters
3560X-cr01#show sdm prefer
The current template is "desktop routing" template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1024 VLANs.
number of unicast mac addresses: 3K
number of IPv4 IGMP groups + multicast routes: 1K
number of IPv4 unicast routes: 11K
number of directly-connected IPv4 hosts: 3K
number of indirect IPv4 routes: 8K
number of IPv4 policy based routing aces: 0.5K
number of IPv4/MAC qos aces: 0.5K
number of IPv4/MAC security aces: 1K
