Problem with IP Helper

[Chris Golden]

Hi,

We have an interface vlan 1024 with an IP helper address for the 1024 network:

interface Vlan1024
 ip address 10.10.24.1 255.255.255.0
 ip helper-address 10.10.34.17
end

All clients (Windows workstations) are connected to switches on the network via vlan1024 all ports are access ports that each machine is connected to.

We have a firewalled DMZ network 10.32.14.0/24 which has our servers on it.

We have 2 dhcp servers running they are:

dhcp linux server - 10.10.12.85 this answers requests for the vlan1014 network but the server also sits on the vlan1012 network

dhcp windows server - 10.10.34.17 this answers requests for the vlan1024 and vlan1034 network.

The main L3 switch we have is a 2 stack 3850.  IP routing is enabled and all other normal things like VTP etc.

So up until today this is when things started to go wrong, without, I must add any changes to the network or servers.  All clients that send a dhcp request are being answered by the linux server (10.10.12.85) even though the helper address is pointing to the windows server (10.10.34.17).  Then the clients receive and IP on the vla 1014 (10.10.14.0/24) network which shouldnt be possible because all clients are on the vlan1024 network (10.10.24.0/24).

I cannot find the problem with all the switches we have nor the servers.

Does anyone have any ideas?

[Peter Paluch]

Hi,

However improbable, this behavior would be explained most easily by admitting that for some reason, VLANs 1024 and 1014 started leaking into each other. When a DHCP server receives a relayed DHCP DISCOVER or REQUEST message, it associates this request with the proper pool based on the giaddr field in the message body. This field, somewhat confusingly named "gateway IP address", is filled in by the DHCP Relay Agent using the address of its interface that intercepted the original client's broadcasted message. The fact that the clients are now served by 10.10.12.85 and that they are actually assigned an address from the pool 10.10.14.0/24 suggests that the giaddr in the relayed DHCP messages must itself contain an addres from the 10.10.14.0/24 network, otherwise it would not be associated with the proper pool on the DHCP server.

This should also be confirmed on the Linux DHCP server by sniffing for the relayed DHCP messages (tcpdump, Wireshark) and observing their contents. I am fairly sure that the giaddr field will carry the address 10.10.14.x (most probably 10.10.14.1 if that is the default gateway and DHCP Relay Agent in VLAN 1014).

Proving that the VLANs 1024 and 1014 leak into each other should not be that difficult - using a normal PC in VLAN 1024, configure a static IP address from VLAN 1014 and try pinging the default gateway in the VLAN 1014. If it responds, make sure that the response comes back in an Ethernet frame whose source MAC address is the MAC address of the default router's interface in VLAN 1014. As MAC addresses are visible only within their own broadcast domain, seeing the MAC address of the default gateway in VLAN 1014 from a station in VLAN 1024 suggests that the leaking truly occurs. Another way of checking this is looking into the ARP cache of the PC - you must see the MAC address of the default gateway in VLAN 1014 and it must be different from the MAC address of the default gateway in VLAN 1024.

If you manage to prove that the leaking occurs via these pings then you should now be actually able to go switch-by-switch, checking their MAC address tables and looking for the path toward the MAC address of the default gateway in VLAN 1014 as recorded by ports in VLAN 1024.

I am not going to hypothesize what could have caused this leaking but sometimes, users (and even admins) are incredibly inventive.

Best regards,
Peter