Problem DHCP intervlan

[Beaurr]

Hello,

on our network, we have been having problems with DHCP for a while. Some workstations, at start-up, cannot retrieve an IP address. By restarting the pc, or by doing several ipconf / release, ipconfig / renew, it ends up working.

Our pc are on distribution cisco switches. these switches are connected to the core switch. The DHCP server is on a an server switch.

The dhcp is provided by 2 windows dhcp server (2016).

We have many VLANs. inter vlan routing is active on our core switch. In the vlan configuration, the ip helper address are configured like this ( for all the vlan):

interface Vlan220

 description XXX

 ip address 10.XXX.xxx.xxx 255.255.255.0

 ip helper-address  10.39.X.XX1

 ip helper-address 10.39.X.XX2

 ip access-group XXXX in

The server and the workstations are in different vlans

We tried by putting a virtual machine in the same vlan as the server. We have run a script on this machine which does an ipconfig relesae / renew every 5 minutes. And monitor with wireshark. No problem found. The PC gets its IP address every time.

We have monitored a client in a different vlan. And there, sometimes it works, other times, it does not. We see the client sending a DHCP DISCOVER, but we don't see it coming while monitoring the server. The client tries several times. After a while, it sometimes passes.

It seems that some client requests via the DSCOVER DHCP are lost at the intervlan routing level on the core network

[pieterh]

- you mention inter-vlan routing done by the core ? the distribution switches do not route ?

- the DHCP servers are configured as redundant-pair?
   if so, active-stand-by or load-balancing ?
   you can start with enabling dhcp-service one of the DHCP-servers and monitor the behavior with each DHCP-server separately

- do you capture the DHCP Discover at the client PC ? or at the switch ?

if at the client PC, check if the switch port is configured as access port and with spanning-tree portfast

if this is not configured, negotiation of access or trunk / speed + duplex / spanning-tree may take too long for the client to succeed

- do you use any port-authentication ? (DOT1x?)

[Beaurr]

Hello,

---

@pieterh wrote:

- you mention inter-vlan routing done by the core ? the distribution switches do not route ?

- the DHCP servers are configured as redundant-pair?
   if so, active-stand-by or load-balancing ?
   you can start with enabling dhcp-service one of the DHCP-servers and monitor the behavior with each DHCP-server separately

- do you capture the DHCP Discover at the client PC ? or at the switch ?

if at the client PC, check if the switch port is configured as access port and with spanning-tree portfast

if this is not configured, negotiation of access or trunk / speed + duplex / spanning-tree may take too long for the client to succeed

- do you use any port-authentication ? (DOT1x?)

---

• No, only the core switch (who is VTP server).
• the DHCP server was configured lin oad-balancing. But there is only one now.
• We have made captures on the client PC, on the DHCP Server, and port mirroring on the core switch to observe all the exchanges between the server vlan and the test vlan data
  
  
• On the switches, the client ports are in access mode (switchport mode access, switchport access vlan XXX), and with spanning-tree portfast active.
• We don't use port-authentication (DOT1x) but we filter the mac addresses on the DHCP server
• This configuration has not been changed lately. and until a few months ago everything was working. And then gradually more and more PCs started to have this problem.

[Giuseppe Larosa]

Hello @Beaurr ,

just in addition to the notes by @pieterh 

your input ACL includes permit lines for traffic destined to bootps ports with 0.0.0.0 source and 255.255.255.255 destination address ?

>> ip access-group XXXX in

Hope to help

Giuseppe

[Beaurr]

---

@Giuseppe Larosa wrote:

Hello @Beaurr ,

just in addition to the notes by @pieterh 

your input ACL includes permit lines for traffic destined to bootps ports with 0.0.0.0 source and 255.255.255.255 destination address ?

>> ip access-group XXXX in

Hope to help

Giuseppe

---

Hello,

all acl start with permit udp any eq bootpc any

                         permit udp any eq bootps any

[Giuseppe Larosa]

Helllo @Beaurr ,

the server side UDP port should stay at the end meaning destination port

>>permit udp any eq bootpc any

>>permit udp any eq bootps any

I would write them as:

permit udp any any eq bootpc

permit udp any any eq bootps

There is difference between the two

Hope to help

Giuseppe

[paul driver]

Hello

---

@Beaurr wrote:

Some workstations, at start-up, cannot retrieve an IP address. By restarting the pc, or by doing several ipconf / release, ipconfig / renew, it ends up working.

the DHCP server was configured load-balancing. But there is only one now

---

Have you made sure your dhcp scope isn't being exhausted, it is large enough to accommodate your client clans now there is only one server?

[Beaurr]

hello paul driver,

yes, our dhcp scopes are big enough, around 200 IP available per scope, while the number of users per scope is around 90-100.
Yes, when we noticed the DHCP problems, we decided to shut down one of the 2 DHCP servers, and we modified the configuration of our TEST VLAN to put only one helper-ip ip. But that didn't change anything. What is strange is that the initial setup had been in production for 1 year, it was working. The problems appeared for 2 or 3 months and are becoming more and more important.

[paul driver]

Hello

Then what has changed pertaining to the network, has any network device been introduced inpath between dhcp server and clients?

Does the dhcpserver log report any errors

Have the clients received any updates or patching since this error was noticed?

Does this occur on both wired/wifi clients or just one of them?

Do you have dhcp snooping enabled on the network?

[Beaurr]

Hello Giuseppe Larosa

The server vlan ( VLAN 2) does not have an ACL, only the workstation VLANS

interface vlan 2

description - VLAN 2 - server

ip address 10.XX.XX.254 255.255.255.0

ip helper-adress 10.XX.XX.XX

ip helper-adress 10.XX.XX.XX

I'm not sure to understand the difference between the two?

permit udp any eq bootpc any

permit udp any eq bootps any

I would write them as:

permit udp any any eq bootpc

permit udp any any eq bootps

[Beaurr]

Hello,

we have added a switch to the core stack ( in trunk mode). 

On the core switch, wa have deleted the ip helper adress of all the vlan. 

On the new  switch connected as a trunk on your stack, we created manuelly ( no VTP like on the core switch), with an ip on the same subnet as each VLAN, an we have added the ip helper adress here. 

So the new switch is only used for DHCP request to ou server.

This solved our DHCP problem.

But that doesn't explain the problem?

The core is a stack of 4 switch ( 2 WS-C3850-24S and 2  WS-C3850-24t)  IOS : 16.12.05b.

the new added switch ( connected in trunk to the stack) is an old 3750 IOS : 12.2