Intermittent connection issues to external DHCP from dynamic interface

[ricpe]

Hi,

We have a 5520 WLC with software version 8.3.150.0. Two dynamic interfaces is configured for guests, seperated because they are for customers in two different countries. We have intermittently been losing connection to our external DHCP-server from one of those interfaces. The connection can be lost ranging from hours to days, then suddenly it comes back up. We have been troubleshooting this, looking at routing, firewalls, VLAN trunks etc. but everything is in order. It's weird because it has happened six times since August this year. The Guest WLANs have central switching applied. Debugging from the WLC shows DHCP Discover packet from client which the WLC then forwards to the external DHCP Server, but doesnt receive a DHCP Offer back. I am aware that the WLC version is old, but for some reason only one dynamic interface loses connection. Any ideas here?

[MHM Cisco World]

I think the AP is flexmode

If yes

Config DHCP in same site as WLC and make dhcp centralize' i.e. dhcp request from wifi client pass to wlc even if AP is switching traffic.

MHM

[ricpe]

Sorry for not clarifying, but yes the APs are in flexmode with central switching. I'm not sure exactly what you mean, but you want me to have the DHCP-server in the same subnet as the WLC?

[MHM Cisco World]

https://www.cisco.com/c/en/us/support/docs/wireless-mobility/wireless-lan-wlan/211325-FlexConnect-Central-DHCP-Configuration-E.html

MHM

[ricpe]

Hi,

The guide assumes FlexConnect local-switching but I don't want to activate that for this customer. To get a general idea for how the WLC is setup:

Dynamic interface 1 which has no problems at all:

VLAN Identifier: 120

IP Address: 10.10.0.10

Netmask: 255.255.0.0

Gateway: 10.10.0.1

Primary DHCP Server: 10.110.30.40

Secondary DHCP Server: 10.110.30.41

Dynamic interface 2 which has intermittent problems with DHCP connectivity:

VLAN Identifier: 140

IP Address: 10.11.0.10

Netmask: 255.255.0.0

Gateway: 10.11.0.1

Primary DHCP Server: 10.110.30.40

Secondary DHCP Server: 10.110.30.41

So for several months I have connectivity to the external DHCP-servers from both dynamic interfaces. But then suddenly Dynamic interface 2 loses it's connectivity, sometimes for several days before it comes up again. Dynamic interface 1 never has any problems whatsoever. I can see that clients that connect to dynamic interface 2 sends DHCP Discover packets, and that the WLC is forwarding them to the DHCP, but the WLC doesnt receive any DHCP-offer from the DHCP-server. Now, I dont know if this is related to the WLC, but we have checked every configuration, routing, trunk-ports and they're accurate.

[MHM Cisco World]

Are wlc connect to Cat9000 series SW?

MHM

[ricpe]

I dont think so, but the problem occurs for every customer with a Guest SSID that is assigned to the dynamic interface. So it's not a specific site.

[MHM Cisco World]

You mention central switching' so are there any cat9k in wlc site?

MHM

[marce1000]

          >... but for some reason only one dynamic interface loses connection. 
 - It can still be related to bug(s) ; as per https://www.cisco.com/c/en/us/support/docs/wireless/wireless-lan-controller-software/200046-tac-recommended-aireos.html  , use (upgrade to) https://software.cisco.com/download/home/286284738/type/280926587/release/8.10.190.0
   
  - TAC support is diminishing and aireos based controllers should move forward to last/latest available software version

 M.

[ricpe]

Thanks, I will look into upgrading the WLC which I know is on very old software.

[Rich R]

It goes without saying that the software is very old and needs upgrading but if it's not a software problem then what might it be?

You say the WLC is forwarding the DHCP discover to the DHCP server but getting no reply.
Have you done a packet capture on the DHCP server to confirm the discover is received by the server?
And that it's a valid discover packet it receives from the WLC?
And if the server is sending a response?
If it is then why does that response not reach the WLC (routing maybe)?
If no response then why?  Check DHCP server logs.

[ricpe]

Hi, thanks for the reply.

We created a lab-host in the same subnet as the Dynamic interface (10.11.0.0/16) and sent a DHCP-request packet to the DHCP-server, with Wireshark enabled on lab-host. But we havent used packet capture from the DHCP-servers perspective. Now everything is working again, after clients didnt receive IP for 3 days.. So we have to do the packet capture from DHCP-server when it fails again, which we dont know when it will happen.  

We have checked the DHCP-server logs and haven't found anything.