9130AX Access Point Ignoring DHCP Offer

[jpl861]

Hello. We are trying to troubleshoot why our 9130 access point ignoring the DHCP offer request of the DHCP server. We tried to capture the traffic on the port facing the AP and this is what we got. Whenever we try to use the hardware DHCP, the AP is just looping between discover and offer but the AP never sends a DHCP request. If we use the L3 switch as the DHCP, everything works fine. What we have noticed is that the hardware DHCP is not sending "client identifier" but it is sending with option 82. That's the major difference we can see between the packet capture of hardware DHCP and L3 switch DHCP traffic.

Do you guys have any idea what's causing the AP not to send a request based on the capture below?

Frame 2: 414 bytes on wire (3312 bits), 414 bytes captured (3312 bits) on interface 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Encapsulation type: Ethernet (1)
Arrival Time: Sep 16, 2021 12:51:11.634450000 utc
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1631796671.634450000 seconds
[Time delta from previous captured frame: 0.110362000 seconds]
[Time delta from previous displayed frame: 0.110362000 seconds]
[Time since reference or first frame: 0.110362000 seconds]
Frame Number: 2
Frame Length: 414 bytes (3312 bits)
Capture Length: 414 bytes (3312 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:ip:udp:bootp]
Ethernet II, Src: 40:b5:c1:03:77:3a (40:b5:c1:03:77:3a), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
Source: 40:b5:c1:03:77:3a (40:b5:c1:03:77:3a)
Address: 40:b5:c1:03:77:3a (40:b5:c1:03:77:3a)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 101
000. .... .... .... = Priority: Best Effort (default) (0)
...0 .... .... .... = CFI: Canonical (0)
.... 0000 0110 0101 = ID: 101
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.27.50.1, Dst: 255.255.255.255
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 396
Identification: 0x0277 (631)
Flags: 0x00
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x2b8c [validation disabled]
[Good: False]
[Bad: False]
Source: 10.27.50.1
Destination: 255.255.255.255
User Datagram Protocol, Src Port: 67 (67), Dst Port: 68 (68)
Source Port: 67
Destination Port: 68
Length: 376
Checksum: 0x1012 [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
[Stream index: 1]
Bootstrap Protocol (Offer)
Message type: Boot Reply (2)
Hardware type: Ethernet (0x01)
Hardware address length: 6
Hops: 1
Transaction ID: 0x557561d5
Seconds elapsed: 0
Bootp flags: 0x8000, Broadcast flag (Broadcast)
1... .... .... .... = Broadcast flag: Broadcast
.000 0000 0000 0000 = Reserved flags: 0x0000
Client IP address: 0.0.0.0
Your (client) IP address: 10.27.50.100
Next server IP address: 0.0.0.0
Relay agent IP address: 10.27.50.1
Client MAC address: cc:8f:75:aa:12:22 (cc:8f:75:aa:12:22)
Client hardware address padding: 00000000000000000000
Server host name not given
Boot file name not given
Magic cookie: DHCP
Option: (53) DHCP Message Type (Offer)
Length: 1
DHCP: Offer (2)
Option: (54) DHCP Server Identifier
Length: 4
DHCP Server Identifier: 22.65.22.111
Option: (51) IP Address Lease Time
Length: 4
IP Address Lease Time: (64800s) 18 hours
Option: (1) Subnet Mask
Length: 4
Subnet Mask: 255.255.255.0
Option: (15) Domain Name
Length: 10
Domain Name: local.com
Option: (3) Router
Length: 4
Router: 10.27.50.1
Option: (28) Broadcast Address
Length: 4
Broadcast Address: 10.27.50.255
Option: (6) Domain Name Server
Length: 12
Domain Name Server: 101.22.22.22
Domain Name Server: 101.22.22.23
Domain Name Server: 101.22.22.24
Option: (43) Vendor-Specific Information
Length: 6
Value: f1040a5da2c5
Option: (82) Agent Information Option
Length: 58
Option 82 Suboption: (1) Agent Circuit ID
Length: 27
Agent Circuit ID: 3049522d48312f312f322f34383537353434333638464634...
Option 82 Suboption: (2) Agent Remote ID
Length: 27
Agent Remote ID: 3049522d48312f312f322f34383537353434333638464634...
Option: (255) End
Option End: 255

Thanks in advance.

[patoberli]

Could you do me a favor and share a screenshot between the capture of the working packet and the not working one directly in wireshark? That would greatly increase the readability. 

Also, is the option 43 for both DHCP servers the same value?

[jpl861]

Thanks for the reply. I did not extract the capture to be opened in Wireshark but I just used the IOSXE output. I have it in Excel just so I can compare. The left side is the actual DHCP server that is not working and the IOS DHCP on the right. Both have identical option 43 in the dhcp offer message.

Based from our capture, the AP is sending DHCP discover message with client-id set but the response from the server doesn't have it. So I'm starting to doubt if that's causing the issue. However, I'm not sure who actually puts in the client-id in the offer message, if it's the DHCP server or the relay. The L3 switch is a 9500 SVL and there's a little bit of suspicion that when the offer message is received on the standby SVL, DHCP will be broken. We tried a laptop to plug in a laptop and even used a router to be a DHCP client and it worked. Only the 9130 AP is not cooperating.

Edit: to add, the dhcp offer that came from the server has option 82. So I do not know what's causing the issue. If it is the lack of option 61 or having option 82 in the offer. 

[Leo Laohoo]

Try reboot the AP.

[jpl861]

There are multiple APs and they are all rebooting non stop. 

[JPavonM]

Are the 9130's running IOS-XE version lower than 16.12.4a or 17.3.1?

I mention this because I suffered a similar issue (CSCvu57562) one year ago and the only way to solve the issue was to manually add the controller in 150 of them through AP's console. After joining the controller and downloading new code it was solved.

HTH
-Jesus
*** Please rate helpful responses ***

[Arshad Safrulla]

Do you have DHCP snooping enabled on the switches?

[netdipace]

Was this issue resolved?

[brettlarkins]

Currently experiencing the same issue with 9130AXI and 9124AXD AP's using a 9800-CL vWLC running 17.3.4.

5 of my 15 AP's do get DHCP just fine even with repeated reboots.  But the other 10 do not get DHCP IP's, even with repeated reboots.

[Leo Laohoo]

A rebuild, 17.3.5b, was just released a few days ago.  Upgrade to that first.

[goehringt]

Did you ever find a resolution to your issue?

[a-shaginjan]

Factory reset

[goehringt]

Your issue does not seem to be the same as mine and brettlarkins. I factory reset does not fix this issue. 

[a-shaginjan]

I had bunch of new APs with the same issue. Resolved by factory reset.