Re: 2820i AP, 5508 WLCs, DHCP Relay - DHCP NOT working - Page 2

Sean Haynes · ‎02-21-2018

Morning

We have 2 Cisco 5508 WLCs, we have around 50 1142N APs across the campus, all of which use DHCP and PoE without any issues.

The WLCs are configured as DHCP relay agents which works perfectly well for both the APs and Clients.

We had a need for an additional AP, so we purchased a 2820i, ran a new cable which terminated into a Cisco 3750 switch which is already supporting several of the said 1142n APs.

The 3750 switch does not provided enough power for the newer AP so I also purchased an inline power injector - which works fine.

I have never had to pre-stage an AP before, so as before the new AP was installed and powered on. I was expecting it to mimic what every other AP has done for the last several years - get a DHCP address and join the controller - NOPE!!!

So in summary:

Yes, all other APs joined to that switch are able to get a DHCP address, all other APs tested on the very same port of that same switch are able to get a DHCP address - just not this AP.

If I console into the AP and debug DHCP events I can see that DHCP requests are indeed being sent, but not responded to:

[02/20/2018 23:26:26.7100] wired0 emac 2: link up
[02/20/2018 23:26:26.7600] wired0: link up
[02/20/2018 23:26:26.8100] wired0: started
[*02/20/2018 23:26:26.8635] aptrace_register_sysproc_fn: duplicate registeration for 'wired'
[*02/20/2018 23:26:27.6503] chatter: DHCP-EVT: Sending DHCP discover packet length 346 bytes
[*02/20/2018 23:26:27.6503] chatter: DHCP-PAK: Sent DHCP_DISCOVER pak:
[*02/20/2018 23:26:27.6503] chatter: sent pkt source: 0.0.0.0, destination: 255.255.255.255
[*02/20/2018 23:26:27.6503] chatter: UDP sport: 68, dport: 67, length: 312
[*02/20/2018 23:26:27.6503] chatter: DHCP op: 1, htype: 1, hlen: 6, hops: 0
[*02/20/2018 23:26:27.6503] chatter: xid: 7cd9e03e, secs: 0, flags: 0
[*02/20/2018 23:26:27.6503] chatter: client: 0.0.0.0, your: 0.0.0.0
[*02/20/2018 23:26:27.6504] chatter: srvr: 0.0.0.0, gw: 0.0.0.0
[*02/20/2018 23:26:30.8648] Waiting for uplink IPv4/IPv6 configuration
[*02/20/2018 23:26:35.8657] Waiting for uplink IPv4/IPv6 configuration
[*02/20/2018 23:26:40.8666] Waiting for uplink IPv4/IPv6 configuration
[*02/20/2018 23:26:41.8667] Resetting wired0, if[02/20/2018 23:26:41.9000] wired0: stopped
config down up

If from console I do a 'sho arp', it does just that and returns dozens of IP addresses from across the wireless network, so broadcasts are working, it just can't seem to get it's own IP address via a DHCP broadcast.

I have run 'Wireshark' on both DHCP servers and can see no requests from the AP's MAC address, though I can see other requests from devices on the wireless network being properly relayed through the controllers to the DHCP servers. So it can't be a relay problem.

If from console I enter a static address, subnet mask and the IP of the default gateway on the AP - it will IMMEADIATELY find both controllers and go through the automated process of joining with out issue. Clients are then able to use that AP, again without issue.

[*02/21/2018 10:33:53.4489] AP IPv4 Address updated from 0.0.0.0 to 172.20.255.230
[*02/21/2018 10:33:53.4561] dtls_init: Use MIC device cert
[*02/21/2018 10:33:53.4564] dtls_init: Use MIC device cert private key
[*02/21/2018 10:33:53.4565]
[*02/21/2018 10:33:53.4565] CAPWAP State: Init
[*02/21/2018 10:33:53.4569]
[*02/21/2018 10:33:53.4569] PNP is not required, Starting CAPWAP discovery
[*02/21/2018 10:33:53.4569]
[*02/21/2018 10:33:53.4571]
[*02/21/2018 10:33:53.4571] CAPWAP State: Discovery
[*02/21/2018 10:33:53.4617] Discovery Request sent to 172.20.255.201, discovery type STATIC_CONFIG(1)
[*02/21/2018 10:33:53.4630] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*02/21/2018 10:33:53.4630]
[*02/21/2018 10:33:53.4630] CAPWAP State: Discovery
[*02/21/2018 10:33:53.4635] Discovery Response from 172.20.255.201
[*02/21/2018 11:05:01.0004] Discovery Response from 172.20.255.202
[*02/21/2018 11:05:01.0001] Discovery Response from 172.20.255.201
[*02/21/2018 11:05:01.0000]
[*02/21/2018 11:05:01.0000] CAPWAP State: DTLS Setup
[*02/21/2018 11:05:01.1328] dtls_load_ca_certs: LSC Root Certificate not present
[*02/21/2018 11:05:01.1328]
[*02/21/2018 11:05:01.2326]
[*02/21/2018 11:05:01.2326] CAPWAP State: Join
[*02/21/2018 11:05:01.2339] Sending Join request to 172.20.255.201 through port 5264
[*02/21/2018 11:05:01.2373] Join Response from 172.20.255.201
[*02/21/2018 11:05:01.3126] HW CAPWAP tunnel is ADDED
[*02/21/2018 11:05:01.3258]
[*02/21/2018 11:05:01.3258] CAPWAP State: Image Data
[*02/21/2018 11:05:01.3562] do NO_UPGRADE, part2 is active part
[*02/21/2018 11:05:01.3609]
[*02/21/2018 11:05:01.3609] CAPWAP State: Configure
[*02/21/2018 11:05:01.3640] NO-ENC-PROVIDER for DOT11R_WLC_MAC_IP_PAYLOAD
[*02/21/2018 11:05:02.0198] Started Radio 0
[*02/21/2018 11:05:02.0389] DOT11_DRV[0]: set_channel Channel set to 1
[*02/21/2018 11:05:02.9272] DOT11_DRV[0]: set_channel Channel set to 1
[*02/21/2018 11:05:04.1417] Started Radio 1
[*02/21/2018 11:05:04.1647] DOT11_DRV[1]: set_channel Channel set to 36
[*02/21/2018 11:05:05.0840] DOT11_DRV[1]: set_channel Channel set to 36
[*02/21/2018 11:05:06.3229] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*02/21/2018 11:05:06.3230] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*02/21/2018 11:05:06.3231] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*02/21/2018 11:05:06.3231] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*02/21/2018 11:05:06.3232] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*02/21/2018 11:05:06.4150] DOT11_DRV[0]: set_channel Channel set to 1
[*02/21/2018 11:05:07.4264] DOT11_DRV[1]: set_channel Channel set to 36
[*02/21/2018 11:05:08.3442] DOT11_DRV[1]: set_channel Channel set to 36
[*02/21/2018 11:05:09.4703]
[*02/21/2018 11:05:09.4703] CAPWAP State: Run
[*02/21/2018 11:05:09.5009] CAPWAP HW tunnel params changed, UPDATING the existing
[*02/21/2018 11:05:09.5559] AP has joined controller Cisco5508_WLC_Primary

I have checked the VLAN config which is on the switches, made sure the IP Helper Addresses are correct, basically gone through everything I can think of - still no closer to figuring this out. It has to be said the Cisco literature is pretty base and of no use when troubleshooting this issue.

Anyone else had this problem?

Sean Haynes · ‎03-07-2018

yes I definitely get this is an AP / Code issue. Rather annoying given he cost of these devices!

Binish · ‎03-23-2018

Hi All,

Anyone seen this issue on code 8.3.133.0 .

-- Binish

Sean Haynes · ‎03-23-2018

..that's the code I'm running...

Rafael Jimenez · ‎05-29-2018

I have 8.2.166, CT5520, 80 x 3802E APs. Suddenly, the wifi clients, do not receive ip from the external windows server dhcp. WLC receive the dhcp discovery, not sure if the discovery go out of the wlc to the LAN, because, wireshark show only the discovery going between source address (the AP) to destination address (the WLC), I don't see a discovery unicast to the dhcp server.

Leo Laohoo · ‎05-29-2018

Check your network for asymmetric routing.
2800/3800 (and later) are very "delicate" (unlike the older models) and they don't like asymmetric routing (even DHCP requests).

Binish · ‎03-23-2018

Hi All,

I am running on code 8.3.133.0 and seeing the same error on few of the access point models AP3802i

"waiting for uplink IP address and reachable default gateway"

-- Binish

Rasika Nayanajith · ‎03-23-2018

Hi Binish,

I had few 3802I behaving same manner.

Here is what we found.

0. AP is stuck in "waiting for uplink IP address and reachable default gateway"

See attached console output from my 3802

1. From switchport side it says not connect
SW#sh int g0/7
GigabitEthernet0/7 is down, line protocol is down (notconnect)

2. Switch is not providing proper PoE to AP
SW#sh power inline | in 0/7
Gi0/7 auto on 15.4 Ieee PD 4 30.0

3. NoCDP neighbor

SW#sh cdp nei g0/7
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
D - Remote, C - CVTA, M - Two-port Mac Relay

Device ID Local Intrfce Holdtme Capability Platform Port ID

4. TDR diagnostic on port come up with certain pairs open

SW#test cable-diagnostics tdr interface g0/7
TDR test started on interface Gi0/7
A TDR test can take a few seconds to run on an interface
Use 'show cable-diagnostics tdr' to read the TDR results.

SW#show cable-diagnostics tdr interface g0/7
TDR test last run on: March 24 05:02:46

Interface Speed Local pair Pair length Remote pair Pair status
--------- ----- ---------- ------------------ ----------- --------------------
Gi0/7 auto Pair A N/A N/A Normal
Pair B 4 +/- 10 meters N/A Open
Pair C 4 +/- 10 meters N/A Open
Pair D 4 +/- 10 meters N/A Open

We thought it was an issue (PoE port issue) of these APs, Trying to confirm with TAC & get these AP replaced. Due to no proper PoE/CDP negotiation it looks like a physical/link layer issue. Hence no further progress on AP get IP or CAPWAP discovery.

Please check your issue showing similar symptoms

HTH

Rasika

*** Pls rate all useful responses ****

Sean Haynes · ‎03-23-2018

Our old 3750's don't provide sufficient power for these APs - so we have the inline PSU, so for us not a power issue.

Rasika Nayanajith · ‎03-25-2018

Hi Sean,

Have a look below thread & see the recovery procedure given. If you having similar issue, you may need to do that for the affected APs

https://supportforums.cisco.com/t5/getting-started-with-wireless/ap3802i-power-on-by-air-pwr-50-but-the-wired-which-connect-to-a/m-p/3354295

HTH

Rasika

Andy.Dixon · ‎04-10-2018

Hi all,

We've very recently updated from 8.1.102.0 to 8.3.140.0 on a 5520 WLC. We now have 7 AIR-AP2802I-E-K9 LWAPs that are failing to register. All are showing the same issues:

show power inline shows "IEEE PD". Interface shows as up/up with speed/duplex of 100mbps/full. No entry in MAC address table and no CDP neighbour displayed. Closed and opened interface but made no difference.

Prior to finding this posting I had begun troubleshooting on one affected AP2802I and found that the LWAP will pick up a DHCP lease and correctly register when directly connected into the 2960X providing PoE and access to the LAN but will not pickup a DHCP lease when connected to it's wall outlet some 62m away from the switch. I had thought that the structured cabling was at fault but a laptop will pickup a DHCP lease when connected to the outlet.

There are other 2802I LWAPs connected to the 2960X that are working without issue.

The only difference I could find was that the 2802I was connecting at 1Gbps at the switch but both the LWAP and laptop connected at 100Mpbs at the outlet. All other 2802Is on the switch are connected at 1Gbps. I suspect there is a cabling issue but the laptop picked up a DHCP lease so there is connectivity and the datasheet for the 2802 (https://www.cisco.com/c/en/us/products/collateral/wireless/aironet-2800-series-access-points/datasheet-c78-736497.html) states that AP is fitted with "2x100/1000BASE-T autosensing (RJ-45)" so should negotiate down to 100Mbps. I've also tried hardcoding the speed on the switchport without success.

I SPANned the non registering 2802I's switchport on the 2960X and could not see any DHCP messages from the LWAP when it was connected to the wall outlet. When consoled into the same LWAP I got the familiar looping "Waiting for uplink IPv4/IPv6 configuration" message along with the wired0 interface being reset.

I've attached the failing LWAP config for info.

Could these symptoms be bug CSCva34879 even though the affected APs are not 3800s?

I've since connected the 2802i to a different 2960X and verified that it picks up a DHCP lease and registers on the WLC. I then forced the connected switchport to 100mbps and closed then opened the port. The LWAP got the familiar looping "Waiting for uplink IPv4/IPv6 configuration" message along with the wired0 interface being reset. The interface was showing as up/up with full duplex. It appears that the LWAP doesn't like 100mbps speed.

Thanks,

Andy

Leo Laohoo · ‎04-10-2018

@Andy.Dixon wrote:

Could these symptoms be bug CSCva34879 even though the affected APs are not 3800s?

CSCva34879 cannot be replicated by Cisco TAC. So it can't be fixed.

DominikKristof · ‎01-24-2020

I've had same problem as you (no MAC, no CDP, port showing not connected, PoE providing power, log from console telling waiting for IP). In my case it was 2802I AP and what fixed the problem was hardcode speed/duplex to 100M (before it was on auto). With 100M/full, WAP works without problem. Maybe it helps in some cases. Looks like power was provided to WAP but L2 link was down making any DHCP service unavailable.

Andy.Dixon · ‎04-10-2018

Hi all,

We've very recently updated from 8.1.102.0 to 8.3.140.0 on a 5520 WLC. We now have 7 AIR-AP2802I-E-K9 LWAPs that are failing to register. All are showing the same issues:

show power inline shows "IEEE PD". Interface shows as up/up with speed/duplex of 100mbps/full. No entry in MAC address table and no CDP neighbour displayed. Closed and opened interface but made no difference.

Prior to finding this posting I had begun troubleshooting on one affected AP2802I and found that the LWAP will pick up a DHCP lease and correctly register when directly connected into the 2960X providing PoE and access to the LAN but will not pickup a DHCP lease when connected to it's wall outlet some 62m away from the switch. I had thought that the structured cabling was at fault but a laptop will pickup a DHCP lease when connected to the outlet.

There are other 2802I LWAPs connected to the 2960X that are working without issue.

The only difference I could find was that the 2802I was connecting at 1Gbps at the switch but both the LWAP and laptop connected at 100Mpbs at the outlet. All other 2802Is on the switch are connected at 1Gbps. I suspect there is a cabling issue but the laptop picked up a DHCP lease so there is connectivity and the datasheet for the 2802 (https://www.cisco.com/c/en/us/products/collateral/wireless/aironet-2800-series-access-points/datasheet-c78-736497.html) states that AP is fitted with "2x100/1000BASE-T autosensing (RJ-45)" so should negotiate down to 100Mbps. I've also tried hardcoding the speed on the switchport without success.

I SPANned the non registering 2802I's switchport on the 2960X and could not see any DHCP messages from the LWAP when it was connected to the wall outlet. When consoled into the same LWAP I got the familiar looping "Waiting for uplink IPv4/IPv6 configuration" message along with the wired0 interface being reset.

I've attached the failing LWAP config for info.

Could these symptoms be bug CSCva34879 even though the affected APs are not 3800s?

Thanks,

Andy

Andy.Dixon · ‎04-11-2018

Sorry, somehow I double posted. Can this extra post be deleted by an admin?

kswiedikon · ‎06-19-2018

Any news on this issue? We are stuck with the same problem. All of our new 2802 APs are affected...