APs Cannot join the Controller

[Allan001]

Hi all,

My APs can't join the C9800-CL ver 17.6.5. The controller is run on a HyperV server.

1) Certificate is configured, and I see it with: sh wireless management trustpoint. 

2) I configured the DHCP scope in the controller 

3) Checking the DHCP binding - it shows that 2 APs have been allocated addresses, and I can ping those 2 APs from the controller. However, the APs are not joining the controller. 

4) Wireless management is an SVI in controller under VLAN 10

5) Gig 2 is configured as a trunk, and VLAN is allowed on Gig2

6) The port-facing APs are configured as access with VLAN 10

Additionally, I haven't registered the controller with a license yet, so I'm uncertain whether it's necessary to do so for the APs to join. The AP model is C9115xai. In the logs, I cannot see any CAPWAP log. And to register the AP license to the controller using a smart account. I noticed some IP addresses need to be configured for the controller to communicate with CCSM - where would I get those IP addresses?

Any help is appreciated.

Thank you,

[Allan001]

Hi Marce,

Thank you again. I will go through these docs. 

[Rich R · Accepted solution]

Just one more thing to check for: if the APs are loaded with a very old version of AireOS then they may not be able to join the 17.6.5 WLC (due to security related changes in CAPWAP protocol).

In which case you'll need to manually update the AP code before they'll be able to join.

[Allan001]

Hi all,

I apologise for taking too long to share the AP console output. The transformer was only fixed later last night. I got the AP output early but was busy with other stuff. At the bottom of the output, I can ping the controller .2 from AP .111. The AP has obtained a correct IP Add from the DHCP configured in the controller. 

Thank you for your patience

[Rich R · Accepted solution]

Your AP is running as an EWC WLC and at least one other AP is running as EWC.  You can convert it to capwap mode by a number of methods:

For EWC: https://www.cisco.com/c/en/us/support/docs/wireless/embedded-wireless-controller-on-catalyst-access-points/215303-embedded-wireless-controller-conversion.html#anc18

Accessing AP Console From EWC (former apciscoshell)

When console cable is plugged into the AP running EWC image, an EWC prompt will be shown by default. If, for any reason, access to the underlying AP shell is required, it can be done using:

EWC#wireless ewc-ap ap shell username admin

admin@192.168.129.1's password: Cisco123

To exit back to EWC shell, use:

AP1>logout

Connection to 192.168.129.1 closed.

EWC#

Note: This command is equivalent to apciscoshell that was previously available in Mobility Express controllers.

Converting EWC Back To Lightweight CAPWAP Mode

If AP running in EWC mode needs to be converted back to lightweight capwap mode, it can be done via:

AP1#ap-type capwap

AP is the Master AP, system will need a reboot when ap type is changed to CAPWAP

. Do you want to proceed? (y/N) y

Important: This command will perform a complete factory reset of both AP and EWC partition. Make sure to backup existing EWC configuration before conversion.

https://www.cisco.com/c/en/us/products/collateral/wireless/embedded-wireless-controller-catalyst-access-points/white-paper-c11-743398.html#Conversion

Converting EWCs to CAPWAP using option 43

DHCP option 43 is a vendor-specific option and is used for providing WLC IP addresses to the access point. Using option 43 with a specific subtype option, you can have the EWC convert to CAPWAP and join a WLC appliance or virtual controller. After the AP receives DHCP option 43 and subtype 0xF2 at bootup, the AP type will be converted to CAPWAP, and the AP will follow the regular joining process.

The DHCP configuration on the switch is shown below.

Switch(dhcp-config)#option 43 hex F2056464645801

Personally I recommend removing the IOS-XE EWC code from them completely because they have a nasty habit of unexpectedly re-activating EWC when they can't reach the WLC.   Also note that running EWC and WLC together at the same time is NOT SUPPORTED.  To completely remove the EWC re-flash them using the process at:
https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9120axi-access-point/217537-repairing-c9120-c9115-access-points-from.html

[Allan001]

Hi Rich,

Thank you so much for your prompt response and detailed info. I am better off removing the IOS-XE EWC code so we won't run into this problem again when the AP fails to reach the controller. I will read the links you shared before implementing them and let you know when I have the results. I will start the implementation tomorrow morning and see how it goes. 

The first option is time-consuming since I have to go through this process on every AP. Option 43 might be quick, as I will only have to remotely configure the option on the switch. 

Once again, thank you so much, and I appreciate your contributions. 

[Allan001]

Good day all

Apologies for not coming back with feedback on time. I only started working on that controller yesterday, and thank you so much for all the answers and solutions you gave. I still haven't got it working, though.

@Rich R, I ended up going with the option 43 route. At one of the sites, I have two APs in the same subnet with a C9800-CL wireless management interface. One of these APs joined the controller with "option 43 hex F2056464645801." However, the other AP did not join the controller, and I suspect it might be the faulty UTP cable. I have a tech driving to the site to test the cable, or swap around the APs should give me a quick answer.

Other sites' APs are not in the same subnet as the controller Management interface. "Option 43 hex f2056464645801" does not seem to work, and APs won't join the controller. So, I tried "option 43 hex F1040ad11902 converted from 10.209.25.2", and 3 APs joined the controller. 10.209.25.2 is the wireless mgmt int. This site has more than 15 APs, and after configuring option 43 hex F1040ad11902 with no APs joining the controller, I asked the on-site tech to walk around and reboot every AP. APs joined as he was rebooting one by one, but they were "OPERATIONAL DOWN". So the APS stayed in that state for about 45 minutes to an hour, and all of a sudden, all APs disappeared, and I was left with one AP, the one in the same subnet with wireless management. So I wonder what went wrong and why they joined with an operational downstate in the first place.

Furthermore, I read the URL that you posted that explains EWC. I have downloaded the Cisco Catalyst application to my phone. If I keep struggling with this setup, I might try the EWC.

Allan001

[Allan001]

Hi @Rich R 

I am grateful for the valuable suggestions and URLs you provided above. They greatly assisted me in finishing the project. All of your recommendations were very much appreciated. After being put on hold for a significant period due to ongoing renovations on the site, we completed the project yesterday. I extend my sincere appreciation to all those who contributed solutions @balaji.bandi @Scott Fella @marce1000

The EWC mode caused some issues for me, but thanks to your help, we were able to get it to function properly.

Thank you. 

[Rich R]

Glad to hear it helped - thanks for the feedback.

[Allan001]

Hi all,

I am running 17.6.5; however, the C9800-CL is throwing a whole lot of errors, including the CAPWAP DTLS session closed for AP cause: DTLS server session shutdown. I saw some recommendations saying 17.6 should be stable. Kindly see attached.

Allan001

[marce1000 · Accepted solution]

  @Allan001 - Did you run the WirelessAnalyze procedure which was requested earlier : (CLI) show tech wireless , feed output into :
                                          https://cway.cisco.com/wireless-config-analyzer/

  =?

 M.

[Rich R]

Your attached screenshot of log shows a traceback.  Tracebacks are caused by software bugs - open a TAC case and let TAC decode and diagnose that fault.

> I haven't registered the controller with a license yet, so I'm uncertain whether it's necessary to do so for the APs to join
Not required - up to 50 APs can join without registration.

> I noticed some IP addresses need to be configured for the controller to communicate with CCSM
No IP addresses need to be configured for smart licensing.  If you need to configure a firewall then you might need to allow specific IP addresses but best to use domain names: https://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/smart-software-manager-satellite/white-paper-c11-743812.html#CiscoSmartLicensingproducts

Smart Transport

●      HTTPS(443): smartreceiver.cisco.com

Yes 17.6 is very stable but 9800-CL requires very specific setup in hypervisor.  IF you have not followed installation guide to the letter that can cause problems.  If you have and still seeing problems then open a TAC case.
https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/9800-cloud/installation/b-c9800-cl-install-guide/b-c9800-cl-install-guide_chapter_01011.html
Pay close attention to supported version of HyperV and Windows Server:
https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-6/release-notes/rn-17-6-9800.html#id_114714