Is it possible to revert an AP (2800) back from IOS-XE to AireOS ?

[Kris_Vangeel]

In our environment, we have 4x 8540, which are used for the bulk of our APs and 2x 9800-Ls, which are mainly used for external APs (typically working-from-home-APs) and for testing purposes.

I had a 2800 connected to the 9800 (17.6.2) and I wanted to revert it back to an AireOS controller, but for some reason this does not seem to work. Is it possible to switch an AP back from IOSXE to AireOS ?

I hardcoded the AireOS WLC while connected to the IOSXE WLC and I also did a factory reset on the AP (AP gets this WLC through DHCP option 43) , but the result remains the same. It just keeps looping through the same cycle shown below.

[*03/22/2022 14:46:46.3055] CAPWAP State: Discovery
[*03/22/2022 14:46:46.3062] Got WLC address 10.119.48.101 from DHCP.
[*03/22/2022 14:46:46.3063] IP DNS query for CISCO-CAPWAP-CONTROLLER.infra.kuleuven.be
[*03/22/2022 14:46:46.3388] Discovery Request sent to 10.119.48.101, discovery type DHCP(2)
[*03/22/2022 14:46:46.3470] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*03/22/2022 14:46:46.3472] Discovery Response from 10.119.48.101
[*03/22/2022 14:46:55.6721] Started wait dtls timer (60 sec)
[*03/22/2022 14:46:55.6725]
[*03/22/2022 14:46:55.6725] CAPWAP State: DTLS Setup
[*03/22/2022 14:46:55.7099] dtls_verify_server_cert: Controller certificate verification successful
[*03/22/2022 14:46:56.0233]
[*03/22/2022 14:46:56.0233] CAPWAP State: Join
[*03/22/2022 14:46:56.0266] Sending Join request to 10.119.48.101 through port 5264
[*03/22/2022 14:47:52.7106]
[*03/22/2022 14:47:52.7106] CAPWAP State: DTLS Teardown
[*03/22/2022 14:47:52.8008] status 'upgrade.sh: Script called with args:[CANCEL]'
[*03/22/2022 14:47:52.8588] do CANCEL, part1 is active part
[*03/22/2022 14:47:52.8736] status 'upgrade.sh: Cleanup tmp files ...'
[*03/22/2022 14:47:52.9073] Dropping dtls packet since session is not established. Peer 10.119.48.101-5246, Local 10.123.161.90-5264, conn (nil)
[*03/22/2022 14:47:52.9074] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*03/22/2022 14:47:52.9075] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*03/22/2022 14:48:07.4617]
[*03/22/2022 14:48:07.4617] CAPWAP State: Discovery
[*03/22/2022 14:48:07.4624] Got WLC address 10.119.48.101 from DHCP.
[*03/22/2022 14:48:07.4624] IP DNS query for CISCO-CAPWAP-CONTROLLER.infra.kuleuven.be
[*03/22/2022 14:48:07.4768] Discovery Request sent to 10.119.48.101, discovery type DHCP(2)
[*03/22/2022 14:48:07.4800] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*03/22/2022 14:48:07.4802] Discovery Response from 10.119.48.101

[marce1000]

         - What's in the controller logs when these AP's try to join ?

 M.

[Kris_Vangeel]

The AP was factory restored. I pointed it via CLI to an almost empty WLC (which is used as backup), just to keep the output of the debug somewhat under control.

From show ver:

cisco AIR-AP2802I-E-K9 ARMv7 Processor rev 1 (v7l) with 1028224/593948K bytes of memory.
Processor board ID FCW2323NK2W
AP Running Image : 17.6.2.43

From debug capwap errors & events, I am seeing the following:

*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 acDtlsPlumbControlPlaneKeys: lrad:10.123.161.90(5256) mwar:10.119.49.103(5246)

*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 Allocated index from main list, Index: 1354

*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 Reserved LCB index for 10.123.161.90:1354

*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 Using CipherSuite AES128-SHA
*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 DTLS keys for Control Plane are plumbed successfully for AP 10.123.161.90. Index 1355

*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 DTLS invalid result code (3)
*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 DTLS Session established server (10.119.49.103:5246), client (10.123.161.90:5256)
*spamApTask6: Mar 23 09:48:56.886: 70:df:2f:e6:d9:35 Starting wait join timer for AP: 10.123.161.90:5256

*spamApTask6: Mar 23 09:54:21.881: 70:df:2f:e6:d9:35 Allocated index from main list, Index: 3706

*spamApTask6: Mar 23 09:54:21.881: 70:df:2f:e6:d9:35 Reserved LCB index for 10.123.161.90:3706

*spamApTask6: Mar 23 09:54:21.881: 70:df:2f:e6:d9:35 Using CipherSuite AES128-SHA
*spamApTask6: Mar 23 09:54:21.881: 70:df:2f:e6:d9:35 DTLS keys for Control Plane are plumbed successfully for AP 10.123.161.90. Index 3707

*spamApTask6: Mar 23 09:54:21.881: 70:df:2f:e6:d9:35 DTLS Session established server (10.119.49.103:5246), client (10.123.161.90:5256)
*spamApTask6: Mar 23 09:54:21.881: 70:df:2f:e6:d9:35 Starting wait join timer for AP: 10.123.161.90:5256

[Flavio Miranda]

Hi

 Access Point 2800 is neither IOS-XE nor AIROS and it should work properly on both WLC.  The different is that 8500 software code will start with 8.x.x.x and 9800 with  17, depending on your setup. Factory Reset should fix it.  Are you sure it works? Did it change the software version after factory reset?

[Arshad Safrulla · Accepted solution]

Upgrade and Downgrade of Catalyst 9800 Controllers : Tips and Tricks - Cisco

An AP who joined a 17.6.1 or later WLC is not be able to join an AireOS WLC anymore unless it runs 8.10.162 or later, or 8.5.176.2 or later 8.5 code.

Best option is to downgrade manually. 

[Kris_Vangeel]

By coincidence, I have 2 of our main controllers running on 8.10.162.0 as test. I just pointed the AP to this one and it reverts back correctly. So this seems indeed to have fixed it:

[*03/23/2022 12:30:15.0389] CAPWAP State: Discovery
[*03/23/2022 12:30:15.0424] Discovery Request sent to 10.119.49.104, discovery type STATIC_CONFIG(1)
[*03/23/2022 12:30:15.0426] Got WLC address 10.119.48.101 from DHCP.
[*03/23/2022 12:30:15.0426] IP DNS query for CISCO-CAPWAP-CONTROLLER.infra.kuleuven.be
[*03/23/2022 12:30:15.0645] Discovery Request sent to 10.119.49.104, discovery type STATIC_CONFIG(1)
[*03/23/2022 12:30:15.0664] Discovery Request sent to 10.119.48.101, discovery type DHCP(2)
[*03/23/2022 12:30:15.0681] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*03/23/2022 12:30:15.0684]
[*03/23/2022 12:30:15.0684] CAPWAP State: Discovery
[*03/23/2022 12:30:15.0692] Discovery Response from 10.119.49.104
[*03/23/2022 12:30:15.0937] Discovery Response from 10.119.49.104
[*03/23/2022 12:30:15.0940] Discovery Response from 10.119.48.101
[*03/23/2022 12:30:19.7815] Start: RPC thread 1950960624 created.
[*03/23/2022 12:30:24.5394]
[*03/23/2022 12:30:24.5394] CAPWAP State: DTLS Setup
[*03/23/2022 12:30:24.9009]
[*03/23/2022 12:30:24.9009] CAPWAP State: Join
[*03/23/2022 12:30:24.9043] Sending Join request to 10.119.49.104 through port 5264
[*03/23/2022 12:30:24.9073] Join Response from 10.119.49.104
[*03/23/2022 12:30:24.9073] AC accepted join request with result code: 0
[*03/23/2022 12:30:24.9105] Received wlcType 0, timer 30
[*03/23/2022 12:30:25.0674]
[*03/23/2022 12:30:25.0674] CAPWAP State: Image Data
[*03/23/2022 12:30:25.0677] AP image version 8.10.162.0 backup 17.6.2.43, Controller 8.10.162.0

[denneht]

Kris,

I have the same exact issue.  3802s were on previous occupants IOX XE controller.  We moved in and APs will not join our controller.  Controller is frozen at an older release as there are a lot of other APs on this controller in different buildings.  I put a small 2504 in there to see if they would join and downgrade, however they did not, and I used 8.5.182.7 code.

I'm wondering if there is something else.  Did you have to set the date back to December 1st 2022 or something else in order to fix the issue?

I'm wondering what else is different in your setup than mine, as we are having the same problem.

[denneht]

Do we know if we downgrade the AP to a controller running  8.10.162 or later, or 8.5.176.2 or later 8.5 code, can we then jump from either one of those versions of code down even further to a production 8.5.171.0 controller?

Meaning, I inherited a ceiling full of 3802s that were on a 9800, but we just occupied the space and they left the APs for us.  Our WLC is a 5520 on 8.5.171.0.  Thinking that I might be able to throw a temporary switch and controller in the rack and patch the APs to it so they downgrade to 8.10.162.0.  Then patch them over to the network with the 5520 and hopefully they will now join that WLC and downgrade themselves to the version that is running.  For reasons out of my control, I cannot jump that 5520 to a later version.

[denneht]

Arshad,

Is there a document on how to "Best option is to downgrade manually"?  APs are on ceiling, and no console access is my scenario.

Previous occupant left their 3802s but took their controller.  Their APs won't join our 5520.

Thanks,

Tim

[Rich R]

Glad to hear that worked @Kris_Vangeel 

Thanks for that link @Arshad Safrulla hadn't seen that before!

This is actually documented in https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvx98176 which is mentioned in the AireOS release notes https://www.cisco.com/c/en/us/td/docs/wireless/controller/release/notes/crn810mr6.html#resolved-caveats and https://www.cisco.com/c/en/us/td/docs/wireless/controller/release/notes/crn810mr7.html#resolved-caveats 

It should have been in the 8.5.182.0 release notes but isn't!

[gnijs]

Hello,

is this also valid if the IOS XE controller runs only 16.12.4.31 ??

or can it then downgrade without problems to AirOS 8.5 ?

[Rich R]

That should work as per CSCvx98176 (workaround 3) but if you make sure you're running an up to date version of 8.5 then you won't have the problem at all. (you should be specific about what version you're using 8.5.x.x)