So this morning we upgraded one of our Cisco 9800-CL clusters from 17.3.5a to 17.3.6.
All C9115AXI-B APs (14) on a certain site did not join after the upgrade. The pre-download went fine, but when I finally got a console to one of the missing APs it was in a constant reload mode (u-boot). I tried to force the AP to boot from the other partition
- setenv BOOT <other-partition>
It then boots up on the previous version (17.3.5a). Finds the WLC, figures out it got the wrong version, and reboots to load the new version (from the other partition (17.3.6)) - and we are back to endless reloads in u-boot.
I checked field notice: FN - 72278 to see if my APs were affected by this bad batch, but they were not. At least that is what the Serial Number Validation Tool tells me.
The strange thing is that other C9115AXI-B APs, at a different site (but at the same WLC), works just fine with the new version.
I have tried the full factory reset, and also have the AP join a different WLC in our network, but to no avail.
I'm thinking that I might have to downgrade the WLC back to 17.3.5a to see if I can "unbrick" the APs. Or, maybe follow the repair guide - even though my APs are not affected.
I know the 17.3.6 version is fairly new, but I'm wondering if anybody else have seen this? And if you have any good suggestion on how to remedy the situation?
I just opened it and pushed it out to our site rep as well to get things rolling quickly. I was also able to replicate this behavior with a different WLC-9800 and a different 9120 and also a 2702 (To a lesser extent, but the latency is still there). The behavior was exactly the same, and rolling back a version fixed it.
The reason we are worried is we have the new 9120 v7s that require this software upgrade to deploy (We have about 100 of these in and cannot deploy them now. They have different hardware than previous versions of 9120s and will not join 17.3.5b or prior. We also have about 5K 2702s deployed on my network and are waiting for the 9136s/9166s to get here when we plan to roll out 17.9.2 site wide, but need to orphan our 5K 2702s on a pair of WLC 9800 80s running 17.3.6 during the process.
So between bad code, new hardware requiring it, and delayed shipment of new Wi-Fi 6E APs, Cisco is really doing a number on our scheduled upgrade.