Lobby ambassador can't log in

[JohnCKirk]

I recently upgraded a 9800 WLC from IOS XE 17.9.3 to 17.9.4a. After that, I've noticed that the lobby ambassador account can't log in. The login page displays normally, and I can enter the username/password, but then I get an error page:

If I log in as a WLC admin user, everything works fine.

I've checked the web server logs after a failed login attempt, and I noticed these lines:

2023/11/03 15:17:08.141254538 {nginx_R0-0}{2}: [stdout] [31224]: UUID: 0, ra: 0 (note):  [error] 31231#0: *13475 lua entry thread aborted: runtime error: /var/scripts/lua/features/indexhtml.lua:40: bad argument #1 to 'eval' (string expected, got nil)

2023/11/03 15:17:08.141254908 {nginx_R0-0}{2}: [stdout] [31224]: UUID: 0, ra: 0 (note): stack traceback:

2023/11/03 15:17:08.141255280 {nginx_R0-0}{2}: [stdout] [31224]: UUID: 0, ra: 0 (note): coroutine 0:

2023/11/03 15:17:08.141255660 {nginx_R0-0}{2}: [stdout] [31224]: UUID: 0, ra: 0 (note): [C]: in function 'eval'

2023/11/03 15:17:08.141256098 {nginx_R0-0}{2}: [stdout] [31224]: UUID: 0, ra: 0 (note): /var/scripts/lua/features/indexhtml.lua:40: in function 'getVersion'

However, I don't know whether it's a problem with the script itself or just with the arguments that are being passed to that script. Either way, it looks like a bug in the WLC software.

Searching the community, I've seen some similar issues with older versions:
WLC And Lobby Ambassador - Cisco Community
Solved: URL fail when access by Lobby-Admin - Cisco Community

Those were solved by an upgrade or by specifying a different URL. However, I can't find anything similar for this version. Is anyone else having the same problem?

[marce1000]

           - FYI : https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwh37783

 M.

[JohnCKirk]

Aha, thank you! I'd checked the release note for 17.9.x, but I didn't do a general bug search.

It's odd that the bug has a status of "Fixed" when the list of fixed releases is empty, but maybe they mean that it's been fixed in the dev build so it will go into the next release.

In my case, this is an HA pair. I forced a failover a few days ago (which involved rebooting the chassis that was active at the time), but that didn't help. I'll try another reboot, but it looks like I'm stuck with it for now.

[marce1000]

                 >...It's odd that the bug has a status of "Fixed" when the list of fixed releases is empty,
 Indeed , you may want to query TAC about it ; other options are trying with latest 17.12.x (adventure) ; or downloading virtual controller with 17.12.x (is downloadable without service contract) ; copy configuration and test on the virtual controller ; of course all of that would need time investment ,

 M.

[resantero]

What does the Workaround means?

The issue eventually disappear by itself. Rebooting the controller can brings back the controller.

The issue will reoccur when you reboot the Controller?  

[JohnCKirk]

M: "other options are trying with latest 17.12.x (adventure) ; or downloading virtual controller with 17.12.x"

Looking at the download page for the WLC, the only version for the 17.12.x branch is 17.12.1 (released in July). According to the bug search tool, the affected versions include 17.12.1, 17.12.1a, and 17.12.1w. So, that means that the the existing version and the future versions (limited release?) will have the same problem.

resantero: "What does the Workaround means?"

I agree that the workaround doesn't really make sense as written. I initially assumed that they meant "rebooting the controller can bring back the website" (i.e. it would work again after a reboot), but I did another reboot/failover last night which didn't help. I now think that you're correct, i.e. they're saying that it will eventually (?!) fix itself, then the problem will reoccur after a reboot and you'll have to wait again.

[Rich R]

> bug has a status of "Fixed" when the list of fixed releases is empty, but maybe they mean that it's been fixed in the dev build so it will go into the next release.
Maybe, but can also just mean the software developer is lazy and has not bothered to update the bug database.
I recently had a bug which TAC told me had already been fixed in 3 published releases but the bug did not list any (like this one).  They only updated the bug with the fixed releases after I made a fuss and insisted that the bug should be updated with correct info!

[JohnCKirk]

That's useful to know, thank you. However, I don't think it's the case here.

Looking at the 17.9.x branch, this bug was introduced in version 17.9.4 and it's still present in 17.9.4a (i.e. the latest version which is available for download). The only difference between those 2 versions is that 17.9.4a fixes the zero day vulnerabilities which were discovered recently:
Multiple Vulnerabilities in Cisco IOS XE Software Web UI Feature

My guess is that the lobby admin bug was fixed internally, ready to be included in 17.9.5. However, Cisco had to fork their source code to fix the zero day vuln (i.e. they had to rush out 17.9.4a as an emergency fix), and 17.9.5 probably won't be out until December.

[Rich R]

I've been hearing Jan/Feb for 17.9.5 (although these dates can always change).  I'm guessing it's had problems that required re-work + adding the vulnerability fix.

[resantero]

Thats too long to wait, they should not suggest it as Recommended Version if there are bugs present or detected on that version.

They should probably at-least make a working workaround if it will take that long to release the fixed version.

[Rich R]

Every release has bugs so it's a judgement of how many and how severe. 

If it's generally stable and no severe issues for most users then it will qualify as a recommended release.

Re "working workaround" - I've complained about the quality of the bug notes numerous times and still continue to do so.  I've pointed out to TAC bugs which have factually incorrect subject lines and notes (literally the opposite meaning of the bug sometimes) and months later some still have not been corrected.  Be sure to rate the bugs as Leo suggests and also ask TAC to escalate with dev/BU to get the notes corrected as well.  The more who do that, the more likely they are to take the trouble to get them right.  It feels like there are no checks and review processes for any of this stuff anymore or the people doing the checks and reviews either aren't capable of spotting the mistakes or just pass things without any checks at all.  The outsourced TAC staff seem to be terrified of contacting devs and BU about anything so you have to escalate to their managers and/or your account team if they refuse to.

[resantero · Accepted solution]

I see they have updated the Known Fixed Releases (1 of 1) table for version 17.12.2 just check that firmware is not yet live on the Download section.

[Rich R]

17.12.2 is available for download now.

[resantero]

Thanks just saw it, has anyone tried it? I see that it is still not a recommended version and might contains more than 1 bug as it's a ED.

Also waiting for the compatibility matrix if the aironet 1800s and 1700s still supported on the version

[JohnCKirk]

There are release notes for the 17.12.x branch as a whole, which say that the 1700 and 1800 APs are supported:
Release Notes for Cisco Catalyst 9800 Series Wireless Controller, Cisco IOS XE Dublin 17.12.x - Cisco

However, there's a note further down the page:

"The DTLS version (DTLSv1.0) is deprecated for Cisco Aironet 1800 based on latest security policies. Therefore, any new out-of-box deployments of Cisco Aironet 1800 APs will fail to join the controller and you will get the following error message:
[..]"