WISM 7.0.235.0 post-upgrade problem?

WILLIAM BAUER · ‎06-19-2012

I upgraded one of our WISM-1 modules from 7.0.98.0 to 7.0.235.0 last night. For some reason, APs don't join with it unless I tell them specifically to do so. We haven't specified primary & secondary controllers on purpose, allowing the APs to determine from DHCP & the controllers' responses and decide on one based on load. This has always worked great for us.

After upgrading, I couldn't get an AP to join the upgraded wism even after I specified the controller in the AP config. So then I changed my 6506 load balancing to src-dst-mac since that is a known (although seldom seen) issue with APs joining a controller for the first time. We usually keep it set at

src-dst-mixed-ip-port. That worked & the AP joined the upgraded wism. Then I reset the load-balancing algorithm on the 6506, removed the specific controller from the AP's config, rebooted the AP, and all is fine. I thought that solved it. Not.

Any other AP that I reboot tries to join the upgraded wism since it has only 1 AP connected, but it fails and the AP joins the other wism running 7.0.98.0. Even if I change the load balancing algorithm to src-dst-mac, src-dst-ip, or src-mac, it won't join unless I specify the upgraded controller, which I don't want to do. I can see the wism responding to the join requests, but the APs still end up on the other controller. It soulds like the years-old load balancing algorithm issue, but that doesn't seem to be the whole answer this time.

I hope this information makes sense to those that are aware of the issues I bring up. Any ideas why the 7.0.235.0 wism isn't getting APs to join it successfully without my specifying that controller? The config hasn't changed, except for what new or changed defaults exist. I suspect it might have to do with one of those.... Or could it be the two different controller versions returning confusing and different responses to the initial query?

Thanks.

Bill

WILLIAM BAUER · ‎06-19-2012

Oh yeah. Before anyone asks, our DHCP servers are external ISC DHCP servers. We're not using the built-in DHCP service on the WISMs. I don't think the problem is DHCP related anyway, unless I need to tweak that config due to an options change I missed in the release notes.

Leo Laohoo · ‎06-19-2012

We're not using the built-in DHCP service on the WISMs.

A wise decision.

I've upgrade our WiSM-1's to this version and we don't have any issues.

However, in the past, we use to host our DHCP server for the WAPs and the clients on a plain Linux box and we noticed that the Linux box took a painfully long, long time to dish out IP addresses to around 1k WAPs. We later moved our DHCP server to InfoBlox and the problems go away.

Any other AP that I reboot tries to join the upgraded wism since it has only 1 AP connected, but it fails and the AP joins the other wism running 7.0.98.0. Even if I change the load balancing algorithm to src-dst-mac, src-dst-ip, or src-mac, it won't join unless I specify the upgraded controller, which I don't want to do. I can see the wism responding to the join requests, but the APs still end up on the other controller. It soulds like the years-old load balancing algorithm issue, but that doesn't seem to be the whole answer this time.

Never saw this problem at all.

WILLIAM BAUER · ‎06-20-2012

I have a working theory, although unproven. With continued testing, I suspect that the APs are choosing the controller with the lower code version. This might be by design of Cisco, or as I pondered earlier, through some indirect means due to a (minor?) difference in responses received by the controllers. The new code works fine otherwise.

I'll just have to wait until I upgrade the other controller to confirm if it's only the difference in code versions between the two that is causing this behavior.

wesleyterry · ‎06-20-2012

Although I've not seen it referenced in any discussions on the AP Join Process, I wonder if there is a preference to a WLC that is running the same code? For example, if all your APs are on 7.0.98.0, and you upgrade 1 WLC, all of those APs might still prefer the 7.0.98.0 before they consider the capacity aspect of AP load-balancing.

I dont know if this is true right now, but it would make sense to do something like that to prevent APs from getting in an AP Upgrade/Downgrade loop. For example, you could have a scenario that an AP that just upgraded code, reboots and now joins the 7.0.98.0 wlc.... downgrades... reboots... joins the 7.0.235.0.... so on.....

it was just a thought.

Saravanan Lakshmanan · ‎06-20-2012

After forcing the AP to join specifically to 7.0.235.0 WLC does it pull the code, join & stays there or coming back to 7.0.98.0 wlc.

Are issue seen on different model AP.

is out of box AP learning 98 WLC through Mobility or option 43, cut off to hold on to 235 for now.

WILLIAM BAUER · ‎06-21-2012

Problem resolved, so it's all speculation....

Upgraded the other WISM to 7.0.235.0, and now they're "option 43 load balancing" again (for lack of a better term). As for the why--?? More testing saw APs that I forced onto 235 unit would usually end up again on the 98 WISM when given the choice, even after they completed the code upgrade. This included 1250, 1040, 1140, 1130, 1120, 1310 series products.

So anyone who has that mental bulletin board covered with notes, clear a space. At least with these two versions, for whatever reason the APs prefer the lower code version until the controllers are again equal.