Guest Anchor Controller - Foreign Controller Control Path Down

[awatson20]

We have a Cisco 4400 series wireless controller deployed as a Guest Anchor in a private DMZ.  We have 13 foreign controllers anchored to this for Guest

Wireless.  We recently anchored 17 additional controllers to this Anchor controller. Since we have done that, periodically on just 3 of the foreign controllers, the control path shows down on the mobility peer, then comes back up.  We have had this issue in the past, but it resolved itself.  However, now we are seeing this issue again. Are we reaching a limit on EoIP tunnels?  I have read that there is a max of 71, and that is per controller, not SSID. We do have a firewall in the middle but all necessary ports are open.

We have had this issue for quite sometime, it just does not happen frequently.  Since we have added the additional controllers, it is now happpening very often, but only with 3 controllers.  There is not much in common with these 3 controllers.  2 are 4400 series, and 1 is a 5508.  All 3 are local on a campus LAN, different networks.  Could it have anything to do with memory or utilization?

Thanks.

[Amjad Abdullah]

Do mobility ping works fine when the problem is happenning?

Try debugs to check further:

Debug mobility handooff enable

Debug mobility keepalive enable

HTH

Amjad

Sent from Cisco Technical Support iPad App

[awatson20]

Yes, mobility pings work when this happens(eping/mping) however it is disruptive to clients. I have run all of the debugs, but nothing stands out. I have 29 foreign controllers anchored. This problem started happening when I added the additional controllers. So, as a test I Removed 5 of them, and since I did that, none have dropped now. I understand the sizing limitations, and an not exceeding that, although it acts like I am.

Sent from Cisco Technical Support iPhone App

[Scott Fella]

Are you using the same mobility group name by chance? You might be hitting the limit of 25 per mobility group. Each building and anchor can be on a different mobility group if there is no roaming between sites.

Sent from Cisco Technical Support iPhone App

[awatson20]

If i understand what your asking, no. Here is how we do it. Anchor has one group name, and the foreign controllers are in different mobility groups, not the same.

Controller. Group Name

Anchor Controller- Anchor-1

Controller-1. Controller-1

Controller-2. Controller-2

Sent from Cisco Technical Support iPhone App

[Scott Fella]

If you put those wlc back on and remove a few others, is it still stable or is it isolated to the ones you remove?

Sent from Cisco Technical Support iPad App

[awatson20]

Thanks Scott, sorry for not getting back on your question. 

When I add the 6 controllers back to the anchor controller as mobility peers, the problem starts re-occurring and it is the same 3.  If I sort the list of wireless controllers in my mobility group, the 3 this is happening too have the highest IP addresses out of the 29 controllers.  (172.31.211.250, 172.31.228.225, 172.31.254.193)  Not that that has anything to do with the problem, that's just the only thing these 3 WLC have in common.

I have a open TAC case with Cisco, but no luck yet on a resolution. 

[Scott Fella]

I'm curious to see what Cisco comes up with.  The ip address really should have no impact on the situation.  Keep us updated.

[Stephen Rodriguez]

in your firewall, are the WLC's allowed to establish the tunnel bi-directionally?

IIRC, the WLC with the lowest MAC will be the 'master' for the pair.  So if the WLC in the DMZ is the master, there could be issues initiating the traffic for keep-alive.

HTH,
Steve

-----------------------------------------
Please remember to rate useful posts, and mark questions as answered

[awatson20]

Yes. We checked the firewall, which is a small checkpoint SOHO device, and the rule is set up so that either side, foreign or anchor controller can initiate the tunnel.

[George Stefanick]

You know I have 1 controller that goes up and down in my enviroment. It just so happens this one controller doesnt have any APs on it.  So I have it on my list to dig into, just not a priority at the moment.

Does your controllers have access points and clients on them ?

[awatson20]

Yes, all of the foreign controllers have access points and clients on them.   

[George Stefanick]

Please let us know what TAC comes back with .. This is a good one ...

[awatson20]

I will.  I do not plan on closing this one until there is a definite resolution from Cisco.  Our Guest wireless network has lost credibility due to this issue.

[George Stefanick]

You my friend, are on the right track. Wireless and Credibility ALWAYS go hand and hand. I cant tell you how many hours I spend a week educating folks on the difference between sucky wireless and sucky clients.