Guest Anchor Controller - Foreign Controller Control Path Down

[awatson20]

We have a Cisco 4400 series wireless controller deployed as a Guest Anchor in a private DMZ.  We have 13 foreign controllers anchored to this for Guest

Wireless.  We recently anchored 17 additional controllers to this Anchor controller. Since we have done that, periodically on just 3 of the foreign controllers, the control path shows down on the mobility peer, then comes back up.  We have had this issue in the past, but it resolved itself.  However, now we are seeing this issue again. Are we reaching a limit on EoIP tunnels?  I have read that there is a max of 71, and that is per controller, not SSID. We do have a firewall in the middle but all necessary ports are open.

We have had this issue for quite sometime, it just does not happen frequently.  Since we have added the additional controllers, it is now happpening very often, but only with 3 controllers.  There is not much in common with these 3 controllers.  2 are 4400 series, and 1 is a 5508.  All 3 are local on a campus LAN, different networks.  Could it have anything to do with memory or utilization?

Thanks.

[Darren Ramsey]

Had a similar issue back in 2008 after upgrading from 4.0 to 4.2 and the issue was not fixed until 7.0.  Basically the WLC was sending out unicast ARP requests every 5 seconds to the gateway to prevent the ARP entries from getting stale. However GLBP changes the gateway with every ARP request and the WLC did not receive any response from the 2nd virtual MAC on the GLBP.  So it's not able to resolve its ARP and hence the UDP packets were being dropped until the GW ARP got updated.

Long story short, it presented itself as after so many (more than 5 or 6) foreign controllers were added to the guest anchor, then the EOIP tunnels would go up and down only for the last ones in the list  It was a timing issue around the number of foreigns going top-down on the list based on MAC address sorting. The anchor trusted interface was pointed to a GLBP VIP and the other interface was in the DMZ for guest access. Now this may not be your exact issue but this might help you steer TAC in the right direction. Cisco ended up sending one of the original Airespace engineers on site and figured out the issue after TAC and the WBU were stumped for 2+ months. Our short term work around was to point the WLC anchor at a real address and not a GLBP VIP with a loss of FHRP. This was finally fixed in 7.0. The bugids are CSCsv21464 and CSCsu67530. Hope this is helpful and good luck.