Re: unable to connect vWLC

Martin Kyrc · ‎10-02-2019

Hello,

I have simple network problem - I can't connect to the vWLC message interface... but solution seems to be not so easy.

Let's have a look closer to the issue. In my subnet are connected: virtual ISE, virtual WLC, Cisco AP and some clients (PC). (virtual devices are running on VMware workstation on my laptop). I can "ping" WLC from all devices but from WLC I can't "ping" no device (no FW on the devices). MAC addresses in the ALL network devices are correct.

The question is, why I can't connect to other devices FROM WLC. In the final, ISE and WLC can't communicate (RADIUS) and no http connection to WLC is possible.

IP adresses:

gateway: 192.168.10.1
laptop: 192.168.10.11
WLC: 192.168.10.231
ISE: 192.168.10.232
web server: 192.168.10.233

troubleshooting from gateway router (mikrotik):

ARP table:
192.168.10.11 F4:30:B9:CF:5C:56 VLAN10 
192.168.10.232 00:0C:29:68:35:D7 VLAN10 
192.168.10.233 00:0C:29:5B:49:17 VLAN10 
192.168.10.231 00:0C:29:D2:41:73 VLAN10

PING to all devices is running.

from laptop (win):

ARP table:
  192.168.10.1          cc-2d-e0-c5-38-a6     dynamic
  192.168.10.231        00-0c-29-d2-41-73     dynamic
  192.168.10.232        00-0c-29-68-35-d7     dynamic
  192.168.10.233        00-0c-29-5b-49-17     dynamic

PING to all devices is running.

from web server (linux):

$ arp -an
? (192.168.10.11) at f4:30:b9:cf:5c:56 [ether] on ens33
? (192.168.10.12) at b0:e1:7e:45:51:2e [ether] on ens33
? (192.168.10.244) at 6c:fa:a7:44:d8:b6 [ether] on ens33
? (192.168.10.1) at cc:2d:e0:c5:38:a6 [ether] on ens33
? (192.168.10.231) at 00:0c:29:d2:41:73 [ether] on ens33
? (192.168.10.232) at 00:0c:29:68:35:d7 [ether] on ens33
? (10.215.87.191) at b0:e1:7e:45:51:2e [ether] on ens33

PING is running correctly to ALL devices in this subnet.

and on the WLC:

PING is not answered, but ARP table looks correct:
CC:2D:E0:C5:38:A6   192.168.10.1     1      0      Host
F4:30:B9:CF:5C:56   192.168.10.11    1      0      Host
00:0C:29:68:35:D7   192.168.10.232   1      0      Host
00:0C:29:5B:49:17   192.168.10.233   1      0      Host

MAC records on all devices are correct.

debug from WLC point of view (arp record for .233 not exists):

(Cisco Controller) >ping 192.168.10.233
Send count=3, Receive count=0 from 192.168.10.233

*emWeb: Oct 02 12:05:21.370: dtlArpFindMobile: No ARP entry found 192.168.10.233
*emWeb: Oct 02 12:05:35.626: dtlArpFindMobile: No ARP entry found 192.168.10.233

Send count=3, Receive count=0 from 192.168.10.233

the same "debug arp all" when arp record exists:

!!! note: this is ping attempt from 192.168.10.233:

(Cisco Controller) >
*dtlArpTask: Oct 02 12:09:59.272: processEtherIcmp: Received ICMP request from wired client,
  Interface no:1, mtu:1280, SRC MAC: 00:0C:29:5B:49:17 
*dtlArpTask: Oct 02 12:09:59.272: processEtherIcmp: Sending ICMP reply Successful !! , 
  SRC MAC: 00:0C:29:D2:41:73 
*dtlArpTask: Oct 02 12:10:00.296: processEtherIcmp: Received ICMP request from wired client,
  Interface no:1, mtu:1280, SRC MAC: 00:0C:29:5B:49:17 
*dtlArpTask: Oct 02 12:10:00.296: processEtherIcmp: Sending ICMP reply Successful !! , 
  SRC MAC: 00:0C:29:D2:41:73 

*dtlArpTask: Oct 02 12:10:02.440: dtlARPProtoRecv: Arp request.
  from = 1, client: 00:0c:29:5b:49:17, 
  src ip: 192.168.10.233, tgt ip: 192.168.10.231  mscb: not found

*dtlArpTask: Oct 02 12:10:02.440: Received dtlArpRequest
  sha: 00:0c:29:5b:49:17 spa: 192.168.10.233
  tha: 00:00:00:00:00:00 tpa: 192.168.10.231
  intf: 1, vlan: 0, node type: 1, mscb: not found, isFromSta: 0

!!! learned ARP table:
(Cisco Controller) >show arp switch 
MAC Address         IP Address       Port   VLAN   Type
------------------- ---------------- ------ ------ ------
00:0C:29:5B:49:17   192.168.10.233   1      0      Host
B0:8B:CF:A2:E0:38   192.168.10.251   1      0      Host

!!!note: and ping
(Cisco Controller) >ping 192.168.10.233
Send count=3, Receive count=0 from 192.168.10.233

tcpdump on the 192.168.10.233 shows no icmp packets coming from WLC (192.168.10.231)

I tried several versions of WLC (8.3, 8.5, 8.8). I tried upgrade/reinstall wmvare workstation. But till now no solution.

The simple problem, but not so simple answer. What else can I try?

martin

patoberli · ‎10-03-2019

Can you ouput a show interface summary from the WLC?

Martin Kyrc · ‎10-03-2019

summary:

(Cisco Controller) >show interface summary 
 Number of Interfaces.......................... 3
Interface Name                   Port Vlan Id  IP Address      Type    Ap Mgr Guest
-------------------------------- ---- -------- --------------- ------- ------ -----
management                       1    untagged 192.168.10.231  Static  Yes    N/A  
service-port                     N/A  N/A      192.168.119.10  Static  No     N/A  
virtual                          N/A  N/A      1.1.1.1         Static  No     N/A

detail:

(Cisco Controller) >show interface detailed management 

Interface Name................................... management
MAC Address...................................... 00:0c:29:d2:41:73
IP Address....................................... 192.168.10.231
IP Netmask....................................... 255.255.255.0
IP Gateway....................................... 192.168.10.1
External NAT IP State............................ Disabled
External NAT IP Address.......................... 0.0.0.0
Link Local IPv6 Address.......................... fe80::20c:29ff:fed2:4173/64
STATE ........................................... REACHABLE
Primary IPv6 Address............................. ::/128
STATE ........................................... NONE
Primary IPv6 Gateway............................. ::
Primary IPv6 Gateway Mac Address................. 00:00:00:00:00:00
STATE ........................................... INCOMPLETE
VLAN............................................. untagged  
Quarantine-vlan.................................. 0
Physical Port.................................... 1         
DHCP Proxy Mode.................................. Global
Primary DHCP Server.............................. 192.168.10.1
Secondary DHCP Server............................ Unconfigured
DHCP Option 82................................... Disabled
DHCP Option 82 bridge mode insertion............. Disabled
DHCP Option 6 Opendns Override................... Disabled
IPv4 ACL......................................... Unconfigured
IPv6 ACL......................................... Unconfigured
mDNS Profile Name................................ Unconfigured
AP Manager....................................... Yes
Guest Interface.................................. N/A
L2 Multicast..................................... Enabled

(Cisco Controller) >show interface detailed service-port 

Interface Name................................... service-port
MAC Address...................................... 00:0c:29:d2:41:69
IP Address....................................... 192.168.119.10
IP Netmask....................................... 255.255.255.0
Link Local IPv6 Address.......................... fe80::20c:29ff:fed2:4169/64 
STATE ........................................... NONE
IPv6 Address..................................... ::/128
STATE ........................................... NONE
SLAAC............................................ Disabled
DHCP Protocol.................................... Disabled
AP Manager....................................... No
Guest Interface.................................. N/A
Speed ........................................... 1Gbps
Duplex .......................................... Full
Auto Negotiation ................................ Enabled
Link Status...................................... Up

I'm using "service-port" for remote access to WLC and for radius communication between ISE and WLC. It's workaround only, because I'm not able communicate using "managment" interface.

patoberli · ‎10-03-2019

That's not what the service-port is for. As far as I remember, the service-port is only for out-of-band management of the WLC.

See here: https://community.cisco.com/t5/wireless-and-mobility/wlc-5508-what-is-the-use-of-service-port/td-p/2476585

Create another virtual-interface for the SSID. Depending on your configuration, either the virtual-interface or the management port is used for Radius communication. By default it's the management interface.

Martin Kyrc · ‎10-04-2019

Yes, I know that management interface is reserved for radius communication. But in my case it's not possible. I'm using service-port for accessing wlc (workaround).

As I wrote, I'm not able to communicate FROM management interface to other devices in the same subnet. I the same time I can communicate (I'm testing icmp only, tcp is not wrking) TO management interface from other devices in the same subnet. in both cases is dynamic arp correct.

I have create new dynamic interface "ap":

(Cisco Controller) >show interface summary 
 Number of Interfaces.......................... 4
Interface Name                   Port Vlan Id  IP Address      Type    Ap Mgr Guest
-------------------------------- ---- -------- --------------- ------- ------ -----
ap                               1    untagged 192.168.10.231  Dynamic Yes    N/A  
management                       1    untagged 192.168.1.231   Static  No     N/A  
service-port                     N/A  N/A      192.168.119.10  Static  No     N/A  
virtual                          N/A  N/A      1.1.1.1         Static  No     N/A

But the same story. FROM WLC I can't ping gateway (or other devices in the same nw), but from other devices I can ping WLC on 192.168.10.231.

Again: this is instalation on my laptop under vmware worstation. that's the reason of "untagged".

patoberli · ‎10-04-2019

This configuration should not work. You have now two different IP Ranges configured on the Port 1, while they are not tagged. I think, if you give a client the 192.168.10.200 address, you might be able to ping the WLC interface. But it's also possible that VMware blocks that.

Martin Kyrc · ‎10-04-2019

Sure, two IPs (different subnets) on the same "interface" (port1 and both untagged). from the subnet 192.168.10.0/24 is possible to ping 192.168.10.231, but wlc can't ping nothing in the network 192.168.10/24. in other words, the same story with dynamic interface "ap" as previously configured "management" interface.
Yes, maybe something with vmware workstation. But some days ago I had vWLC version 8.0 (because old AP) and no issue like this. I will try install v8.0 again for testing.
I consulted this issue with some "wifi" colleagues, but no solution. It's not clear for me. I will discard all networking end security certificates :).

patoberli · ‎10-04-2019

Oh one more thing, don't use 1.1.1.1 on the virtual-interface, that will cause issues if you want to offer a guest-network. Use 192.0.2.0/24 or another reserved block: https://en.wikipedia.org/wiki/Reserved_IP_addresses

I don't assume you created any ACL on the WLC? That could also block the outgoing ICMP.

mariusst · ‎01-23-2020

Hi Martin

Have you been able to resolve this issue, I am facing the same issue.

Please share the solution should you have found one.

Thanks

lloydplayfair · ‎03-11-2020

Sadly I have had this issue for well over a year, I haven't found a fix bar the fact that if I use a wireless (controlled by the same vWLC) or wired connection from a physical PC I can access vWLC GUI, anything (VM) running on the Hyper-V (not ESXi in my case) Hypervisor gets the "broken" GUI access, e.g. I can get as far as a malformed login dialog but further pages never display.

lloydplayfair · ‎03-11-2020

I should said I've tried multiple things to no avail, my Hyper-V is using Jumbo Frames I suspect that this might be an issue, but I cant be ask change all settings as I've just used the physical PC workaround... at any-rate disabling Jumbo Frames isn't a fix as i'd like to use Jumbo Frames. Someone needs to raise Cisco TAC, not me as I don't have full cover.

Razmeth · ‎10-23-2020

I have this exact same issue, were you ever able to find a resolution?

lloydplayfair · ‎10-24-2020

Unfortunately not, I can still only connect via Wi-Fi or Physically cabled devices, the Hypervisor or any VM on Hypervisor doesn't communicate correctly.

If you have Cisco support that's best, I don't so just lived with the above scenario in my home lab.

DENNIS BAAS · ‎02-24-2021

Maybe a bit late. But today I ran into the same issue, when trying to run vWLC on VMware Workstation. v16 in this case.

I could ping the management interface on my host, but could not connect to it.

My fix:

- deploy ova

- after deployment:

> assign ethernet0 interface to a localhost network, with DHCP enabled. This interface will be assigned to the service port during installation.

> leave ethernet1 bridged. This will be assigned to the management port during installation.

During the setup wizard, use DHCP to assign the address to the service port.

After installation, you can connect, i.e with a browser, using the service port IP (use command 'show interface summary' on the wlc.

Hope this helps a bit.

Worko · ‎03-12-2021

Unfortunately, this doesn't solve the problem Martin was describing here. Of course your will let you enter the GUI / SSH via Service Port. But the question was - why the vWLC is pingable from the same subnet, but you cannot https/ssh to it from the very same network. Inspite you can do enter via service port. I just installed VM-Player16 and AireOS 8.10.151.0 today and I am facing the same problem.

As you can see below, my network where my PC is connected to and the home-router provides the gateway (192.168.180.1/24) is up and running. But I cannot ping anything outside the vWLC, even not the GW.

I am using the service port (first Network Adapter inside the vmware player Network settings) for entering HTTPS or SSH.

(Cisco Controller) >show interface summary


 Number of Interfaces.......................... 3

Interface Name                   Port Vlan Id  IP Address      Type    Ap Mgr Guest
-------------------------------- ---- -------- --------------- ------- ------ -----
management                       1    untagged 192.168.180.100 Static  Yes    N/A
service-port                     N/A  N/A      192.168.171.100 Static  No     N/A
virtual                          N/A  N/A      192.0.250.1     Static  No     N/A

(Cisco Controller) >show interface detailed management

Interface Name................................... management
MAC Address...................................... 00:0c:29:b9:4c:24
IP Address....................................... 192.168.180.100
IP Netmask....................................... 255.255.255.0
IP Gateway....................................... 192.168.180.1
External NAT IP State............................ Disabled
External NAT IP Address.......................... 0.0.0.0
Link Local IPv6 Address.......................... fe80::20c:29ff:feb9:4c24/64
STATE ........................................... REACHABLE
Primary IPv6 Address............................. ::/128
STATE ........................................... NONE
Primary IPv6 Gateway............................. ::
Primary IPv6 Gateway Mac Address................. 00:00:00:00:00:00
STATE ........................................... INCOMPLETE
VLAN............................................. untagged
Quarantine-vlan.................................. 0
Physical Port.................................... 1
DHCP Proxy Mode.................................. Global
Primary DHCP Server.............................. 192.168.180.1
Secondary DHCP Server............................ Unconfigured
DHCP Option 82................................... Disabled

--More-- or (q)uit
DHCP Option 82 bridge mode insertion............. Disabled
DHCP Option 6 Opendns Override................... Disabled
IPv4 ACL......................................... Unconfigured
IPv6 ACL......................................... Unconfigured
mDNS Profile Name................................ Unconfigured
AP Manager....................................... Yes
Guest Interface.................................. N/A
L2 Multicast..................................... Enabled

(Cisco Controller) >
(Cisco Controller) >
(Cisco Controller) >ping 192.168.180.1

Send count=3, Receive count=0 from 192.168.180.1