3850 stack dropping packets

[awojtowicz]

i have a 3850 stack of 2 switches in my mdf.  i have another switch in my idf connected by fiber.  everything has been good for years.  i just added another 3850 switch into my stack in my mdf.  now i cant access by idf switch at all.  cant ping or putty it.  but it shows in show cdp neighbor.  i have some FortiAP's on switch 1 in my mdf stack.  i keep losing them every so often.  i'll run a constant ping to something else behind the network and i lose 6 packets every time when this happens.  i drop all packets into this network whenever this 6 packet drop happens.  i even took that switch i added out of the stack and it doesnt matter, the same problem.  It was a bare switch.  same OS.  ive done this many times in my network but never had this happen before.

[Mark Elsen]

   - @awojtowicz                Check logs on the  stack when manipulations are done, such as adding or deleting
                                          a stack member , do the same on other switches involved in the issue; if any
                                          Use and or define a common syslog server which is a better solution for collecting logs

 M.

[NetworkNerd1337]

A couple of things in your description narrow this down a lot.

1. You can see the IDF switch in CDP but can't ping or SSH to it. CDP is a layer 2 protocol and doesn't care about IP, VLANs or spanning tree state on your management VLAN. So the fiber and the physical link are fine. The break is somewhere at layer 2/3 on the management path: the management VLAN not being forwarded across that trunk (allowed list, native VLAN mismatch, VLAN missing, or STP blocking/flapping in that VLAN), or an ARP/duplicate IP problem.

2. Everything behind the network drops for about 6 pings at a time, and the APs on switch 1 drop with it. That's an outage of the whole path, not congestion. Buffer or QoS output drops lose some packets under load. They don't take everything down for a few seconds and then come back clean. This looks like a layer 2 control plane event: a spanning tree reconvergence (root change, a flapping port in the active topology, or a loop), or the stack ring itself bouncing.

3. Pulling the new switch back out didn't fix it. So whatever changed is still there. Usual suspects: the stack cabling isn't what it was before (a ring that's now half cabled, or a stack cable/port that's flapping), the spanning tree root moved, or a config or STP change that happened while the third switch was in.

One detail worth checking: how are you pinging? Windows waits about 4 seconds for each request that times out, so 6 lost pings there is roughly 25 to 30 seconds of outage. That's very close to legacy PVST+ convergence (2 x 15 second forward delay) and would point hard at spanning tree. If it's Linux or a network device sending one per second, it's more like 5 to 6 seconds, which fits a stack ring or link event better.

Here's the order I'd go in.

Step 1. Check the stack is actually healthy

show switch
show switch detail
show switch stack-ports summary
show switch stack-ring speed

You want every member in Ready state, nothing in V-Mismatch or Lic-Mismatch. In stack-ports summary, look at the "#Changes to LinkOK" column. A small number like 1 after boot is normal. If any port has a count that keeps going up between runs, that stack cable or port is flapping. Every flap disturbs traffic that crosses the stack. Run it a few times, including right after you see a drop. Also confirm the cabling is a complete ring. With 2 members you want both cables between 1 and 2. If you went 1 to 2, 2 to 3, 3 to 1 and then pulled switch 3, put the second cable back between 1 and 2. stack-ring speed should say Full, not Half. Check the uptime of each member too (show version lists them). A member that's been quietly reloading will show up there.

Step 2. Pull the logs around a drop

This is where the earlier reply is pointing, and I agree it's the first thing to look at. Run a timestamped continuous ping, then as soon as it drops:

show logging | include STACKMGR|SPANTREE|MACFLAP|DUPADDR|UPDOWN|HA_|REDUNDANCY

What each one tells you:

• STACKMGR: stack port or member changes, see step 1
• SPANTREE: loop guard, root guard, BPDU guard or other STP events
• MACFLAP (%SW_MATM-4-MACFLAP_NOTIF): the same MAC learned on two ports flipping back and forth. That's the classic loop signature, and the log tells you which ports
• DUPADDR (%IP-4-DUPADDR): duplicate IP, which could also explain the IDF management IP not answering
• UPDOWN: an interface bouncing at the same time as the drop, e.g. the IDF uplink or an AP port

If your logging buffer is small or the clock isn't set, fix that first so you can line events up with the drops:

conf t
 logging buffered 128000 informational
 service timestamps log datetime msec localtime show-timezone
end

Like the earlier reply said, sending this to a syslog server from all the switches makes correlating much easier.

Step 3. Spanning tree, which is my top suspect

show spanning-tree summary
show spanning-tree root
show spanning-tree detail | include ieee|occurr|from|is exec

The first one shows the mode (pvst vs rapid-pvst) and confirms which you're actually running. The second shows who the root is per VLAN. If the root for your user and management VLANs isn't the MDF stack, or it's not the bridge you expect, that's a problem worth fixing on its own. The last one shows for each VLAN how many topology changes there have been, how long ago the last one was, and the port it came in on. If the topology change count climbs every time you lose your 6 pings, follow "from <port>" to the device generating it and repeat the command there until you find the source.

If the stack isn't explicitly set as root, set it so a change elsewhere can't move the root around:

conf t
 spanning-tree vlan <your vlans> root primary
end

(or spanning-tree vlan <vlans> priority 4096 if you'd rather set it explicitly). Also make sure your AP and end user ports have portfast, so an AP rebooting doesn't generate a topology change:

interface range <AP and access ports>
 spanning-tree portfast

If you're still on pvst and everything in the L2 domain supports it, moving to rapid-pvst cuts convergence from roughly 30 seconds to around a second. Plan that for a maintenance window, since changing the mode reconverges everything.

Step 4. The IDF management path

On the MDF stack:

show cdp neighbors <uplink to IDF> detail
show interfaces <uplink to IDF> trunk
show spanning-tree vlan <mgmt vlan>
show ip arp <IDF mgmt IP>

CDP detail gives you the IP address the IDF switch is advertising and its native VLAN. If the native VLANs don't match you'll also see %CDP-4-NATIVE_VLAN_MISMATCH in the log. In the trunk output, the management VLAN has to show up in all three sections: allowed, allowed and active, and forwarding state and not pruned. If it's missing from the last one, it's blocked by STP or pruned by VTP. For ARP, compare the MAC you get for the IDF management IP with the IDF switch's real MAC. If it's a different MAC, something else has that IP. If it's incomplete, ARP isn't making it across.

Since you can't get to the IDF switch remotely, a console session there would help a lot:

show ip interface brief
show running-config | include default-gateway
show spanning-tree vlan <mgmt vlan>
show interfaces trunk
show vtp status
show vlan brief
show logging

If the IDF switch can ping its default gateway but you can't reach it from another subnet, look at the gateway and routing side. If it can't ping its gateway, it's the VLAN/trunk/STP side. Also check show vtp status on both ends. If VTP is in server/client mode, make sure the domain and revision look sane and the management VLAN still exists in show vlan brief everywhere.

Step 5. The uplink itself

show interfaces <uplink to IDF>
show interfaces counters errors
show etherchannel summary

Look for CRC/input errors and how many times the interface has reset or flapped. If the IDF uplink is an EtherChannel, make sure all members are bundled (P) and nothing is sitting in suspended or individual (s / I) state.

If you can post the output of show version (just the version and the switch table), show switch, show switch stack-ports summary, show spanning-tree detail | include ieee|occurr|from|is exec, and the show logging lines from around one of the drops, it should be pretty clear which of these it is. Also helpful: is the IDF uplink on switch 1, 2 or both, is it a single link or a port-channel, and is the IDF switch layer 2 only with an ip default-gateway, or does it do any routing?

[awojtowicz]

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48P 16.3.6 CAT3K_CAA-UNIVERSALK9 INSTALL
2 56 WS-C3850-48T 16.3.6 CAT3K_CAA-UNIVERSALK9 INSTALL
3 56 WS-C3850-48P 16.3.6 CAT3K_CAA-UNIVERSALK9 INSTALL

sho swi
Switch/Stack Mac Address : dca5.f4b4.2c00 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active dca5.f4b4.2c00 15 V02 Ready
2 Standby d0c7.8957.4c80 13 V02 Ready
3 Member dca5.f453.5400 10 V02 Ready

BOE_MDF_3850STK#show switch stack-ports summary
Sw#/Port# Port Status Neighbor Cable Length Link OK Link Active Sync OK #Changes to LinkOK In Loopback
-------------------------------------------------------------------------------------------------------------------
1/1 OK 3 50cm Yes Yes Yes 1 No
1/2 OK 2 50cm Yes Yes Yes 1 No
2/1 OK 1 50cm Yes Yes Yes 1 No
2/2 OK 3 50cm Yes Yes Yes 1 No
3/1 OK 2 50cm Yes Yes Yes 1 No
3/2 OK 1 50cm Yes Yes Yes 1 No

BOE_MDF_3850STK#show spanning-tree detail | include ieee|occurr|from|is exec
VLAN0001 is executing the rstp compatible Spanning Tree protocol
Number of topology changes 1 last change occurred 1d01h ago
from Port-channel70
VLAN0100 is executing the rstp compatible Spanning Tree protocol
Number of topology changes 49 last change occurred 00:19:28 ago
from Port-channel70
VLAN0101 is executing the rstp compatible Spanning Tree protocol
Number of topology changes 16 last change occurred 13:43:05 ago
from Port-channel70
BOE_MDF_3850STK#show logging
Syslog logging: enabled (0 messages dropped, 55 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.

No Inactive Message Discriminator.

Console logging: level emergencies, 0 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 12512 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Persistent logging: disabled

No active filter modules.

Trap logging: level debugging, 12516 message lines logged
Logging Source-Interface: VRF Name:

Log Buffer (4096 bytes):
: Error parsing from CAPWAP socket
*Oct 20 04:08:57.150: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:08:59.151: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:08:59.150: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:09:44.308: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:09:44.306: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:09:48.313: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:09:48.311: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:10:27.153: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:27.151: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:28.192: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:28.190: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:31.155: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:31.153: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:59.308: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:10:59.307: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:11:03.307: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:11:03.306: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:11:36.154: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:11:36.152: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:11:40.154: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:11:40.153: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:11:41.155: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:11:41.153: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:12:36.313: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:12:36.311: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:12:38.310: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:12:38.308: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:13:13.156: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:13.155: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:15.155: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:15.158: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:46.312: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:13:46.311: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
*Oct 20 04:13:47.315: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:47.314: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:48.321: *%SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
*Oct 20 04:13:48.323: *%SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from

[NetworkNerd1337]

Thanks for posting all that, it narrows things down a lot.

The stack itself looks healthy. All three members are Ready, every stack port shows OK and Sync OK, and each has only 1 change to LinkOK, so the stack ring isn't flapping. I'd take the stack cabling off the list.

Two things in your output stand out.

1) Spanning tree on VLAN 100

VLAN 100 has had 49 topology changes, the last one about 19 minutes before your capture, and every one was received on Port-channel70 (VLAN 101 has 16, also from Po70). With RSTP, every topology change makes each switch flush the MAC addresses learned on its non-edge ports for that VLAN, and if the change comes from a real link or root change you also get a short reconvergence gap. Repeated bursts of lost pings that hit everything at once fit that pattern well.

"From Port-channel70" means the change came in from the switch on the other side of Po70, so the next step is to follow it to where it starts:

- What is on the far end of Po70? Is that the IDF uplink?

- show etherchannel 70 summary (are all members (P), or are any suspended (s) or standalone (I)?)

- show interfaces port-channel 70, and show interfaces counters errors for the member ports

- On the switch at the other end of Po70, run show spanning-tree vlan 100 detail | include occurr|from and see which of its ports the changes come from. Repeat hop by hop until you find the port where they originate. That's usually a flapping link or a port facing an end device that isn't set as an edge port.

- show spanning-tree vlan 100 root on the stack and on the IDF switch, to confirm the root is where you expect and isn't moving. If the root isn't pinned, set it on the MDF stack with spanning-tree vlan 1,100,101 priority 4096 (or root primary).

- Make sure AP and other end-device ports are spanning-tree portfast (portfast trunk if the FortiAPs are on trunk ports), so an AP rebooting doesn't generate a topology change.

2) The CAPWAP errors

%SOCKET-2-SOCK_RECV_ERR ... wcm: Error parsing from CAPWAP socket comes from the 3850's built-in wireless controller process (wcm). FortiAPs use CAPWAP to find and join their FortiGate, but the 3850 can't parse their packets, so if their discovery traffic reaches the 3850's wireless management interface you get exactly this message. Two per second from Switch 1 and 2 suggests the APs are spending a lot of time in discovery, which is what you'd expect if they keep losing their controller. So I'd treat it as a symptom of the drops more than the cause. Two practical points:

- If you don't use the 3850 as a wireless controller, check show running-config | include wireless management. Removing that interface should stop wcm from listening for these packets.

- Your 4K log buffer is full of these messages, so any STP or link messages are being pushed out. Increase it with logging buffered 512000 debugging, and send logs to a syslog server as Mark suggested. Then look at show logging | include LINK|LINEPROTO|SPANTREE|EC-5 around the time of a drop.

3) The IDF switch

Seeing it in CDP but not being able to ping or SSH to it means the physical link and Layer 2 are up, but the management VLAN isn't making it end to end. If the IDF uplink is Po70, the two problems are probably related. On both ends check show interfaces trunk (management VLAN allowed and forwarding, not pruned), show spanning-tree vlan <mgmt vlan> on the IDF uplink (forwarding, not blocking), and show ip arp <IDF mgmt IP> on the stack. A native VLAN mismatch would also log %CDP-4-NATIVE_VLAN_MISMATCH.

Since taking the new switch back out didn't help, something that changed when you added it may still be in place. If you have a config backup from before the change, a diff of the running config against it (especially Po70, its member ports, and the uplink trunk) is worth a look.

If you can post the Po70 output and the STP detail from the switch on the far end, that should pin down where the changes start.

[awojtowicz]

im looking at my config thats running and a copy from a few months ago.  i see one difference.

in the old config it shows.

interface GigabitEthernet0/0
 vrf forwarding Mgmt-vrf
 no ip address
 negotiation auto

but in new running config it shows

interface GigabitEthernet0/0
 vrf forwarding Mgmt-vrf
 no ip address

shutdown

negotiation auto

[NetworkNerd1337]

that one's almost certainly not your problem. Gi0/0 on the 3850 is the out-of-band mgmt port and it lives in Mgmt-vrf, completely separate from the data plane. it had no ip address in the old config either, so it wasn't doing anything before it got shut. someone (or a hardening template/script) admin-shut an unused port, which is honestly good practice, not a cause of drops.

the more useful takeaway is that if that's the only diff between a config from a few months ago and today, the config probably didn't change into this problem. a few caveats before you rule config out completely:

1. make sure you're comparing the full running config, not just interface sections. if the old copy is in flash you can diff it right on the box:
show archive config differences flash:<old-config-file> system:running-config
2. VLANs won't show up in the running config if VTP is in server or client mode, they live in vlan.dat. check show vtp status and look at the configuration revision. if it's been climbing, something is pushing VLAN changes.
3. check whether the code or uptime changed even if the config didn't. show version gives you the running version, uptime and last reload reason. a reload or upgrade around when the drops started would be worth knowing.
4. if you want to know who shut Gi0/0 and when, show archive log config all will tell you, but only if config logging is enabled (archive / log config / logging enable). if it isn't, worth turning on so the next change is attributable.

with the config looking stable, i'd go back to the 49 topology changes on VLAN 100, that's still your strongest lead. every TC flushes MAC entries for that vlan, traffic floods until it relearns, and on a busy vlan that looks a lot like random drops. follow it hop by hop:
show spanning-tree vlan 100 detail | i ieee|occurr|from|is exec

it shows the port the last change came in on. if it's Po70, find what's on the other end (show etherchannel 70 summary for the member ports, then show cdp neighbors <member-port> detail) and run the same command on that switch. keep going until the "from" port is an access port or a downstream device, that's your source. most of the time it's an edge port without portfast flapping (a phone, AP or PC rebooting), and spanning-tree portfast on that access port stops it from generating TCs.

also worth a quick look at show logging | i MACFLAP|LINK-3|LINEPROTO to see if a link flap or a MAC moving between ports lines up with the drops.

[awojtowicz]

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----

*
1 56 WS-C3850-48P 16.3.6 CAT3K_CAA-UNIVERSALK9 INSTALL
*
2 56 WS-C3850-48T 16.3.6 CAT3K_CAA-UNIVERSALK9 INSTALL
3 56 WS-C3850-48P 16.3.6 CAT3K_CAA-UNIVERSALK9 INSTALL

sho swi
Switch/Stack Mac Address : dca5.f4b4.2c00 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
1 Active dca5.f4b4.2c00 15 V02 Ready
2 Standby d0c7.8957.4c80 13 V02 Ready
3 Member dca5.f453.5400 10 V02 Ready

BOE_MDF_3850STK#show switch stack-ports summary
Sw#/Port# Port Status Neighbor Cable Length Link OK Link Active Sync OK #Changes to LinkOK In Loopback
-------------------------------------------------------------------------------------------------------------------
1/1 OK 3 50cm Yes Yes Yes 1 No
1/2 OK 2 50cm Yes Yes Yes 1 No
2/1 OK 1 50cm Yes Yes Yes 1 No
2/2 OK 3 50cm Yes Yes Yes 1 No
3/1 OK 2 50cm Yes Yes Yes 1 No
3/2 OK 1 50cm Yes Yes Yes 1 No
BOE_MDF_3850STK#show spanning-tree detail | include ieee|occurr|from|is exec
VLAN0001 is executing the rstp compatible Spanning Tree protocol
Number of topology changes 1 last change occurred 1d01h ago
from Port-channel70
VLAN0100 is executing the rstp compatible Spanning Tree protocol
Number of topology changes 49 last change occurred 00:19:28 ago
from Port-channel70
VLAN0101 is executing the rstp compatible Spanning Tree protocol
Number of topology changes 16 last change occurred 13:43:05 ago
from Port-channel70
BOE_MDF_3850STK#show logging
Syslog logging: enabled (0 messages dropped, 55 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
No Active Message Discriminator.

No Inactive Message Discriminator.

Console logging: level emergencies, 0 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 12512 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Persistent logging: disabled
No active filter modules.
Trap logging: level debugging, 12516 message lines logged
Logging Source-Interface: VRF Name:
Log Buffer (4096 bytes):
: Error parsing from CAPWAP socket
Oct 20 04:08:57.150: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:08:59.151: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:08:59.150: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:09:44.308: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:09:44.306: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:09:48.313: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:09:48.311: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:10:27.153: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:27.151: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:28.192: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:28.190: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:31.155: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:31.153: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:59.308: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:10:59.307: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:11:03.307: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:11:03.306: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:11:36.154: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:11:36.152: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:11:40.154: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:11:40.153: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:11:41.155: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:11:41.153: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:12:36.313: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:12:36.311: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:12:38.310: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:12:38.308: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:13:13.156: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:13.155: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:15.155: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:15.158: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:46.312: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:13:46.311: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket[...It occurred 2 times/sec!.]
Oct 20 04:13:47.315: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:47.314: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:48.321: %SOCKET-2-SOCK_RECV_ERR:Switch 2 R0/0: wcm: Error parsing from CAPWAP socket
Oct 20 04:13:48.323: %SOCKET-2-SOCK_RECV_ERR:Switch 1 R0/0: wcm: Error parsing from