Hi, everyone!
Have two core catalyst 3750X switches in a stack with 15.2(1)E IOS version. When I add VLAN to configuration (doesn't matter what number it is) there is an increase in CPU utilization:
CPU utilization for five seconds: 99%/1%; one minute: 99%; five minutes: 99%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
247 2732742 130875 20880 59.29% 58.81% 58.65% 0 VMATM Callback
159 3503376928 3283516600 0 11.05% 11.62% 11.53% 0 Hulc LED Process
162 786990505 109715445 7173 5.60% 5.18% 5.20% 0 HL3U bkgrd proce
Also all network goes down like in broadcast-storm and on neighbour switches bpduguard is triggered in added VLAN:
%SPANTREE-CLUSTER_MEMBER_13-2-LOOPGUARD_BLOCK: Loop guard blocking port Port-channel1 on VLAN0210.
%SPANTREE-CLUSTER_MEMBER_13-2-LOOPGUARD_UNBLOCK: Loop guard unblocking port Port-channel1 on VLAN0210.
%SPANTREE-CLUSTER_MEMBER_13-2-LOOPGUARD_BLOCK: Loop guard blocking port Port-channel1 on VLAN0210.
But there are no loops on physical topology level.
Logs on 3750X contain this:
%HLFM-6-ADDR_SYNC: The system is retrying to sync the MAC addresses: reason 2
%HLFM-3-SEND_FAIL: Failed to send RPC message, req 3,
-Traceback= 67945Cz 26880C0z 2D61754z 2D61FECz 2D5EEF0z 2D5F044z 2D5771Cz 2D51C54z
%HLFM-3-SEND_FAIL: Failed to send RPC message, req 3,
-Traceback= 67945Cz 26880C0z 2D61754z 2D61FECz 2D5EEF0z 2D5F044z 2D5
%HLFM-3-SEND_FAIL: Failed to send RPC message, req 3,
-Traceback= 67945Cz 26880C0z 2D61754z 2D61FECz 2D5EEF0z 2D5F044z 2D5771Cz 2D51C54z
%HLFM-3-SEND_FAIL: Failed to send RPC message, req 3,
-Traceback= 67945Cz 26880C0z 2D61754z 2D61FECz 2D5EEF0z 2D5F044z 2D5771Cz 2D51C54z
%HLFM-6-ADDR_SYNC: The system is retrying to sync the MAC addresses: reason 2
%HLFM-3-SEND_FAIL: Failed to send RPC message, req 3,
-Traceback= 67945Cz 26880C0z 2D61754z 2D61Ftopoid 0
No entries in MAC-table in added VLAN.
Switches in a stack functioned in vtp client mode (don't ask why...). I tried to turn them in transparent mode, but that didn't help. Also I tried to replace one useless VLAN (there are 77 VLANs in total) on the new one, but unsuccessfully. Moreover when I returned deleted VLAN in configuration CPU rised to 99% and the same logs appeared in a log buffer.
I found similar problem: https://bst.cisco.com/bugsearch/bug/CSCto85486 and tried to capture BPDU frames with topology change flag. But there is no such traffic in captured dump. Perhaps the problem lies in VP instances. Unfortunately I have no strong knowledges about that and didn't find commands to disable it on 3750X.
Maybe someone have some ideas?
I think it is a software bug and don't know how to solve this problem (organization doesn't have service contract to download new IOS version or start a ticket in Cisco TAC). My last step is just reload the device. Maybe somebody know decision that doesn't require rebooting?
