I have 3650 switches. one of the switches where downstrem switches are connected was rebooted and all rest downstrem switches went unreachable as the trunk ports blocked ..can someone help to understand why this is happening and how to resolve it.
000925: Jun 17 09:06:32.445: %LINK-3-UPDOWN: Interface GigabitEthernet1/1/3, changed state to down 000926: Jun 17 09:15:06.826: %LINK-3-UPDOWN: Interface GigabitEthernet1/1/3, changed state to up 000927: Jun 17 09:15:07.940: %SPANTREE-2-BLOCK_PVID_PEER: Blocking GigabitEthernet1/1/3 on VLAN0200. Inconsistent peer vlan. 000928: Jun 17 09:15:07.941: %SPANTREE-2-BLOCK_PVID_LOCAL: Blocking GigabitEthernet1/1/3 on VLAN0100. Inconsistent local vlan. 000929: Jun 17 09:15:07.941: %SPANTREE-2-BLOCK_PVID_PEER: Blocking GigabitEthernet1/1/3 on VLAN0900. Inconsistent peer vlan. 000930: Jun 17 09:15:08.835: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/1/3, changed state to up 000931: Jun 17 09:15:22.940: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet1/1/3 on VLAN0200. Port consistency restored. 000932: Jun 17 09:15:22.942: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet1/1/3 on VLAN0900. Port consistency restored. 000933: Jun 17 09:15:22.942: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet1/1/3 on VLAN0100. Port consistency restored.
the issue was solved quickly in 15 seconds according to your log messages.
Most of the downtime was caused by the switch reload that takes some minutes to complete.
The inconsistent peer Vlan message usually means the following:
on a trunk link all non native Vlans when using PVST+ or Rapid PVST use the Cisco proprietary version of BPDU.
This BPDU has an internal field that tells to what Vlan-id the BPDU is related to (the instance or vlan-id)
On receiving a BPDU the consistency check is performed by comparing the 802.1Q vlan-id in the 802.1Q header of the BPDU with the value in the internal field if the two values are different an inconsistency is declared and the port is placed in inconsistent state for the offending vlan(s).
In your case after 15 seconds all inconsistencies are cleared and the port is recoved for all three VLANs.
If this happens again or you have ports stucked in inconsistent state you need to look at possible errors in cabling that may join two Vlans for example 100 and 80.
With the modern approach to create bridge ID the priority is always N*4096 + Vlan-id# so if you make to communicate by error two Vlans connecting two access ports one in vlan 100 and one in vlan 80 you are doing the following:
you are joining two Vlans / broadcast domains
the BPDU of the lowered number Vlan might win over the one of the higher numbered Vlan and this BPDU will try to go to the core upstream.
Spanning-tree bpu guard on access port and spanning-tree guard root on distribution switches can help.