09-25-2018 08:56 AM - edited 03-08-2019 04:14 PM
I am banging my head against an issue that has been ongoing on my school network. Every day (or if i'm lucky...every other day) i am seeing a switch closet go down. Its not the same one each time, and its at random times. All the closets are in a hub topography connected back to a core switch with 10 gig multimode fiber trunk links. At first i was focusing on equipment maybe going bad.....but i've swapped out tranceivers, patch cables and entire switches and the issue persists.
Some times when looking at the switch logs, i see the Ten gig port actually go DOWN.....other times i see nothing...but no traffic is going over the port. Im not seeing any spanning tree or BPDU guard errors either. Sometimes i can move the transceiver to another slot and traffic will flow again.....sometimes i need to reboot the switch (a reboot always gets the traffic flowing again, btw)
I could give specifics of equipment and configurations.....but i thought i'd reach out and ask if anyone has any ideas as to the kind of things that can cause a trunk interface just stop passing traffic....and have a reboot fix it.
Thanks in advance.
09-25-2018 10:12 AM
Hello,
there are a lot of bugs that could cause this. Which switches and IOS are you using ?
09-25-2018 10:34 AM
Core Switch for one building is 4510 with IOS for the supervisors 3.06.00.E
Closet switches are 2960X with IOS's of 15.2(4)E6 and 15.0(2)EX5
Core Switch (and actually only switch) for second building is 3850 with IOS 03.06.03E
Core Switch for third building is 3850 with IOS 03.06.03E
Closet switches are 2960X with IOS's 15.2(4)E6
09-25-2018 11:02 AM
Hello,
I checked for bugs, none found for your IOS and platforms. Can you post a schematic drawing of how every device in your network is connected ? Although you say no spanning tree event is being logged, it might be worth checking the root for every VLAN...
09-25-2018 11:07 AM
I do have some other older switches chained off the closet 2960X's.....because we needed more ports.
I'll get those to you shortly.
09-25-2018 11:10 AM
Also....what do you mean by checking the root for each VLAN?
09-25-2018 11:12 AM
When you add a switch, the switch might inadvertently become the root for one or more VLANs, that is why it is a good idea to set the root switch manually. Do you have something like a core switch, a switch where all other switches are connecting to ?
This could then be a candidate root switch...
09-25-2018 11:28 AM
yes....the 4510 is the actual core switch for the entire network and the branch schools. So I should manually set that to be the root switch for all of my VLANS?
What are the commands to check where all the vlan roots are?
09-25-2018 12:05 PM
Hello,
post the output of 'show spanning-tree' from the 4510...
09-25-2018 03:40 PM
MST0
Spanning tree enabled protocol mstp
Root ID Priority 24576
Address b0aa.7722.2280
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 24576 (priority 24576 sys-id-ext 0)
Address b0aa.7722.2280
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/2 Desg FWD 200000 128.2 Shr Edge
Gi1/3 Desg FWD 200000 128.3 P2p Edge
Gi1/4 Desg FWD 200000 128.4 P2p Edge
Gi1/5 Desg FWD 200000 128.5 P2p Edge
Gi1/6 Desg FWD 20000 128.6 P2p Edge
Gi1/7 Desg FWD 200000 128.7 P2p Edge
Gi1/9 Desg FWD 20000 128.9 P2p Edge
Gi1/10 Desg FWD 2000000 128.10 P2p Edge
Gi1/11 Desg FWD 200000 128.11 P2p Edge
Gi1/12 Desg FWD 20000 128.12 P2p Edge
Gi1/13 Desg FWD 20000 128.13 P2p Edge
Gi1/14 Desg FWD 20000 128.14 P2p Edge
Gi1/15 Desg FWD 20000 128.15 P2p Edge
Gi1/16 Desg FWD 20000 128.16 P2p Edge
Gi1/17 Desg FWD 20000 128.17 P2p Edge
Gi1/18 Desg FWD 200000 128.18 P2p Edge
Gi1/19 Desg FWD 20000 128.19 P2p Edge
Gi1/20 Desg FWD 20000 128.20 P2p Edge
Gi1/21 Desg FWD 20000 128.21 P2p Edge
Gi1/22 Desg FWD 20000 128.22 P2p Edge
Gi1/39 Desg FWD 20000 128.39 P2p Edge
Gi1/40 Desg FWD 20000 128.40 P2p Edge
Gi1/45 Desg FWD 20000 128.45 P2p Edge
Gi1/46 Desg FWD 20000 128.46 P2p Edge
Gi2/1 Desg FWD 20000 128.129 P2p
Gi2/2 Desg FWD 20000 128.130 P2p
Gi2/7 Desg FWD 20000 128.135 P2p
Te3/1 Desg FWD 2000 128.257 P2p
Te3/2 Desg FWD 2000 128.258 P2p
Te3/3 Desg FWD 2000 128.259 P2p
Te3/4 Desg FWD 2000 128.260 P2p
Te3/5 Desg FWD 2000 128.261 P2p
Te3/6 Desg FWD 2000 128.262 P2p
Gi4/1 Desg FWD 20000 128.385 P2p Edge
Gi4/2 Desg FWD 20000 128.386 P2p Edge
Gi4/3 Desg FWD 20000 128.387 P2p
Gi4/4 Desg FWD 20000 128.388 P2p Edge
Gi4/5 Desg FWD 20000 128.389 P2p
Gi4/6 Desg FWD 20000 128.390 P2p
Gi4/7 Desg FWD 20000 128.391 P2p
Gi4/8 Desg FWD 20000 128.392 P2p
Gi4/9 Desg FWD 20000 128.393 P2p
Gi4/10 Desg FWD 20000 128.394 P2p
Gi4/11 Desg FWD 20000 128.395 P2p
Gi4/12 Desg FWD 20000 128.396 P2p
Gi4/13 Desg FWD 200000 128.397 P2p Edge
Gi4/14 Desg FWD 200000 128.398 P2p Edge
Gi4/15 Desg FWD 200000 128.399 P2p Edge
Gi4/16 Desg FWD 200000 128.400 P2p Edge
Gi4/17 Desg FWD 200000 128.401 P2p Edge
Gi4/23 Desg FWD 2000000 128.407 Shr Edge
Gi4/24 Desg FWD 20000 128.408 P2p Edge
Te5/2 Desg FWD 2000 128.514 P2p
Te5/3 Desg FWD 2000 128.515 P2p
Te5/4 Desg FWD 2000 128.516 P2p
Te6/2 Desg FWD 2000 128.642 P2p
Te6/3 Desg FWD 2000 128.643 P2p
Te6/4 Desg FWD 2000 128.644 P2p
Gi10/1 Desg FWD 20000 128.1153 P2p Edge
Gi10/2 Desg FWD 200000 128.1154 P2p Edge
Gi10/3 Desg FWD 200000 128.1155 P2p Edge
Gi10/4 Desg FWD 200000 128.1156 P2p Edge
Gi10/5 Desg FWD 200000 128.1157 P2p Edge
Gi10/6 Desg FWD 200000 128.1158 P2p Edge
Gi10/7 Desg FWD 200000 128.1159 P2p Edge
Gi10/8 Desg FWD 200000 128.1160 P2p Edge
Gi10/11 Desg FWD 200000 128.1163 P2p Edge
Gi10/12 Desg FWD 200000 128.1164 P2p Edge
Gi10/13 Desg FWD 200000 128.1165 P2p Edge
Gi10/14 Desg FWD 200000 128.1166 P2p Edge
Gi10/15 Desg FWD 200000 128.1167 P2p Edge
Gi10/16 Desg FWD 200000 128.1168 P2p Edge
Gi10/17 Desg FWD 200000 128.1169 P2p Edge
Gi10/18 Desg FWD 200000 128.1170 P2p Edge
Gi10/19 Desg FWD 200000 128.1171 P2p Edge
Gi10/20 Desg FWD 200000 128.1172 P2p Edge
Gi10/21 Desg FWD 200000 128.1173 P2p Edge
Gi10/22 Desg FWD 200000 128.1174 P2p Edge
Gi10/25 Desg FWD 200000 128.1177 P2p Edge
Gi10/26 Desg FWD 200000 128.1178 P2p Edge
Gi10/27 Desg FWD 200000 128.1179 Shr Edge
Gi10/28 Desg FWD 200000 128.1180 Shr Edge
Gi10/32 Desg FWD 200000 128.1184 P2p Edge
Gi10/36 Desg FWD 2000000 128.1188 P2p Edge
Gi10/45 Desg FWD 200000 128.1197 P2p Edge
Po1 Desg FWD 1000 128.1281 P2p
Po10 Desg FWD 2500 128.1290 P2p
Po11 Desg FWD 2500 128.1291 P2p
MST1
Spanning tree enabled protocol mstp
Root ID Priority 24577
Address b0aa.7722.2280
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 24577 (priority 24576 sys-id-ext 1)
Address b0aa.7722.2280
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/2 Desg FWD 200000 128.2 Shr Edge
Gi1/3 Desg FWD 200000 128.3 P2p Edge
Gi1/4 Desg FWD 200000 128.4 P2p Edge
Gi1/5 Desg FWD 200000 128.5 P2p Edge
Gi1/6 Desg FWD 20000 128.6 P2p Edge
Gi1/7 Desg FWD 200000 128.7 P2p Edge
Gi1/9 Desg FWD 20000 128.9 P2p Edge
Gi1/10 Desg FWD 2000000 128.10 P2p Edge
Gi1/11 Desg FWD 200000 128.11 P2p Edge
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/12 Desg FWD 20000 128.12 P2p Edge
Gi1/13 Desg FWD 20000 128.13 P2p Edge
Gi1/14 Desg FWD 20000 128.14 P2p Edge
Gi1/15 Desg FWD 20000 128.15 P2p Edge
Gi1/16 Desg FWD 20000 128.16 P2p Edge
Gi1/17 Desg FWD 20000 128.17 P2p Edge
Gi1/18 Desg FWD 200000 128.18 P2p Edge
Gi1/19 Desg FWD 20000 128.19 P2p Edge
Gi1/20 Desg FWD 20000 128.20 P2p Edge
Gi1/21 Desg FWD 20000 128.21 P2p Edge
Gi1/22 Desg FWD 20000 128.22 P2p Edge
Gi1/39 Desg FWD 20000 128.39 P2p Edge
Gi1/40 Desg FWD 20000 128.40 P2p Edge
Gi1/45 Desg FWD 20000 128.45 P2p Edge
Gi1/46 Desg FWD 20000 128.46 P2p Edge
Gi2/1 Desg FWD 20000 128.129 P2p
Gi2/2 Desg FWD 20000 128.130 P2p
Gi2/7 Desg FWD 20000 128.135 P2p
Te3/1 Desg FWD 2000 128.257 P2p
Te3/2 Desg FWD 2000 128.258 P2p
Te3/3 Desg FWD 2000 128.259 P2p
Te3/4 Desg FWD 2000 128.260 P2p
Te3/5 Desg FWD 2000 128.261 P2p
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te3/6 Desg FWD 2000 128.262 P2p
Gi4/1 Desg FWD 20000 128.385 P2p Edge
Gi4/2 Desg FWD 20000 128.386 P2p Edge
Gi4/3 Desg FWD 20000 128.387 P2p
Gi4/4 Desg FWD 20000 128.388 P2p Edge
Gi4/5 Desg FWD 20000 128.389 P2p
Gi4/6 Desg FWD 20000 128.390 P2p
Gi4/7 Desg FWD 20000 128.391 P2p
Gi4/8 Desg FWD 20000 128.392 P2p
Gi4/9 Desg FWD 20000 128.393 P2p
Gi4/10 Desg FWD 20000 128.394 P2p
Gi4/11 Desg FWD 20000 128.395 P2p
Gi4/12 Desg FWD 20000 128.396 P2p
Gi4/13 Desg FWD 200000 128.397 P2p Edge
Gi4/14 Desg FWD 200000 128.398 P2p Edge
Gi4/15 Desg FWD 200000 128.399 P2p Edge
Gi4/16 Desg FWD 200000 128.400 P2p Edge
Gi4/17 Desg FWD 200000 128.401 P2p Edge
Gi4/23 Desg FWD 2000000 128.407 Shr Edge
Gi4/24 Desg FWD 20000 128.408 P2p Edge
Te5/2 Desg FWD 2000 128.514 P2p
Te5/3 Desg FWD 2000 128.515 P2p
Te5/4 Desg FWD 2000 128.516 P2p
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te6/2 Desg FWD 2000 128.642 P2p
Te6/3 Desg FWD 2000 128.643 P2p
Te6/4 Desg FWD 2000 128.644 P2p
Gi10/1 Desg FWD 20000 128.1153 P2p Edge
Gi10/2 Desg FWD 200000 128.1154 P2p Edge
Gi10/3 Desg FWD 200000 128.1155 P2p Edge
Gi10/4 Desg FWD 200000 128.1156 P2p Edge
Gi10/5 Desg FWD 200000 128.1157 P2p Edge
Gi10/6 Desg FWD 200000 128.1158 P2p Edge
Gi10/7 Desg FWD 200000 128.1159 P2p Edge
Gi10/8 Desg FWD 200000 128.1160 P2p Edge
Gi10/11 Desg FWD 200000 128.1163 P2p Edge
Gi10/12 Desg FWD 200000 128.1164 P2p Edge
Gi10/13 Desg FWD 200000 128.1165 P2p Edge
Gi10/14 Desg FWD 200000 128.1166 P2p Edge
Gi10/15 Desg FWD 200000 128.1167 P2p Edge
Gi10/16 Desg FWD 200000 128.1168 P2p Edge
Gi10/17 Desg FWD 200000 128.1169 P2p Edge
Gi10/18 Desg FWD 200000 128.1170 P2p Edge
Gi10/19 Desg FWD 200000 128.1171 P2p Edge
Gi10/20 Desg FWD 200000 128.1172 P2p Edge
Gi10/21 Desg FWD 200000 128.1173 P2p Edge
Gi10/22 Desg FWD 200000 128.1174 P2p Edge
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi10/25 Desg FWD 200000 128.1177 P2p Edge
Gi10/26 Desg FWD 200000 128.1178 P2p Edge
Gi10/27 Desg FWD 200000 128.1179 Shr Edge
Gi10/28 Desg FWD 200000 128.1180 Shr Edge
Gi10/32 Desg FWD 200000 128.1184 P2p Edge
Gi10/36 Desg FWD 2000000 128.1188 P2p Edge
Gi10/45 Desg FWD 200000 128.1197 P2p Edge
Po1 Desg FWD 1000 128.1281 P2p
Po10 Desg FWD 2500 128.1290 P2p
Po11 Desg FWD 2500 128.1291 P2p
09-25-2018 11:24 AM
the following devices are chained off of either a 3850 or a 2960X
3560-8PC---IOS 12.2(55)SE7
2960CX---IOS 15.2(3)E1
another C2960X that isn't in a closet but in a classroom---IOS 15.0(2a)EX5
2960G----IOS 15.0(2)SE8
2960S----IOS 15.0(2)EX5
3560X----IOS 15.2(3)E1
2960CX---IOS 15.2(3)E
2960CX---IOS 15.2(3)E2
2960----IOS 15.0(2)SE8
2960CX---IOS 15.2(4)E2
09-25-2018 11:49 PM
10-02-2018 04:22 AM
have successfully upgraded both the 4510 and all the 2960X......so far so good.
10-08-2018 08:08 AM
Things had been good for over a week.....no failed trunk ports. However, we have had the problem recur.
Any thoughts on what to try next?
Maybe take the Switch IOS's up higher?
10-08-2018 02:13 PM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide