cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1703
Views
0
Helpful
23
Replies

Ten Gig Trunk Ports that stop passing traffic

darinheilman
Level 1
Level 1

I am banging my head against an issue that has been ongoing on my school network.  Every day (or if i'm lucky...every other day) i am seeing a switch closet go down.  Its not the same one each time, and its at random times.  All the closets are in a hub topography connected back to a core switch with 10 gig multimode fiber trunk links.  At first i was focusing on equipment maybe going bad.....but i've swapped out tranceivers, patch cables and entire switches and the issue persists.

 

Some times when looking at the switch logs, i see the Ten gig port actually go DOWN.....other times i see nothing...but no traffic is going over the port.  Im not seeing any spanning tree or BPDU guard errors either.   Sometimes i can move the transceiver to another slot and traffic will flow again.....sometimes i need to reboot the switch (a reboot always gets the traffic flowing again, btw)

 

I could give specifics of equipment and configurations.....but i thought i'd reach out and ask if anyone has any ideas as to the kind of things that can cause a trunk interface just stop passing traffic....and have a reboot fix it.

 

Thanks in advance.

 

 

23 Replies 23

Hello,

 

there are a lot of bugs that could cause this. Which switches and IOS are you using ?

Core Switch for one building is 4510 with IOS for the supervisors 3.06.00.E

Closet switches are 2960X with IOS's of 15.2(4)E6 and 15.0(2)EX5 

 

Core Switch (and actually only switch) for second building is 3850 with IOS 03.06.03E

 

Core Switch for third building is 3850 with IOS 03.06.03E

Closet switches are 2960X with IOS's 15.2(4)E6

Hello,

 

I checked for bugs, none found for your IOS and platforms. Can you post a schematic drawing of how every device in your network is connected ? Although you say no spanning tree event is being logged, it might be worth checking the root for every VLAN...

I do have some other older switches chained off the closet 2960X's.....because we needed more ports.

 

I'll get those to you shortly.

 

 

Also....what do you mean by checking the root for each VLAN?

 

 

When you add a switch, the switch might inadvertently become the root for one or more VLANs, that is why it is a good idea to set the root switch manually. Do you have something like a core switch, a switch where all other switches are connecting to ?

This could then be a candidate root switch...

yes....the 4510 is the actual core switch for the entire network and the branch schools.  So I should manually set that to be the root switch for all of my VLANS?

 

What are the commands to check where all the vlan roots are?

 

 

Hello,

 

post the output of 'show spanning-tree' from the 4510...

MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    24576
             Address     b0aa.7722.2280
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    24576  (priority 24576 sys-id-ext 0)
             Address     b0aa.7722.2280
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/2               Desg FWD 200000    128.2    Shr Edge
Gi1/3               Desg FWD 200000    128.3    P2p Edge
Gi1/4               Desg FWD 200000    128.4    P2p Edge
Gi1/5               Desg FWD 200000    128.5    P2p Edge
Gi1/6               Desg FWD 20000     128.6    P2p Edge
Gi1/7               Desg FWD 200000    128.7    P2p Edge
Gi1/9               Desg FWD 20000     128.9    P2p Edge
Gi1/10              Desg FWD 2000000   128.10   P2p Edge
Gi1/11              Desg FWD 200000    128.11   P2p Edge
Gi1/12              Desg FWD 20000     128.12   P2p Edge
Gi1/13              Desg FWD 20000     128.13   P2p Edge
Gi1/14              Desg FWD 20000     128.14   P2p Edge
Gi1/15              Desg FWD 20000     128.15   P2p Edge
Gi1/16              Desg FWD 20000     128.16   P2p Edge
Gi1/17              Desg FWD 20000     128.17   P2p Edge
Gi1/18              Desg FWD 200000    128.18   P2p Edge
Gi1/19              Desg FWD 20000     128.19   P2p Edge
Gi1/20              Desg FWD 20000     128.20   P2p Edge
Gi1/21              Desg FWD 20000     128.21   P2p Edge
Gi1/22              Desg FWD 20000     128.22   P2p Edge
Gi1/39              Desg FWD 20000     128.39   P2p Edge
Gi1/40              Desg FWD 20000     128.40   P2p Edge
Gi1/45              Desg FWD 20000     128.45   P2p Edge
Gi1/46              Desg FWD 20000     128.46   P2p Edge
Gi2/1               Desg FWD 20000     128.129  P2p
Gi2/2               Desg FWD 20000     128.130  P2p
Gi2/7               Desg FWD 20000     128.135  P2p
Te3/1               Desg FWD 2000      128.257  P2p
Te3/2               Desg FWD 2000      128.258  P2p
Te3/3               Desg FWD 2000      128.259  P2p
Te3/4               Desg FWD 2000      128.260  P2p
Te3/5               Desg FWD 2000      128.261  P2p
Te3/6               Desg FWD 2000      128.262  P2p
Gi4/1               Desg FWD 20000     128.385  P2p Edge
Gi4/2               Desg FWD 20000     128.386  P2p Edge
Gi4/3               Desg FWD 20000     128.387  P2p
Gi4/4               Desg FWD 20000     128.388  P2p Edge
Gi4/5               Desg FWD 20000     128.389  P2p
Gi4/6               Desg FWD 20000     128.390  P2p
Gi4/7               Desg FWD 20000     128.391  P2p
Gi4/8               Desg FWD 20000     128.392  P2p
Gi4/9               Desg FWD 20000     128.393  P2p
Gi4/10              Desg FWD 20000     128.394  P2p
Gi4/11              Desg FWD 20000     128.395  P2p
Gi4/12              Desg FWD 20000     128.396  P2p
Gi4/13              Desg FWD 200000    128.397  P2p Edge
Gi4/14              Desg FWD 200000    128.398  P2p Edge
Gi4/15              Desg FWD 200000    128.399  P2p Edge
Gi4/16              Desg FWD 200000    128.400  P2p Edge
Gi4/17              Desg FWD 200000    128.401  P2p Edge
Gi4/23              Desg FWD 2000000   128.407  Shr Edge
Gi4/24              Desg FWD 20000     128.408  P2p Edge
Te5/2               Desg FWD 2000      128.514  P2p
Te5/3               Desg FWD 2000      128.515  P2p
Te5/4               Desg FWD 2000      128.516  P2p
Te6/2               Desg FWD 2000      128.642  P2p
Te6/3               Desg FWD 2000      128.643  P2p
Te6/4               Desg FWD 2000      128.644  P2p
Gi10/1              Desg FWD 20000     128.1153 P2p Edge
Gi10/2              Desg FWD 200000    128.1154 P2p Edge
Gi10/3              Desg FWD 200000    128.1155 P2p Edge
Gi10/4              Desg FWD 200000    128.1156 P2p Edge
Gi10/5              Desg FWD 200000    128.1157 P2p Edge
Gi10/6              Desg FWD 200000    128.1158 P2p Edge
Gi10/7              Desg FWD 200000    128.1159 P2p Edge
Gi10/8              Desg FWD 200000    128.1160 P2p Edge
Gi10/11             Desg FWD 200000    128.1163 P2p Edge
Gi10/12             Desg FWD 200000    128.1164 P2p Edge
Gi10/13             Desg FWD 200000    128.1165 P2p Edge
Gi10/14             Desg FWD 200000    128.1166 P2p Edge
Gi10/15             Desg FWD 200000    128.1167 P2p Edge
Gi10/16             Desg FWD 200000    128.1168 P2p Edge
Gi10/17             Desg FWD 200000    128.1169 P2p Edge
Gi10/18             Desg FWD 200000    128.1170 P2p Edge
Gi10/19             Desg FWD 200000    128.1171 P2p Edge
Gi10/20             Desg FWD 200000    128.1172 P2p Edge
Gi10/21             Desg FWD 200000    128.1173 P2p Edge
Gi10/22             Desg FWD 200000    128.1174 P2p Edge
Gi10/25             Desg FWD 200000    128.1177 P2p Edge
Gi10/26             Desg FWD 200000    128.1178 P2p Edge
Gi10/27             Desg FWD 200000    128.1179 Shr Edge
Gi10/28             Desg FWD 200000    128.1180 Shr Edge
Gi10/32             Desg FWD 200000    128.1184 P2p Edge
Gi10/36             Desg FWD 2000000   128.1188 P2p Edge
Gi10/45             Desg FWD 200000    128.1197 P2p Edge
Po1                 Desg FWD 1000      128.1281 P2p
Po10                Desg FWD 2500      128.1290 P2p
Po11                Desg FWD 2500      128.1291 P2p


MST1
  Spanning tree enabled protocol mstp
  Root ID    Priority    24577
             Address     b0aa.7722.2280
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    24577  (priority 24576 sys-id-ext 1)
             Address     b0aa.7722.2280
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/2               Desg FWD 200000    128.2    Shr Edge
Gi1/3               Desg FWD 200000    128.3    P2p Edge
Gi1/4               Desg FWD 200000    128.4    P2p Edge
Gi1/5               Desg FWD 200000    128.5    P2p Edge
Gi1/6               Desg FWD 20000     128.6    P2p Edge
Gi1/7               Desg FWD 200000    128.7    P2p Edge
Gi1/9               Desg FWD 20000     128.9    P2p Edge
Gi1/10              Desg FWD 2000000   128.10   P2p Edge
Gi1/11              Desg FWD 200000    128.11   P2p Edge

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------

Gi1/12              Desg FWD 20000     128.12   P2p Edge
Gi1/13              Desg FWD 20000     128.13   P2p Edge
Gi1/14              Desg FWD 20000     128.14   P2p Edge
Gi1/15              Desg FWD 20000     128.15   P2p Edge
Gi1/16              Desg FWD 20000     128.16   P2p Edge
Gi1/17              Desg FWD 20000     128.17   P2p Edge
Gi1/18              Desg FWD 200000    128.18   P2p Edge
Gi1/19              Desg FWD 20000     128.19   P2p Edge
Gi1/20              Desg FWD 20000     128.20   P2p Edge
Gi1/21              Desg FWD 20000     128.21   P2p Edge
Gi1/22              Desg FWD 20000     128.22   P2p Edge
Gi1/39              Desg FWD 20000     128.39   P2p Edge
Gi1/40              Desg FWD 20000     128.40   P2p Edge
Gi1/45              Desg FWD 20000     128.45   P2p Edge
Gi1/46              Desg FWD 20000     128.46   P2p Edge
Gi2/1               Desg FWD 20000     128.129  P2p
Gi2/2               Desg FWD 20000     128.130  P2p
Gi2/7               Desg FWD 20000     128.135  P2p
Te3/1               Desg FWD 2000      128.257  P2p
Te3/2               Desg FWD 2000      128.258  P2p
Te3/3               Desg FWD 2000      128.259  P2p
Te3/4               Desg FWD 2000      128.260  P2p
Te3/5               Desg FWD 2000      128.261  P2p

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------

Te3/6               Desg FWD 2000      128.262  P2p
Gi4/1               Desg FWD 20000     128.385  P2p Edge
Gi4/2               Desg FWD 20000     128.386  P2p Edge
Gi4/3               Desg FWD 20000     128.387  P2p
Gi4/4               Desg FWD 20000     128.388  P2p Edge
Gi4/5               Desg FWD 20000     128.389  P2p
Gi4/6               Desg FWD 20000     128.390  P2p
Gi4/7               Desg FWD 20000     128.391  P2p
Gi4/8               Desg FWD 20000     128.392  P2p
Gi4/9               Desg FWD 20000     128.393  P2p
Gi4/10              Desg FWD 20000     128.394  P2p
Gi4/11              Desg FWD 20000     128.395  P2p
Gi4/12              Desg FWD 20000     128.396  P2p
Gi4/13              Desg FWD 200000    128.397  P2p Edge
Gi4/14              Desg FWD 200000    128.398  P2p Edge
Gi4/15              Desg FWD 200000    128.399  P2p Edge
Gi4/16              Desg FWD 200000    128.400  P2p Edge
Gi4/17              Desg FWD 200000    128.401  P2p Edge
Gi4/23              Desg FWD 2000000   128.407  Shr Edge
Gi4/24              Desg FWD 20000     128.408  P2p Edge
Te5/2               Desg FWD 2000      128.514  P2p
Te5/3               Desg FWD 2000      128.515  P2p
Te5/4               Desg FWD 2000      128.516  P2p

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------

Te6/2               Desg FWD 2000      128.642  P2p
Te6/3               Desg FWD 2000      128.643  P2p
Te6/4               Desg FWD 2000      128.644  P2p
Gi10/1              Desg FWD 20000     128.1153 P2p Edge
Gi10/2              Desg FWD 200000    128.1154 P2p Edge
Gi10/3              Desg FWD 200000    128.1155 P2p Edge
Gi10/4              Desg FWD 200000    128.1156 P2p Edge
Gi10/5              Desg FWD 200000    128.1157 P2p Edge
Gi10/6              Desg FWD 200000    128.1158 P2p Edge
Gi10/7              Desg FWD 200000    128.1159 P2p Edge
Gi10/8              Desg FWD 200000    128.1160 P2p Edge
Gi10/11             Desg FWD 200000    128.1163 P2p Edge
Gi10/12             Desg FWD 200000    128.1164 P2p Edge
Gi10/13             Desg FWD 200000    128.1165 P2p Edge
Gi10/14             Desg FWD 200000    128.1166 P2p Edge
Gi10/15             Desg FWD 200000    128.1167 P2p Edge
Gi10/16             Desg FWD 200000    128.1168 P2p Edge
Gi10/17             Desg FWD 200000    128.1169 P2p Edge
Gi10/18             Desg FWD 200000    128.1170 P2p Edge
Gi10/19             Desg FWD 200000    128.1171 P2p Edge
Gi10/20             Desg FWD 200000    128.1172 P2p Edge
Gi10/21             Desg FWD 200000    128.1173 P2p Edge
Gi10/22             Desg FWD 200000    128.1174 P2p Edge

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------

Gi10/25             Desg FWD 200000    128.1177 P2p Edge
Gi10/26             Desg FWD 200000    128.1178 P2p Edge
Gi10/27             Desg FWD 200000    128.1179 Shr Edge
Gi10/28             Desg FWD 200000    128.1180 Shr Edge
Gi10/32             Desg FWD 200000    128.1184 P2p Edge
Gi10/36             Desg FWD 2000000   128.1188 P2p Edge
Gi10/45             Desg FWD 200000    128.1197 P2p Edge
Po1                 Desg FWD 1000      128.1281 P2p
Po10                Desg FWD 2500      128.1290 P2p
Po11                Desg FWD 2500      128.1291 P2p

the following devices are chained off of either a 3850 or a 2960X

 

3560-8PC---IOS 12.2(55)SE7

2960CX---IOS 15.2(3)E1

another C2960X that isn't in a closet but in a classroom---IOS 15.0(2a)EX5

2960G----IOS 15.0(2)SE8

2960S----IOS 15.0(2)EX5

3560X----IOS 15.2(3)E1

2960CX---IOS 15.2(3)E

2960CX---IOS 15.2(3)E2

2960----IOS 15.0(2)SE8 

2960CX---IOS 15.2(4)E2

 

 

You can stop banging your head.
1. The 4500 is running 3.6.0. Last digit is a "0". This means the first version.
2. Some of the 2960X are running "EX" version. These are Interim versions.
My recommendation is to upgrade the 4500 and the 2960X and see if it improve things.

have successfully upgraded both the 4510 and all the 2960X......so far so good.

 

 

Things had been good for over a week.....no failed trunk ports.  However, we have had the problem recur.

 

Any thoughts on what to try next?

 

Maybe take the Switch IOS's up higher?

 

 

IF the link dropped, then post the complete output to the command "sh interface <PORT>" from both ends.
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco