MAC Flapping on 2960-24TC-L - Netlab POD

swillet41 · ‎09-23-2015

Hello,

I am exp MAC Flapping on a 2960-24-TC-L switch between multiple ports (flapping ports move as I shutdown interfaces). The only way to stop the flapping is to shutdown all unused ports.

This is a Netlab POD setup for student labs. The flap does not occur until I use "switchport trunk native vlan 99" on my trunk ports between two switches. If I set the trunk native vlan back to "switchport trunk native vlan 1", the flapping stops. Below is a screenshot of the problem.

Does anyone have a resolution for this? I am not using Etherchannel.

Thanks,

Scott

S1#
!
! Last configuration change at 00:15:19 UTC Mon Mar 1 1993
!
version 15.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname S1
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$RskD$PbIZIEaed1HdOSL7I2rVj.
!
username admin privilege 15 secret 5 $1$XT2W$PG48x0DN4rdYeNlCUxFRV0
no aaa new-model
system mtu routing 1500
!
!
--More--
*Mar 1 00:15:19.206: %SYS-5-CONFIG_I: Configured from console by console
no ip domain-lookup
ip domain-name ccnasecurity.com
!
!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
spanning-tree vlan 1 priority 0
!
vlan internal allocation policy ascending
!
ip ssh time-out 90
ip ssh authentication-retries 2
!
!
!
!
!
interface FastEthernet0/1
switchport trunk native vlan 99
switchport mode trunk
!
interface FastEthernet0/2
!
interface FastEthernet0/3
!
interface FastEthernet0/4
!
interface FastEthernet0/5
!
interface FastEthernet0/6
!
interface FastEthernet0/7
!
interface FastEthernet0/8
!
interface FastEthernet0/9
!
interface FastEthernet0/10
!
interface FastEthernet0/11
!
interface FastEthernet0/12
!
interface FastEthernet0/13
!
interface FastEthernet0/14
!
interface FastEthernet0/15
!
interface FastEthernet0/16
!
interface FastEthernet0/17
!
interface FastEthernet0/18
!
interface FastEthernet0/19
!
interface FastEthernet0/20
!
interface FastEthernet0/21
!
interface FastEthernet0/22
!
interface FastEthernet0/23
!
interface FastEthernet0/24
!
interface GigabitEthernet0/1
!
interface GigabitEthernet0/2
!
interface Vlan1
ip address 192.x.x.x 255.255.255.0
!
no ip http server
no ip http secure-server
!
!
line con 0
exec-timeout 120 0
password 7 02050D4808090C2E425E080A16
logging synchronous

login
line vty 0 4
exec-timeout 120 0
privilege level 15
password 7 070C285F4D060F110E020A1F17
login local
transport input ssh
line vty 5 15
login
transport input none
!
end

____________________________________________

S2#sh run
Building configuration...

Current configuration : 1572 bytes
!
! Last configuration change at 00:10:38 UTC Mon Mar 1 1993
!
version 15.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname S2
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$2Bh6$IfyNy.dkybQmHGvmLPS/v1
!
no aaa new-model
system mtu routing 1500
!
!
no ip domain-lookup

!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
!
!
!
interface FastEthernet0/1
switchport trunk native vlan 99
switchport mode trunk
!
interface FastEthernet0/2
!
interface FastEthernet0/3
!
interface FastEthernet0/4
!
interface FastEthernet0/5
!
interface FastEthernet0/6
!
interface FastEthernet0/7
!
interface FastEthernet0/8
!
interface FastEthernet0/9
!
interface FastEthernet0/10
!
interface FastEthernet0/11
!
interface FastEthernet0/12
!
interface FastEthernet0/13
!
interface FastEthernet0/14
!
interface FastEthernet0/15
!
interface FastEthernet0/16
!
interface FastEthernet0/17
!
interface FastEthernet0/18
!
interface FastEthernet0/19
!
interface FastEthernet0/20
!
interface FastEthernet0/21
!
interface FastEthernet0/22
!
interface FastEthernet0/23
!
interface FastEthernet0/24
!
interface GigabitEthernet0/1
!
interface GigabitEthernet0/2
!
interface Vlan1
ip address 192.x.x.x 255.255.255.0
!
no ip http server
no ip http secure-server
!
!
line con 0
exec-timeout 120 0
password (#@)(($%
logging synchronous
login
line vty 0 4
exec-timeout 120 0
password (#@)(($%
login
line vty 5 15
login
!
end

devils_advocate · ‎09-23-2015

What connects to the following ports:

Fa0/1
Fa0/2
Fa0/6

?

We really need to see the full config to help further.

MAC flapping is almost always to do with a configuration issue.

swillet41 · ‎09-23-2015

Hi,

I posted the run config files in my main post for you.

fa0/1-Switch 2

fa0/2-Switch2

fa0/6-PC-A

Scott

devils_advocate · ‎09-23-2015

Hi Scott

I don't see Fa0/2 connected to Switch 2 in the diagram?

The log in the original post references fa0/1 and fa0/2 so it must be connected to something?

The config below shows fa0/1 is configured as a Trunk but fa0/2 does't seem to be?

interface FastEthernet0/1
switchport trunk native vlan 99
switchport mode trunk
!
interface FastEthernet0/2
!

Can you post the results of the following command for me?

#show int trunk

swillet41 · ‎09-23-2015

Because this is a "lab", there is not any config on fa0/2, but it is actually physically wired to S2 via fa0/2. Its confusing due to the lab setup. See the topology above with the Control switch. That will give u an idea of how its physically wired.

S1#sh int trunk

Port Mode Encapsulation Status Native vlan
Fa0/1 on 802.1q trunking 1

Port Vlans allowed on trunk
Fa0/1 1-4094

Port Vlans allowed and active in management domain
Fa0/1 1

Port Vlans in spanning tree forwarding state and not pruned
Fa0/1 1
S1#

devils_advocate · ‎09-23-2015

So just to clarify.

Switch 1 is Connected to Switch 2 via two physical links.

One is a Trunk port (Fa0/1)
One is an Access Port in VLAN 1 (Fa0/2)

Correct?

Spanning tree should be blocking one of these links for Vlan 1 but I wonder if because you have One Trunk Line and One Access Link, whether this is confusing the Issue.

If you are going to connect two switches together using 2x layer 2 links, they should be configured the same, i.e two Trunk ports or two access ports.

Configure Fa0/2 on both switches to be the same as Fa0/1.
You should then find that Spanning Tree kicks in and blocks one of the links.

swillet41 · ‎09-23-2015

You are correct sir. I config'd both fa0/1-2 port on bith switches for trunking and still have the switching loop.

The vendor wrote me and we are going to pwr every device down to reset the vlan assignment the the PODs are receiving from the Netlab server. I will post my results from this next week and we can continue troubleshooting - Thanks a bunch!

Scott

LJ Gabrillo · ‎09-23-2015

Okay like this if its okay can you give us the current running config you have? Also give us a "show version" and "show processes cpu sorted" and "show processes cpu history" log

PS: Please attach them as *.txt files so the posts wont be long :)

lets start from There.

Additionally
1. Checking your flap sources, may we know what devices are connected there?
2. Can we get an overview on your network? If there are switches connected on those ports, you might have an incoming loop issue. we need to resolve this fast since this will cause your CPU to be quite high

swillet41 · ‎09-23-2015

LJ,

Running-config and topology is in the main post

I uploaded the version and the cpu sorted/history outputs for you to view.

ports:

fa0/1-Switch 2

fa0/2-Switch2

fa0/6-PC-A

Thanks,

Scott

LJ Gabrillo · ‎09-23-2015

Dude, based on the diagram you provided(the one with the 3 switches with loop topology), you have multiple ports connected to and fro switch hence you need to properly setup your ports since you are using STP

STP is required for your topology to avoid looping in your network

I'd like to point out that:
1. Look at your configuration, the port configuration are not consistent.
2. You've got wasted ports, i.e., note that STP will tend to block traffic on one specific port to avoid it from looping, hence does redundant links BW is useless. I highly recommend to configure Etherchannel to fully utilize the ports and please use LACP protocol so that there will be protocol negotiation

3. based on your configuration you didn't tweak any STP settings this is very important
-Make sure you define your root switch, it's up to you to decide who your root switch is on your network

Dude seriously, your network is very susceptible to looping
-S1, S2, S3
-S1, S2, Control SW

What's your purpose of having that many cascading ports? Your network is not really that in good shape dude

PS: Look at your switches CPU!!! they are at 70% that is not normal, in a few hours you whole network will experience highly intermittent connection add another few hours and your network will go down

Dude, remove your unecessary redundant connections, whats your purpose of having uplink ports from each of SW1, SW2 and SW3 goign to control SW of yours? that a big loop

swillet41 · ‎09-23-2015

LJ,

Thanks. All of the lab switches are interconnected with each other w multiple links (2 each). That is the physical cabling requirement of the POD setup.

I did make S1 the root bridge with the priority 0 command.

I'm going to reset the equipment on Friday and will let you know what happens. According to the vendor, my config is ok.

Thanks,

Scott

LJ Gabrillo · ‎09-23-2015

Again even if STP is in place it doesn't mean that it will protect your network 100%
look at your CPU it's alread clocking in 70% that's not normal, even if the switch is under heavy usage, i've seen switches that only clock in at 30% CPU

A complete redesign of your network, I would highly recommend this. I do not know how you setup your router and ASA, but i will still assume that these are switchports, hence again STP is involved.

Configuration wise, there is really no issue (except for the inconsistency of your port configuration) but again once STP is involved, you must always consider its effects

LJ Gabrillo · ‎09-23-2015

Anyway, If I was you, and my only redundancy connection is through switchports hence I am forced to use direct connections with STP highly involved, I would set it up like this

MAKE SURE:
1. Consistent port configuration
2. STP priority setting is harcoded
-MAKE SURE TO INCLUDE ALL VLANS INVOLVED!! Most people tend to miss this e.g., spanning-tree vlan 10,20-25,30,50 priority 0

STILL STILL STILL
Note that I still based the diagram on your current setup, just removed certain port connections.
Though if possible for all your routers and the ASA, to have only one uplink to your central switch. This removes possible scenarios of looping

Better yet, if those 3 switches are in just one location, then just stack them

LJ Gabrillo · ‎09-23-2015

Dude, you need to do this urgently, remove your redundant connections on the switches. See image below on what connections to remove.

You might be asking why to remove this
1. Well to remove the loop that is slowly building up in your network
-Flapping and high CPU proves that, also how you setup your network it's very messy

2. I would really use the word "overdosage of redundant ports" without considering the setup and its effect
-Yes STP will always kick in but it does not mean it will help you mitigate a loop 100% of the time. Especially on your setup, in w/c the switch has a lot of ports to consider to avoid looping. Add to that no STP tweaking on your side. it makes it worse

Side note:
I can see that the routers and ASA have multiple onnections as well, i will assume these are switchports NOTE switchports, they act as switches, and will sitll consider STP. w/c again adds to the many loopholes on your network that can cause looping

PS: I myself is a fan of redundant connection but on your network I wouldn't do it like that, too many loopholes that can cause looping. I highly recommend that you get an experienced SI(Systems Integrator) on your place wherever you are and ask for design recommendations.

swillet41 · ‎09-23-2015

I understand the concern about loops, but since this is a "lab environment" it needs to be wired according to the vendor. The vendor could not duplicate the issue and advised it's my equipment.

Question?? Why are we only getting the loop when config "switchport trunk native vlan 99" on the trunk ports between S1 and S2. If your theory was correct, wouldn't we get a loop regardless of any config?

Thanks,

Scott