04-15-2013 09:08 AM - edited 03-07-2019 12:49 PM
Hi Guys,
Iam seeing above issue on two of my switches connected to core switch ....i know there are quite of few discussion open on same issue but mine is diff....
i see same issue on two switches connected via core swicth on same vlan ( 112)....when i do mac address lookup it says the mac thats generating this error is invalid so cant track the source of this mac....also just saw on topoogy change notification on core traced it back to originating switch which is also generating this error but dnt see any change on the switch that is generated topology change notification....prob is vlan 112 all interface on both switches conected via core are generating this message so five interfaces each .....any expert advise on how to approach it as i cant get to source port generating this as nearly five ports in vlan 112 on bloth switches generating this error. thanks
Apr 15 15:56:08: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 15:56:50: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 15:56:51: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 15:58:29: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 15:59:27: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46
Apr 15 15:59:45: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:00:14: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa3/0/46
Apr 15 16:00:36: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:02:40: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 16:03:22: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 16:03:31: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46
Apr 15 16:04:03: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa3/0/46 and port Gi3/0/1
Apr 15 16:04:34: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:04:41: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa2/0/46
Apr 15 16:05:05: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Fa2/0/46 and port Gi3/0/1
Apr 15 16:05:13: %SW_MATM-4-MACFLAP_NOTIF: Host 00ff.ffff.ffff in vlan 112 is flapping between port Gi3/0/1 and port Fa3/0/46
sh spanning-tree vlan 112
VLAN0112
Spanning tree enabled protocol rstp
Root ID Priority 8192
Address 001e.13c1.5a70
Cost 3004
Port 109 (GigabitEthernet3/0/1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 49264 (priority 49152 sys-id-ext 112)
Address 001f.261c.1d80
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300
UplinkFast enabled but inactive in rapid-pvst mode
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Fa2/0/46 Desg FWD 3019 128.104 P2p
Fa1/0/46 Desg FWD 3019 128.50 P2p
Gi3/0/1 Root FWD 3004 128.109 P2p
Fa3/0/46 Desg FWD 3019 128.158 P2p
Fa3/0/47 Desg FWD 3100 128.159 P2p
Fa3/0/48 Desg FWD 3019 128.160 P2p
04-15-2013 09:22 AM
ASAK Mohammed,
There are lots of thread discussing about this, you should do a search before creating a new post.
Anyway, this is how you approach these types of flapping:
1. Is the the given MAC flapping in the log flapping only 1 time or you see it multiple times over a reasonobly short time?
If you see it only once or once every 2-3 hours this might be not an issue worth being investigated. Sporadic one time flapping are expected in L2 broadcast domain.
If you see it often continue to step 2.
2. Identify and locate the flapping mac in vlan 125: 3270.990a.a504
Is the mac of a dual-homes server using some kind of load balancing algorithm (active/active) for which the same address is used from both NICs?
If yes, the message is not and issue but just an indication. Fix this type of LB (make it active/standby or make sure the server uses 2 different mac addresses, one per NIC) or if it is not possible leave it like this.
3. Is the MAC a the wireless NIC of a PC?
Make sure that the user was not moving from one AP to another (flapping is normal in this case)
4.
See if you have increasing TCN's and check if they are coming from the same interface.
From this point on you keep on troubleshooting STP until you find the offending link (likely going up and down) or the switch. You also need to check if STP in vlan112 is coherent with the actual L2 topology you have.
=====================================================
2- Some more details information which might be helpfull to you.
http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a
00801434de.shtml#subtopic1k
Problem
The switch generates %SYS-3-P2_ERROR: Host xx:xx:xx:xx:xx:xx is flapping
between ports? messages, where xx:xx:xx:xx:xx:xx is a MAC address.
Description
This example shows the console output that you see when this error occurs:
%SYS-4-P2_WARN: 1/Host 00:50:0f:20:08:00 is flapping between port 1/2 and port
4/39
Use the steps and guidelines in this section in order to understand and
troubleshoot the cause of this error message.
The message indicates that your Catalyst 4500/4000 switch has learned a MAC
address that already exists in the content-addressable memory (CAM) table, on
a port other than the original one. This behavior repeatedly occurs over short
periods of time, which means that there is address flapping between ports..
If the message appears for multiple MAC addresses, the behavior is not normal.
This behavior indicates a possible network problem because the MAC addresses
move quickly from one port to another port before the default aging time. The
problem can be looping traffic on the network. Typical symptoms include:
· High CPU utilization
· Slow traffic throughout the network
· High backplane utilization on the switch
For information on how to identify and troubleshoot issues with spanning tree,
refer to Spanning Tree Protocol Problems and Related Design Considerations
<http://www.cisco.com/en/US/tech/tk389/tk621/technologies_tech_note09186a00800
951ac.shtml> .
If the error message appears for one or two MAC addresses, locate these MAC
addresses in order to determine the cause. Issue the show cam mac_addr command
in order to identify from where these MAC addresses have been learned. In this
command, mac_addr is the MAC address that the error reports as flapping.
After you determine between which ports this MAC address is flapping, track
down the MAC address. Connect to the intermediate devices between your
Catalyst 4500/4000 and the device that has the problem MAC address. Do this
until you are able to identify the source and how this device connects to the
network.
Note: Because the MAC address is flapping between two ports, track down both
of the paths.
This example shows how to track both of the paths from which this MAC address
has been learned:
Note: Assume that you have received this message and you have begun to
investigate it.
%SYS-4-P2_WARN: 1/Host 00:50:0f:20:08:00 is flapping between port 1/2 and port
4/39
In order to track down how this MAC address was learned from both ports,
complete these steps:
1. Consider port 1/2 first, and issue the show cam dynamic 1/2 command.
If you see the MAC address 00:50:0f:20:08:00 in the list of the MAC addresses
that have been learned on this port, determine if this is a single host that
is connected or if there are multiple hosts that are registered on that port.
2. On the basis of whether there is a single or multiple hosts,
investigate the device:
o If there is a single host (00:50:0f:20:08:00) that is connected, check the
other port that is registered and see if the host is dually attached to the
switch.
In this example, the other port is port 4/39.
o If the host has connections to other devices that can eventually lead back
to this switch, try to track down the intermediate devices.
With Cisco devices, issue the show cdp neighbors mod/port detail command. The
output provides information about intermediate devices.
Here is sample output:
Cat4K> (enable) show cdp neighbors 1/2 detail
Port (Our Port): 1/2
Device-ID: brigitte
Device Addresses:
IP Address: 172.16.1.1
Novell address: aa.0
Holdtime: 171 sec
Capabilities: ROUTER
Version:
Cisco Internetwork Operating System Software
IOS (tm) 2500 Software (C2500-JS-L), Version 12.0(7)T, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-1999 by cisco Systems, Inc.
Compiled Mon 06-DEC-99 17:10 by phanguye
Platform: cisco 2500
Port-ID (Port on Neighbors's Device): Ethernet0
VTP Management Domain: unknown
Native VLAN: unknown
Duplex: half
System Name: unknown
System Object ID: unknown
Management Addresses: unknown
Physical Location: unknown
Cat4K> (enable)
3. Establish a Telnet session with the device and follow the path of the
MAC address.
In this example, the IP address is 172.16.1.1.
Repeat the procedure for all MAC addresses that the error message reports as
flapping.
4. Create a simple diagram of the source device with that MAC address and
of the physical connections (the Catalyst 4500/4000 ports) from which and to
which this MAC address is flapping.
The diagram enables you to determine if this is a valid port and path for your
network layout.
If you verify that both ports on which the MAC address is flapping provide a
path toward that network node, there is a possibility that you have a
spanning-tree failure issue. Refer to Spanning Tree Protocol Problems and
Related Design Considerations
<http://www.cisco.com/en/US/tech/tk389/tk621/technologies_tech_note09186a00800
951ac.shtml> in order to isolate and troubleshoot this loop.
In large networks in which multiple hosts from multiple vendors are
interconnected, difficulty arises as you try to track down the host with use
of just the MAC address. Use the search utility for the IEEE OUI and
Company_id Assignments <http://standards.ieee.org/regauth/oui/index.shtml> in
order to track down these MAC addresses. This list is the front end of the
database where IEEE has registered all MAC addresses that have been assigned
to all vendors. Enter the first three octets of the MAC address in the Search
for: field of this page in order to find the vendor that is associated with
this device. The first three octets in the example are 00:50:0f.
These are other issues that can cause this message to appear:
· Server NIC redundancy problem?There is a server with a dual-attached
NIC that misbehaves and does not follow the standards. The server uses the
same MAC address for both ports that connect to the same switch.
· Hot Standby Router Protocol (HSRP) flapping?Flapping HSRP can cause
these messages to appear in the Supervisor Engine console. If you notice that
HSRP implementation in your network is unstable, refer to Understanding and
Troubleshooting HSRP Problems in Catalyst Switch Networks
<http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a00800
94afd.shtml> in order to resolve the problem.
· EtherChannel misconfiguration?A misconfigured EtherChannel connection
can also cause these symptoms. If ports that the flapping message reports are
members of the same channel group, check your EtherChannel configuration and
refer to Understanding EtherChannel Load Balancing and Redundancy on Catalyst
Switches
<http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a00800
94714.shtml> in order to troubleshoot the configuration.
· Host reflects packets back onto the network?The reflection of packets
back onto the network by a host can also cause flapping. Typically, the root
cause of this packet reflection is a broken NIC or any failure of the physical
interface of the host that is connected to the port.
If the reflection of packets by the host is your root cause, obtain a sniffer
trace and examine the traffic that goes to and from the ports on which the
messages have appeared. If a host reflects packets, you typically see
duplicate packets in the trace. The duplicate packets are a possible symptom
of this flapping of the MAC address.
Refer to Configuring SPAN and RSPAN
<http://www.cisco.com/en/US/docs/switches/lan/catalyst4000/6.3and6.4/configura
tion/guide/span.html> for details on how to configure a port for use with a
sniffer.
· Software or hardware defect?If you have tried to troubleshoot the
flapping message with the instructions in this section but you still notice
the issue, seek further assistance from Cisco Technical Support
<http://www.cisco.com/en/US/support/tsd_cisco_worldwide_contacts.html> . Be
sure to mention and provide documentation of the information that you have
collected while you followed the steps. This information makes further
troubleshooting quicker and more efficient.
HTH
REgards
Inayath
*Plz rate all usefull posts.
04-15-2013 11:14 AM
Hi Inayat,
Thanks for reply just so u know i did check and tried everything suggested in other thread simmilar to mine issue but didnt get to bottom of the issue and my issue is different tpo other posts thats y posted.....i tried everything in ur post too only thing i found is moment i turn off some wireless ap's then i dont see any flaps in logs ...before that could see flaps every minute.....also cpu utilization on core and other switches is normal ....backplane traffoc is normal too...dont see any anamolies apart from ap's generating this broadcast type of mac....anyone else with any more suggestion...
04-16-2013 08:55 AM
HI Inyat,
Found offending link ....prob solved
thanks alot
03-04-2014 08:44 AM
Hi Mohammad,
Can you explain how you fixed this error?, we have the same error here, with some AP connected to the switches and giving this same flapping error.
Thanks.
03-04-2014 09:51 AM
Hi Creyes
If i remember correctly the issue i found was that two wireless AP's in production area were connected to same switch and the wireless card of one of the tools was jumping from one ap to another which was casuing this issue, i traced the mac in question back to the access switch then AP's and saw that device would jump from one AP to another. this would cause for switch to generate error as the same source it saw 30 sec ago on port 1/46 is now being learned on port 1/47...my advise would be to track it back to the source (port by port) to access switch.if you suspect loop then cpu process would clearly indicate that.thanks
03-04-2014 09:55 AM
if you suspect loop have a look at below link which will help you step by step in establishing if loop exist
my fav command from below link
ITLABSW#show spanning-tree detail | inc ieee|occurr|from|is exec
The output from this command will show you the port the last TCN was received on and the time which it was received.
Look for the port that received a TCN in the last few seconds.
08-22-2018 11:57 AM
if this is a user device it could be a user roaming and picking up different AP's in different switches. this is only if you are using local breakout at the AP's
10-13-2016 10:12 AM
Dear all,
I have a similar problem. I can see that the same Mac address is learnt via a physical interface and also an svi .. that is physical interface is part of vlan 20 and the svi is also vlan 20. User gets disconnected very often. Any help will be appreciated
07-13-2018 01:25 PM
These forums are used so people can interact and ask questions with problems that arise. I normally Google most my issues but I rely on these forums quit often. Please don't assume someone has not done there due diligence to find the problem to their issue. When you make statements like you did at the beginning of your terrific response you come across as you are being putout to have to answer questions on this forum. This is just a little friendly advice.
01-23-2015 04:07 PM
I just wanted to share that I had a similar problem with this and our fix turned out to simply be a bad patch cable. So, it's worth looking at if your are running out of options.
03-06-2019 08:20 AM
We resolved this by not allowing all vlan traffic through a trunked port on another interface even though it showed which two ports were flapping. It must of kept seeing the mac address traffic looping through the trunked port.
07-14-2018 03:54 AM
Excellent Tips. I think it will be very helpful for me.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide