MAC Flapping - Assitance Needed

[davidhvoss]

Hello,

I have an extensive network of Cisco 2960 switches.  I have been experiencing frequent cases of MAC flapping on the network.  The issue is that mac addresses are flapping between a port that leads to the mac address and a port that could not possibly lead to the mac address (the port is up, but there is no possible path to that mac address via that port).  This has occurred about 10-12 times over the past few months.  I don't want to assume it's a bug, but want to rule out if I'm missing something that I should check.

Here is an example:

Apr  2 04:37:48 UTC: %SW_MATM-4-MACFLAP_NOTIF: Host 0023.ab79.d044 in vlan 200 is flapping between port Gi0/24 and port Gi0/20

In this case, that mac address cannot be reached via Gig 0/20 - there is no path via Gig 0/20 to that mac address.  Sure enough, when it flaps to that port, we experience an outage to that mac address.

Any thoughts?

Here is the configuration:

interface GigabitEthernet0/20
 description Uplink to PROVIDER A (330)
 switchport trunk allowed vlan 1-200
 switchport mode trunk
end

1360Basement#sh run int gig 0/24
Building configuration...

Current configuration : 145 bytes
!
interface GigabitEthernet0/24
 description Feed to Roof Switch
 switchport trunk allowed vlan 1-200
 switchport protected
 media-type sfp
end

Show version

Cisco IOS Software, C2960 Software (C2960-LANBASEK9-M), Version 12.2(44)SE6, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Mon 09-Mar-09 18:10 by gereddy
Image text-base: 0x00003000, data-base: 0x01100000

ROM: Bootstrap program is C2960 boot loader
BOOTLDR: C2960 Boot Loader (C2960-HBOOT-M) Version 12.2(44)SE6, RELEASE SOFTWARE (fc1)

1360Basement uptime is 23 weeks, 13 hours, 6 minutes
System returned to ROM by power-on
System restarted at 19:37:08 CST Tue Oct 27 2015
System image file is "flash:c2960-lanbasek9-mz.122-44.SE6/c2960-lanbasek9-mz.122-44.SE6.bin"

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C2960G-24TC-L (PowerPC405) processor (revision F0) with 61440K/4088K bytes of memory.
Processor board ID FOC1402Z2DS
Last reset from power-on
2 Virtual Ethernet interfaces
24 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

64K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : 9C:4E:20:E6:AA:80
Motherboard assembly number     : 73-10015-07
Power supply part number        : 341-0098-02
Motherboard serial number       : FOC14152772
Power supply serial number      : AZS141325AU
Model revision number           : F0
Motherboard revision number     : A0
Model number                    : WS-C2960G-24TC-L
System serial number            : XXXXXXXXXXXXXX
Top Assembly Part Number        : 800-26673-04
Top Assembly Revision Number    : C0
Version ID                      : V04
CLEI Code Number                : COMFX00BRA
Hardware Board Revision Number  : 0x01

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 24    WS-C2960G-24TC-L   12.2(44)SE6           C2960-LANBASEK9-M

Configuration register is 0xF

[Vasilii Mikhailovskii]

Hello.

Could it be a loop over G0/20?

[davidhvoss]

There could be a loop over Gig 0/20 but that mac address could not be within that loop.  The mac address is only possibly known via Gig 0/24.

[pwwiddicombe]

The switch wouldn't be lying...  B-)

As said above, there is probably a second path, probably intended for redundancy, that's causing packets from that workstation to be seen via the alternate port.  If that is the case, though, you should probably be seeing a lot of them and causing network issues at the time of the flapping.

If it's only one you're seeing, is there any chance that's a workstation getting reconnected periodically to an alternate port for testing?  Do you see one flip and minutes later flip back, or is the log filled with flap entries?

[davidhvoss]

That's what I thought when I started to troubleshoot it but the customer assures me that there is no possible way for it to learn of that mac off of gig 0/20. 

That being said, I will have to likely visit onsite to verify as the answers I've received are what I expect - check for a redundant path otherwise it would have to be a bug.

[Mohammad Ramadan A.Hafiez]

Hello Bro.

          I seems you have something vague in your network, If am in your situation I would find answers for these questions...

1- what is the device that generate traffic using this mac address "Obviously it's something cisco".

2- Is this device connected through single or dual network ports? and may be they are connected to multiple switches to provide network redundancy.

notice : If it was a device that may be connected via two network ports, and at the device side was configured as any kind of bonding, and at the switch side was not configured as bonded ports "channel", this will cause this mac flap.

I have experienced this issue before and was matter of port channel configuration needed on the switch.