Network outage form a faulty network switch

Debabrata Majhi · ‎02-09-2015

Dear All,

We have 3 Core switc 2 of the work as root switch for ODD and Even vlan load balancing and one switch act as backup swith for all vlan.

Priority like-

Core-1-4096--Root for vlan 2 8192 Root for vlan 3

Core-2-4096--Root for vlan 3 8192 Root for vlan 2

Core-3 -12288 for all vlan

All access switch is connected from distribution switch and distribution swithc is connected from core swithc.

All uplink by fiber optic cable.

We have configured HSRP for vlans.

Core-1

interface Vlan3
description " Voice VLAN_UCS"
ip address 172.30.28.2 255.255.252.0
ip helper-address 172.16.100.53
standby version 2
standby 1 ip 172.30.28.1
standby 1 priority 90
standby 1 preempt

Core-2

interface Vlan3
description " Voice VLAN_UCS"
ip address 172.30.28.2 255.255.252.0
ip helper-address 172.16.100.53
standby version 2
standby 1 ip 172.30.28.1
standby 1 priority 120
standby 1 preempt

Core-3

description " Voice VLAN_UCS"
ip address 172.30.28.2 255.255.252.0
ip helper-address 172.16.100.53
standby version 2
standby 1 ip 172.30.28.1
standby 1 priority 70
standby 1 preempt

Recently we have faced a major network outage for a faulty switch .The switch was hanged.

We have collect the below mentioned log from diferent switch

1-02-2015 12:12:25 Local7.Warning x.x.x.x 1102: 001086: *Sep 11 23:16:59.671 UTC: %SW_MATM-4-MACFLAP_NOTIF: Host 3037.a612.40c1 in vlan 2 is flapping between port Gi1/0/2 and port Po7
01-02-2015 12:12:25 Local7.Warning X.X.X.X 1102: 001086: *Sep 11 23:16:59.671 UTC: %SW_MATM-4-MACFLAP_NOTIF: Host 3037.a612.40c1 in vlan 2 is flapping between port Gi1/0/2 and port Po7
01-02-2015 12:12:25 Local7.Warning X.X.X.X169335: Feb 1 12:12:24.289 IST: %SW_MATM-4-MACFLAP_NOTIF: Host 4061.86c0.d312 in vlan 34 is flapping between port Gi0/9 and port Gi0/48
01-02-2015 12:12:25 Local7.Warning X.X.X.X1100: 001084: *Sep 11 23:16:59.344 UTC: %SW_MATM-4-MACFLAP_NOTIF: Host 4061.86c0.d4d4 in vlan 20 is flapping between port Po7 and port Gi1/0/2
01-02-2015 12:12:25 Local7.Warning X.X.X.X1100: 001084: *Sep 11 23:16:59.344 UTC: %SW_MATM-4-MACFLAP_NOTIF: Host

Please help how will we stop this type of network failure ?

Regards

Debabrata

Leo Laohoo · ‎02-09-2015

%SW_MATM-4-MACFLAP_NOTIF: Host 4061.86c0.d4d4 in vlan 20 is flapping between port Po7 and port Gi1/0/2

That is not a "network failure". It's more like a "configuration" failure.

This is saying that the MAC address can be found to be originating from Po7 and Gi 1/0/2. This normally happens when the MAC address was NOT configured for EtherChannel and there machine has two (or more) NICs connected to the network.

Debabrata Majhi · ‎02-09-2015

Hi,

I think for this failure creates forwarding loops (or STP loops). for that switch learn the mac addaress from deiferent switch.No pc has two nic.

Please response if I am wrong

Debabrata

Leo Laohoo · ‎02-09-2015

No pc has two nic.

How sure are you? My work PC has THREE (3) NIC cards.

Debabrata Majhi · ‎02-09-2015

I am sure because here are around 550 PC and No pc has two nic port.Probem not for single MAC address at the time failure lots of MAC addess came with this type of alert.

Leo Laohoo · ‎02-20-2015

I am sure because here are around 550 PC and  No pc has two nic port.Probem not for single MAC address at the time failure lots of MAC addess came with this type of alert.

Then track down the MAC address. The error message posted above is a wealth of information. Track down the MAC address and you should be able to determine whether the PC/Servers are dual linked or the switches they're connected to are multi-linked or not.