10-02-2017 05:12 AM - edited 03-08-2019 12:13 PM
We had an issue last week at a remote location. Every few minutes the IPTV system would pixilate for a few seconds - there was no significant impact to unicast traffic. PIM-SM was enabled on the L3 VLAN interface - creating the local IGMP querier - the IPTV headend was connected to a separate VLAN - with users in a "User" VLAN. We monitored the PIM state during a problem period - no change was seen - Inbound & outbound interfaces remained unchanged. It was clear this issue was at a IGMP snooping level.
Debugging for IGMP snooping showed no root cause - and IGMP state appeared to remain stable during the issue.
To cut a long story very short, the issue turned out to be related to Spanning Tree. A command making user interfaces spanning tree "edge" ports by default had somehow been removed. This resulted in all ports participating in Spanning tree, and all interfaces triggering Spanning Tree Topology Changes (TCN's) every time the interface went up and down. These TCN's resulted in Spanning Tree flushing the MAC table & flooding all traffic - with IGMP using the MAC address table for forwarding decisions, this caused IGMP to flood traffic.
So - if IGMP is configured correctly & you are getting occasional multicast floods, check whether they coincide with Spanning Tree changes!
A closer look at our Wireshark captures showed some unicast flooding at the start - but it was dwarfed by the Multicast flooding - simply due to the traffic levels from the local headend. As expected, the Spanning Tree TCN was there just before the start of the issue....... isn't hind-sight wonderful!
Just thought I'd share as looking for a multicast issue when it was an underlying Spanning Tree issue caused a lot of pain!
10-02-2017 06:44 AM
Thanks for sharing this with us Marc!
Very helpful.
02-09-2018 10:42 PM
Dear Marc ,
we have similar kind of issue
we have mulicast streamers ( video wall ) , for that we have enable the IGMP snooping and querier on the switch ( cisco 4506 E ) not on the VLAN .
We have one vlan only , when this multicast streamer is connected to the switch it make all other device very slow and even loss of packets .
we need to kniow what can be done to solve this issue .
i have attached the wireshark output and the snapshot for the ping
https://drive.google.com/open?id=14jX9PcxwzmdC2upydX-9v5rKtgh_jS1x
so what have you done to solve the problem what CLI command you have use.
02-10-2018 04:44 PM
Hi, I have checked your capture and it shows spanning tree topology changes. Apply the following filter to Wireshark & you will see the BPDU's with the "TC" marker - fileter: stp.protocol == 0x0000
Ensure your user interfaces are configured as edge ports.
Figure out what is causing the topology changes - then the flooding will stop.
Good luck!
Marc
02-12-2018 03:58 AM
A good starting point will be the "show spanning-tree" command - check the status of the ports listed ensure that all the end user ports are listed as Type - "P2P Edge" - only uplinks should be Type "P2P". Enduser ports should be set as portfast edge ports with the command "spanning-tree portfast edge" - or even better, use the "spanning-tree portfast edge default" command to make all ports portfast / edge ports by default & then just configure Spanning tree on the uplinks where you want spanning tree working. This will prevent end users switching their PC's on and off from triggering network wide spanning-tree topology changes - which will cause multicast flooding.
"Show Spanning-tree detail" can be used to show the stability of spanning tree - this details the number of topology changes along with the time since last topology change for each VLAN configured - use this to check whether you have stabilized spanning tree.
There are lots of support posts on sorting out spanning-tree issues - so put the multicast flooding issue to one side at the moment & fix the spanning tree issue. Once that is fixed, multicast will probably be fixed too - you wont be able to fix the multicast flooding issue with an unstable spanning tree.
Thanks
Marc
02-12-2018 11:24 PM
Thanks Marc for your suggestion and reply.
Will share with you regarding the spanning-tree detail output as soon i am at site .
07-12-2019 03:12 PM
Hi i have a similar problem with the difference that my port is down and up when i make changes physical as internal changes but all my ports is in "edge", can you help me?. Sorry for revive the post.
01-28-2020 07:52 PM
Hi,
So I just got this issue from last week and solved it, opening case to TAC it's not helping much, they said it's a bug https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuj74358/?referring_site=ss&dtid=osscdc000283
but after upgrade to recent IOS the issue is still persist.
1. STP instability is a thing, you should have a stable TCN, check from
show spanning-tree vlan X detail
2. But if you cannot improve much like on my environment, just issue this command to stop multicast flooding when TCN appear on each access interface
no ip igmp snooping tcn flood
check also from
show ip igmp snooping vlan X detail
you will see if there's TCN on the output
Topology change : Yes Protocol generating TCN : STP
I have ~800 IGMP groups of CCTV, the multicast flood eats ~800mbps to the end device which is insane, total output drops were increasing fast.
tested on 3750 and 3850, the stream is stable now
To optimize more, read here:
find section "Configuring TCN-Related Commands":
find section "Suppressing Multicast Flooding"
https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst4500/12-2/53SG/configuration/config/multi.html#wp1049520
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide