Multicast group membership not being registered

[MaxCaines]

Hi

I have a multicast setup based on Catalyst 9500 switches which are terminating SVIs for clients and servers. There are two of them involved: the source and the client are both on VLANs terminated on one (R1), and the RP for the group in question is a loopback interface on the other (R2). The server is on VLAN 80, the client on VLAN 32, and VLAN 65 is the link between R1 and R2. I'm using iperf v2 for testing. The first odd thing is that although IGMP snooping on R1 shows that there is a client for the group, if I do "show ip igmp membership" it shows no entry for that group. So, with server and client both running, if I do "show ip mroute 239.255.11.11" (group in question) on R1, I get:

(*, 239.255.11.11), 01:41:54/stopped, RP <valid-RP>, flags: SPF
Incoming interface: Vlan65, RPF nbr <valid neighbour>
Outgoing interface list: Null

(<source>, 239.255.11.11), 01:41:54/00:02:19, flags: PFT
Incoming interface: Vlan80, RPF nbr 0.0.0.0
Outgoing interface list: Null

But if I do "ip igmp join-group 239.255.11.11" on the SVI that the client is connected to, that changes to:

(*, 239.255.11.11), 01:39:49/stopped, RP <valid-RP>, flags: SJCLF
Incoming interface: Vlan65, RPF nbr <valid neighbour>
Outgoing interface list:
Vlan32, Forward/Sparse, 00:00:20/00:02:52

(<source>, 239.255.11.11), 01:39:49/00:02:24, flags: LFT
Incoming interface: Vlan80, RPF nbr 0.0.0.0
Outgoing interface list:
Vlan32, Forward/Sparse, 00:00:22/00:02:50

At the same time, the client starts receiving packets from the source (being on the VLAN whose SVI I joined. So it's clearly not the snooping that is failing. It just seems that routing engine on R1 is not seeing the client as a member of the group, even though the switching engine is. I am completely baffled by this. Does anyone have any ideas?

Thanks

Max

[MaxCaines]

I've checked, by the way, and R1 is the DR on client's VLAN

[Giuseppe Larosa]

Hello @MaxCaines ,

I would suggest to perform a packet capture to verify what is the aspect of IGMP reports from the host in VLAN 32.

You can use a local  SPAN session on R1 to see this.

Hope to help

Giuseppe

[MaxCaines]

Hi Giuseppe

I should have mentioned in my original post that I had already done that. The Cat9500 is good for this because captures are so simple. The capture showed that R1 did receive an IGMP report for the group which looked valid to me. That’s why I was surprised that the VLAN didn’t show up as having any members of this group

Thanks for the idea though

Regards

Max

[Giuseppe Larosa]

Hello @MaxCaines ,

that was my first thought .

The second attempt is noting that the group address  G = 239.255.11.11 is in the admin scoped range

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750x_3560x/software/release/15-2_2_e/multicast/configuration_guide/b_mc_1522e_3750x_3560x_cg/b_mc_3750x_3560x_chapter_01.html#GUID-A57E5B23-6206-4FA1-9B36-5180B4230119

I would try to use a group like 225.255.11.11 just to see if anything changes.

Hope to help

Giuseppe

[MaxCaines]

Good point. Thanks, I’ll try that

Max

[paul driver]

Hello

---

@MaxCaines wrote:

Catalyst 9500 switches which are terminating SVIs for clients and servers
VLAN 80, the client on VLAN 32, and VLAN 65 is the link between R1 and R2
RP for the group in question is a loopback interface on the other (R2

---

What switch is the DR for client vlan 32 as that would responsible to forward the joins towards RP of rtr2 loopback0, also check for any rpf failures
sh ip pim neigbour 
sh ip mroute 239.255.11.11 count
sh ip route x.x.x.x (RP)
sh ip route <source ip of mc>

[MaxCaines]

Hi Paul

The DR on VLAN 32 is the 9500 (R2) itself. I did check for routing or RPF issues, and couldn’t find any. What’s odd is that if I make the SVI for VLAN 32 a static member of the multicast group, everything works fine. IGMP snooping on the 9500 is marking the VLAN as a receiver for the group when the client starts up, but IGMP itself doesn’t add it as a member

Thanks

Max

[paul driver]

Hello

Well the cause suggests the igmp joins are not being received by the RP from the switch as you state that switch is the DR so hence the request to post the output of those commands with addition of the below:

sh ip igmp interface 

[MaxCaines]

Thanks Paul. I’ll look at that tomorrow

Regards

Max

[MaxCaines]

Hi Paul

Here are the outputs from those commands. I've also included a couple of "show ip rpf" where that seemed relevant:

R1 (FHR and LHR)
================
sh ip pim neigh
PIM Neighbor Table
Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,
P - Proxy Capable, S - State Refresh Capable, G - GenID Capable,
L - DR Load-balancing Capable
Neighbor Interface Uptime/Expires Ver DR
Address Prio/Mode
134.220.255.245 Vlan65 9w4d/00:01:35 v2 1 / S P G
134.220.255.241 Vlan65 40w5d/00:01:41 v2 1 / S P G
134.220.255.2 Vlan65 5w2d/00:01:44 v2 1 / G
134.220.255.242 Vlan65 1y25w/00:01:22 v2 10/ DR S P G
134.220.255.251 Vlan65 1y22w/00:01:21 v2 1 / S P G
134.220.255.252 Vlan65 1y25w/00:01:40 v2 1 / S P G
134.220.255.243 Vlan65 1y22w/00:01:32 v2 1 / S P G

134.220.255.242 is R1
134.220.255.244 is R2

sh ip route 134.220.191.1 (RP)
Routing entry for 134.220.191.1/32
Known via "ospf 2", distance 110, metric 2, type intra area
Last update from 134.220.255.242 on Vlan65, 7w0d ago
Routing Descriptor Blocks:
* 134.220.255.242, from 134.220.255.242, 7w0d ago, via Vlan65
Route metric is 2, traffic share count is 1

sh ip rpf 134.220.191.1
RPF information for ? (134.220.191.1)
RPF interface: Vlan65
RPF neighbor: mi040-95-01-staff.rid.wlv.ac.uk (134.220.255.242)
RPF route/mask: 134.220.191.1/32
RPF type: unicast (ospf 2)
Doing distance-preferred lookups across tables
RPF topology: ipv4 multicast base, originated from ipv4 unicast base

sh ip route 134.220.141.176 (source)
Routing entry for 134.220.140.0/22
Known via "connected", distance 0, metric 0 (connected, via interface)
Routing Descriptor Blocks:
* directly connected, via Vlan80
Route metric is 0, traffic share count is 1

sh ip igmp int vl32 (LHR)
Vlan32 is up, line protocol is up
Internet address is 134.220.199.1/24
IGMP is enabled on interface
Current IGMP host version is 2
Current IGMP router version is 2
IGMP query interval is 60 seconds
IGMP configured query interval is 60 seconds
IGMP querier timeout is 120 seconds
IGMP configured querier timeout is 120 seconds
IGMP max query response time is 10 seconds
Last member query count is 2
Last member query response interval is 1000 ms
Inbound IGMP access group is not set
IGMP activity: 65290 joins, 65290 leaves
Multicast routing is enabled on interface
Multicast TTL threshold is 0
Multicast designated router (DR) is 134.220.199.1 (this system)
IGMP querying router is 134.220.199.1 (this system)
No multicast groups joined by this system

R2 (RP)
=======
sh ip pim neigh
PIM Neighbor Table
Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,
P - Proxy Capable, S - State Refresh Capable, G - GenID Capable,
L - DR Load-balancing Capable
Neighbor Interface Uptime/Expires Ver DR
Address Prio/Mode
134.220.255.245 Vlan65 9w4d/00:01:30 v2 1 / S P G
134.220.255.241 Vlan65 40w5d/00:01:39 v2 1 / S P G
134.220.255.2 Vlan65 5w2d/00:01:34 v2 1 / G
134.220.255.252 Vlan65 1y43w/00:01:37 v2 1 / S P G
134.220.255.251 Vlan65 1y22w/00:01:17 v2 1 / S P G
134.220.255.244 Vlan65 1y25w/00:01:20 v2 1 / S P G
134.220.255.243 Vlan65 1y22w/00:01:30 v2 1 / S P G

134.220.255.242 is R1
134.220.255.244 is R2

sh ip route 134.220.191.1 (RP)
Routing entry for 134.220.191.1/32
Known via "connected", distance 0, metric 0 (connected, via interface)
Routing Descriptor Blocks:
* directly connected, via Loopback1
Route metric is 0, traffic share count is 1

sh ip route 134.220.141.176 (source)
Routing entry for 134.220.140.0/22
Known via "ospf 2", distance 110, metric 2, type intra area
Last update from 134.220.255.244 on Vlan65, 7w0d ago
Routing Descriptor Blocks:
* 134.220.255.244, from 134.220.255.244, 7w0d ago, via Vlan65
Route metric is 2, traffic share count is 1

sh ip rpf 134.220.141.176 
RPF information for sccm02a.unv.wlv.ac.uk (134.220.141.176)
RPF interface: Vlan65
RPF neighbor: ? (134.220.255.244)
RPF route/mask: 134.220.140.0/22
RPF type: unicast (ospf 2)
Doing distance-preferred lookups across tables
RPF topology: ipv4 multicast base, originated from ipv4 unicast base

Thanks

Max

[MaxCaines]

I forgot the mroute counts:

R1 (FHR and LHR)

sh ip mroute 239.255.11.11 count

IP Multicast Statistics
162 routes using 290672 bytes of memory
65 groups, 1.49 average sources per group
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kilobits per second
Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit etc)

R2 (RP)

IP Multicast Statistics
348 routes using 508672 bytes of memory
66 groups, 4.27 average sources per group
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kilobits per second
Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit etc)

[paul driver]

Hello
Can you post a topology diagram just to clarify the physcial setup of this?

[MaxCaines]

Hi Paul

Here is a diagram. A couple of things I should mention:

• This is all going on inside a VRF which is used on both R1 and R2. I don't think that should change anything, but I should mention it.
• This morning I've tried putting a client on another VLAN whose SVI I moved from R1 to the 6509 which used to occupy the role of R1, so has a very similar configuration, including the multicast settings and VRF. The IGMP join worked, the client showed up as a member of the IGMP group, and I was able to run a multicast data transfer from the source whose SVI is on R1 without configuring static groups on the SVI. The 6509 is also shown below.

[MaxCaines · Accepted solution]

I've now found what was causing this problem. It is basically self-inflicted. Some time ago a I followed a recommendation from the Nessus software we run to block use of IP Options on our Cisco switches/routers. I followed this without much checking about possible side-effects. However, IGMP v2 packets include an IP Option in the header, so they were being dropped by the routers. Once I removed the "ip option drop" statement, multicast joins started working again. Thanks to anyone who looked at this, and particularly to Paul