Multicast Issue with NX-OS (vPC) - Even groups work - Odd groups do not work

[ulrickfr2001]

Hi everyone,

We are facing an issue with Multicast using PIM-SM (ASM). All devices seem to be configured correctly but the multicast streams do not work as expected. Bellow the description of the behavior using the information provided in the diagram.

 - RP IP address : 172.19.1.241

 - Receiver IP address : 172.18.66.1

 - There is vPC configured between N5Ks and N7Ks with Vlan tag

 - The interconnections are configured with HSRP (for redundancy)

 - When the IP source address is 172.19.35.5, the stream on multicast group 239.225.1.3 is received but the stream on multicast group 239.225.1.4 is not.

 - When the IP source address is 172.19.35.6 the stream on multicast group 239.225.1.4 is received but the stream on multicast group 239.225.1.3 is not.

Can someone help ?

[chrihussey]

So it sounds like the receiver can only get one multicast stream at a time and it's not an odd even thing. Correct?

Guess the first place to start would be to do a "show ip mroute" on all the switches and "sh ip igmp groups" on the switch with the receiver to see if the multicast groups are being requested and in fact are active at the same time.

Also, if you could have two hosts each request a single stream, one odd, one even (at the same time) and see if that works.

Just a starting point anyway.

[ulrickfr2001]

Hi Chrihussey and thanks for your support.

No the receiver can receive more than 2 multicast stream from differente sources and I have tested it. Anyway, in this case i am doing the test one by one and the result is what I described which is:

 - 1st test - IP source address is even and it streams on even multicast group - the stream is not received.

 - 2nd test - IP source address is even and it streams on odd multicast group - the stream is received.

 - 3rd test - IP source address is odd and it streams on odd multicast group - the stream is not received.

 - 4th test - IP source addres is odd and it streams on even multicast group - the stream is received.

So it really looks like even odd thing due to hashing with vPC

Bellow you will fine the ouput of what you asked for. (note we are in case 3 and 4)

N7K-1 VRF
*********
NPGVDC01-N7K%lumiere# sh ip igmp groups
IGMP Connected Group Membership for VRF "lumiere" - 3 total entries
Type: S - Static, D - Dynamic, L - Local, T - SSM Translated
Group Address Type Interface Uptime Expires Last Reporter
224.0.1.140 D Vlan1762 5y11w 00:03:59 172.19.6.249
239.225.1.3 L Vlan2010 00:00:10 00:04:14 172.18.66.1
239.225.1.4 L Vlan2010 00:44:00 00:02:51 172.18.66.1

NPGVDC01-N7K%lumiere# sh ip mroute 239.225.1.3
IP Multicast Routing Table for VRF "lumiere"

(*, 239.225.1.3/32), uptime: 00:31:48, pim ip igmp
Incoming interface: Vlan1762, RPF nbr: 172.19.6.249
Outgoing interface list: (count: 1)
Vlan2010, uptime: 00:00:59, igmp

NPGVDC01-N7K%lumiere# sh ip mroute 239.225.1.4
IP Multicast Routing Table for VRF "lumiere"

(*, 239.225.1.4/32), uptime: 00:31:54, pim igmp ip
Incoming interface: Vlan1762, RPF nbr: 172.19.6.249
Outgoing interface list: (count: 1)
Vlan2010, uptime: 00:31:54, igmp

(172.19.35.5/32, 239.225.1.4/32), uptime: 00:03:34, ip mrib pim
Incoming interface: Vlan1762, RPF nbr: 172.19.6.249
Outgoing interface list: (count: 1)
Vlan2010, uptime: 00:03:34, mrib

N7K-2 VRF
*********
NPGVDC02-N7K%lumiere# sh ip igmp groups
IGMP Connected Group Membership for VRF "lumiere" - 1 total entries
Type: S - Static, D - Dynamic, L - Local, T - SSM Translated
Group Address Type Interface Uptime Expires Last Reporter
224.0.1.140 D Vlan1762 5y11w 00:04:11 172.19.6.249

NPGVDC02-N7K%lumiere# sh ip mroute 239.225.1.4
IP Multicast Routing Table for VRF "lumiere"

Group not found
NPGVDC02-N7K%lumiere# sh ip mroute 239.225.1.3
IP Multicast Routing Table for VRF "lumiere"

Group not found

Firewall
********
FPGFG3140-1-ST1 (N3LumAtlas) # get router info multicast pim sparse-mode table 239.225.1.3
IP Multicast Routing Table

(*,*,RP) Entries: 0
(*,G) Entries: 3
(S,G) Entries: 4
(S,G,rpt) Entries: 3
FCR Entries: 1

(*, 239.225.1.3)
RP: 172.19.1.241
RPF nbr: 172.19.1.252
RPF idx: IntercoDefAtlas
Upstream State: JOINED
Local:
Joined:
IntercoDefAtlas
IntercoLumAtlas
Asserted:
FCR:
Source: 172.19.35.5
Outgoing:
IntercoLumAtlas
KAT timer running, 88 seconds remaining
Packet count 8

(172.19.35.5, 239.225.1.3)
RPF nbr: 172.19.1.252
RPF idx: IntercoDefAtlas
SPT bit: 0
Upstream State: JOINED
Local:
Joined:
Asserted:
Outgoing:
IntercoLumAtlas

(172.19.35.5, 239.225.1.3, rpt)
RP: 172.19.1.241
RPF nbr: 172.19.1.252
RPF idx: IntercoDefAtlas
Upstream State: NOT PRUNED
Local:
Pruned:
IntercoDefAtlas
Outgoing:
IntercoLumAtlas

FPGFG3140-1-ST1 (N3LumAtlas) #
FPGFG3140-1-ST1 (N3LumAtlas) # get router info multicast pim sparse-mode table 239.225.1.4
IP Multicast Routing Table

(*,*,RP) Entries: 0
(*,G) Entries: 3
(S,G) Entries: 4
(S,G,rpt) Entries: 3
FCR Entries: 1

(*, 239.225.1.4)
RP: 172.19.1.241
RPF nbr: 172.19.1.252
RPF idx: IntercoDefAtlas
Upstream State: JOINED
Local:
Joined:
IntercoDefAtlas
IntercoLumAtlas
Asserted:
FCR:

(172.19.35.5, 239.225.1.4)
RPF nbr: 172.19.1.252
RPF idx: IntercoDefAtlas
SPT bit: 1
Upstream State: JOINED
Local:
Joined:
IntercoLumAtlas
Asserted:
Outgoing:
IntercoLumAtlas

(172.19.35.5, 239.225.1.4, rpt)
RP: 172.19.1.241
RPF nbr: 172.19.1.252
RPF idx: IntercoDefAtlas
Upstream State: NOT PRUNED
Local:
Pruned:
IntercoDefAtlas
Outgoing:
IntercoLumAtlas

N7K-1 Global
************
NPGVDC01-N7K# sh ip mroute 239.225.1.3
IP Multicast Routing Table for VRF "default"

Group not found
NPGVDC01-N7K# sh ip mroute 239.225.1.4
IP Multicast Routing Table for VRF "default"

Group not found

N7K-2 Global
************
NPGVDC02-N7K# sh ip mroute 239.225.1.3
IP Multicast Routing Table for VRF "default"

(*, 239.225.1.3/32), uptime: 00:50:03, pim ip
Incoming interface: Vlan3131, RPF nbr: 172.19.1.241
Outgoing interface list: (count: 1)
Vlan3131, uptime: 00:50:03, pim, (RPF)

(172.19.35.5/32, 239.225.1.3/32), uptime: 00:07:10, pim mrib ip
Incoming interface: Vlan3105, RPF nbr: 172.19.0.83
Outgoing interface list: (count: 1)
Vlan3131, uptime: 00:04:27, pim

NPGVDC02-N7K# sh ip mroute 239.225.1.4
IP Multicast Routing Table for VRF "default"

(*, 239.225.1.4/32), uptime: 00:50:05, pim ip
Incoming interface: Vlan3131, RPF nbr: 172.19.1.241
Outgoing interface list: (count: 1)
Vlan3131, uptime: 00:50:05, pim, (RPF)

(172.19.35.5/32, 239.225.1.4/32), uptime: 00:07:04, pim mrib ip
Incoming interface: Vlan3105, RPF nbr: 172.19.0.83
Outgoing interface list: (count: 1)
Vlan3131, uptime: 00:07:04, pim

N5K-1 Global
************
NPGDIF01-N5548# sh ip mroute 239.225.1.3
IP Multicast Routing Table for VRF "default"

(172.19.35.5/32, 239.225.1.3/32), uptime: 00:07:42, ip pim
Incoming interface: Vlan35, RPF nbr: 172.19.35.5
Outgoing interface list: (count: 0)

NPGDIF01-N5548# sh ip mroute 239.225.1.4
IP Multicast Routing Table for VRF "default"

Group not found

N5K-2 Global
************
NPGDIF02-N5548# sh ip mroute 239.225.1.3
IP Multicast Routing Table for VRF "default"

(172.19.35.5/32, 239.225.1.3/32), uptime: 00:07:34, pim ip
Incoming interface: Vlan35, RPF nbr: 172.19.35.5
Outgoing interface list: (count: 1)
Vlan3005, uptime: 00:07:34, pim

NPGDIF02-N5548# sh ip mroute 239.225.1.4
IP Multicast Routing Table for VRF "default"

(172.19.35.5/32, 239.225.1.4/32), uptime: 00:07:29, ip pim
Incoming interface: Vlan35, RPF nbr: 172.19.35.5
Outgoing interface list: (count: 1)
Vlan3005, uptime: 00:07:27, pim

[chrihussey]

This may be overly simple and I know you did say all configurations are correct, but can you verify that "ip pim sparse-mode" is configured on all the proper interfaces, especially on the .252 N7K?

Perhaps post the output of "sh ip pim interface brief" of the two 7Ks with the receiver.

Thanks

[ulrickfr2001]

The output

N7K-1 VRF

***********

NPGVDC01-N7K%lumiere# sh ip pim interface brief
PIM Interface Status for VRF "lumiere"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Vlan2310 172.18.82.1 172.18.82.1 0 no
Vlan2089 172.18.76.33 172.18.76.34 1 no
Vlan2010 172.18.66.1 172.18.66.1 0 no
Vlan1762 172.19.6.253 172.19.6.253 2 no
Vlan508 172.19.110.2 172.19.110.3 1 no

N7K-2 VRF

**********

NPGVDC02-N7K%lumiere# sh ip pim interface brief
PIM Interface Status for VRF "lumiere"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Vlan2610 172.18.114.1 172.18.114.2 1 no
Vlan2089 172.18.76.34 172.18.76.34 1 no
Vlan1762 172.19.6.252 172.19.6.253 2 no
Vlan508 172.19.110.3 172.19.110.3 1 no

I found something interesting and maybe the answer of the problem i am facing. The architecture vPC/HSRP with PIM is just not supported because when the source is connected to a vPC one router can send the pim join and the other one receive the stream. In that case the stream cannot just be routed throught a vPC vlan.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/operations/n5k_L3_w_vpc_5500platform.html

[chrihussey]

Why isn't the VLAN 2010 interface on N7K-2 VRF listed?

Think you don't have multicast enabled on that particular interface.

[ulrickfr2001]

Because there is no interface vlan 2010 on N7K-2 VRF. This Vlan is not a vPC vlan. 

[chrihussey]

And how does the firewall know the route to 172.18.66.1?

[ulrickfr2001]

Static routing

FPGFG3140-1-ST1 (N3LumAtlas) # get router info routing-table details 172.18.66.1
Routing entry for 172.18.66.0/25
Known via "static", distance 10, metric 0, best
* 172.19.6.254, via IntercoLumAtlas

Is that matter to know the receiver IP address ?

[chrihussey]

I think that is your problem. The static should point to .253.

Short of that, create the VLAN2010 interface on the other 7K and if not part of the vPC establish a separate link to route between the two 7Ks.

Hope that makes sense.

[ulrickfr2001]

I cannot right now perfom the test you recommanded. But I did another easier test by skipping both N7Ks VRF on the routing process and configured interface vlan 1762 to join both multicast groups 239.225.1.3 and 239.225.1.4 and we have the same behavior.

Only one stream is received. I do not think the issue is related to the receiver IP address.

NPGVDC02-N7K# sh ip mroute 239.225.1.3 detail
IP Multicast Routing Table for VRF "default"

Total number of routes: 34
Total number of (*,G) routes: 12
Total number of (S,G) routes: 22
Total number of (*,G-prefix) routes: 0

(*, 239.225.1.3/32), uptime: 07:42:18, pim ip
Stats: 0/0 [Packets/Bytes], 0.000 bps
Attached oif(s) : No
Incoming interface: Vlan3131, RPF nbr: 172.19.1.241
Outgoing interface list: (count: 1)
Vlan3131, uptime: 07:42:18, pim, (RPF)

(172.19.35.5/32, 239.225.1.3/32), uptime: 00:01:05, pim mrib ip
Stats: 0/0 [Packets/Bytes], 0.000 bps
Attached oif(s) : No
Incoming interface: Vlan3105, RPF nbr: 172.19.0.83
Outgoing interface list: (count: 1)
Vlan3131, uptime: 00:01:05, pim

NPGVDC02-N7K#
NPGVDC02-N7K#
NPGVDC02-N7K# sh ip mroute 239.225.1.4 detail
IP Multicast Routing Table for VRF "default"

Total number of routes: 34
Total number of (*,G) routes: 12
Total number of (S,G) routes: 22
Total number of (*,G-prefix) routes: 0

(*, 239.225.1.4/32), uptime: 07:42:23, pim ip
Stats: 0/0 [Packets/Bytes], 0.000 bps
Attached oif(s) : No
Incoming interface: Vlan3131, RPF nbr: 172.19.1.241
Outgoing interface list: (count: 1)
Vlan3131, uptime: 07:42:23, pim, (RPF)

(172.19.35.5/32, 239.225.1.4/32), uptime: 00:01:22, pim mrib ip
Stats: 421/567508 [Packets/Bytes], 75.668 kbps
Attached oif(s) : No
Incoming interface: Vlan3105, RPF nbr: 172.19.0.83
Outgoing interface list: (count: 1)
Vlan3131, uptime: 00:01:22, pim

[paul driver]

Hello

On the interface source of the MC.

sh ip igmp interface x/x

sh ip igmp  groups | ex 224.0.1.40
sh ip mroute

[paul driver]

Hello

Also check each m/c interface between scr/dest

sh ip pim interface count  -  this should tell if something is/isnt getting through

[ulrickfr2001]

Looks like "sh ip pim interface count" command is not valid on NX-OS

NPGVDC01-N7K# sh ip pim interface ethernet 3/33 count
^
% Invalid command at '^' marker.

**********************

NPGVDC01-N7K# sh ip pim interface count
^
Invalid interface format at '^' marker.

[ulrickfr2001]

Hi Paul,

I do not really understand. From which device do you need those "show" commands ?