Cisco confirmed me, that

schwarzenbachcyb · ‎01-20-2016

I have a problem with IP Multicast

Setup

The Setup: Because the layer2 segment between the both routers doesn't support PIM-snooping, I have to build a GRE tunnel between the both routers. C3650 is the rendezvous point (autorp). Over the GRE Tunnel i have EIGRP and PIM enabled.

/--GRE+PIM--\
| |
HOST-A <--> C981F <---> C3650 <--> HOST-B
^
|
RP

My problem: If the multicast source (sender) is on Host-A and the client, who likes to receive the multicast stream, is on Host-B, it works. But if I change the direction of the multicast stream, Host-B is the sender and Host-A is the receiver, it doesn't work. On the C891F I can only see a (*,G) but no (S,G) entry.

The output if Host-B is the sender and Host-A is the receiver:

C3650#sh ip mroute 239.193.5.51
(*, 239.193.5.51), 00:14:00/00:03:28, RP 10.10.10.10, flags: SF
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    Tunnel121, Forward/Sparse-Dense, 00:13:51/00:03:28
(10.65.203.2, 239.193.5.51), 00:14:00/00:03:29, flags: FT
  Incoming interface: Vlan10, RPF nbr 0.0.0.0
  Outgoing interface list:
    Tunnel121, Forward/Sparse-Dense, 00:13:51/00:03:28

C891F#sh ip mroute 239.193.5.51
(*, 239.193.5.51), 00:14:21/00:02:18, RP 10.10.10.10, flags: SJC
  Incoming interface: Tunnel121, RPF nbr 10.111.0.1
  Outgoing interface list:
    Vlan1, Forward/Sparse-Dense, 00:14:21/00:02:18
C891F#
C891F#sh ip pim rp
Group: 239.193.5.51, RP: 10.10.10.10, uptime 00:05:56, expires 00:01:57
C891F#
C891F#sh ip igmp snooping groups
Vlan Group Version Port List
---------------------------------------------------------
1 239.193.5.51 v2 Gi7

Does anyone can help me with this problem? Is it, because the multicast source in directly connected to the RP?

The relevant configs from both routers:

The C3650:

ip routing
ip multicast-routing
!
interface Loopback10
 ip address 10.10.10.10 255.255.255.255
 ip pim sparse-dense-mode
!
interface Vlan3986
 description WAN
 ip address 10.65.206.10 255.255.255.0
!
interface GigabitEthernet1/0/1
 switchport access vlan 3986
 switchport mode access
!
interface Vlan10
 description LAN
 ip address 10.65.203.1 255.255.255.248
 ip pim sparse-dense-mode
!
interface GigabitEthernet1/0/10
 switchport access vlan 10
 switchport mode access
!
interface Tunnel121
 ip address 10.111.0.1 255.255.255.252
 ip pim sparse-dense-mode
 load-interval 30
 tunnel source Vlan3986
 tunnel destination 10.65.206.21
!
router eigrp 100
 network 10.0.0.0
 passive-interface Vlan3986
!
ip pim autorp listener
ip pim send-rp-announce Loopback10 scope 16
ip pim send-rp-discovery scope 16

The C891F:

ip multicast-routing
ip cef
!
interface GigabitEthernet8
 description WAN
 ip address 10.65.206.21 255.255.255.0
!
interface Vlan1
 description LAN
 ip address 10.202.1.1 255.255.255.248
 ip pim sparse-dense-mode
!
interface Tunnel121
 ip address 10.111.0.2 255.255.255.252
 ip pim sparse-dense-mode
 tunnel source GigabitEthernet8
 tunnel destination 10.65.206.10
!
router eigrp 100
 network 10.0.0.0
 passive-interface GigabitEthernet8
!
ip pim autorp listener

Peter Paluch · ‎01-20-2016

Hi Matthias,

At first sight, I do not see anything particularly wrong with your setup.

One thing I would suggest changing: Instead of running EIGRP over the Tunnel interface - which should not be done in your setup anyway, as you do not want to run unicast traffic over the tunnel - please configure both C3650 and C891F with the same command as follows:

ip mroute 0.0.0.0 0.0.0.0 Tunnel1121

This will install a static route to the multicast routing table saying that a multicast packet is allowed to come through (be sourced from behind) the Tunnel1121 interface, satisfying the multicast RPF check performed on the multicast packet source. You can safely declare the Tunnel interface as passive interfaces in EIGRP - once again, most certainly, don't run EIGRP over that tunnel.

If this change does not help then we need to start looking further. However, the outputs of the show ip mroute seem to be fine. There is no immediate problem visible. The fact that you're seeing only the (*,G) entry on the C891F may be related to the fact that this router did not decide to switch over from the shared tree to the source tree - for now, it's not an indication of a certain problem.

Can you confirm that the multicast sources are sending the stream with a sufficient TTL so that it doesn't expire anywhere along the way?

Best regards,
Peter

schwarzenbachcyb · ‎01-20-2016

Hi Peter

I have to route also unicast traffic, so I need to use EIGRP (or another routing protocol). In an earlier setup, I configured EIGRP over the WAN port (Gi8, vlan3986) and just PIM over the GRE tunnel, but in that setup multicast didn't worked in both directions. Then I changed EIGRP to speak only over the GRE tunnel, that solved the problem for one direction of the multicast stream (the setup that I have now).

I can confirm, that TTL is not the problem. I solved this already a few days ago.

I added the static mroute on both devices and reloaded them, but it do not solve the problem.

The final setup is more complex: There I will have two C3650 for redundancy, and a lot more C891F, that are connect to the same layer2 segment and also to other ports on the C3650. So each C891F will have two GRE tunnels, one to each C3650. And the C3650 will have a lot more GRE tunnels, one to every C891F. So i'm not sure, if a static 0.0.0.0 0.0.0.0 mroute can used in that setup anyway.

But what I have to solve first, is receive the multicast stream in this simple setup in both directions. That's what I like to solve with this discussion.

I also tried some different IOS versions and found some bugs. Currently I have on the C3650 Version Denali 16.1.1E and on the C891F 15.4(3)M3.

Jon Marshall · ‎01-20-2016

Matthias

I thought Peter's solution would fix it to be honest because it did look like it might be an RPF failure causing the (S,G) group entry not to be installed.

When you have it connected up and you are testing what does a "sh ip mroute count" show ?

Jon

schwarzenbachcyb · ‎01-20-2016

C3650#show ip mroute count
Use "show ip mfib count" to get better response time for a large number of mroutes.

IP Multicast Statistics
5 routes using 6440 bytes of memory
3 groups, 0.66 average sources per group
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kilobits per second
Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit etc)

Group: 239.193.5.51, Source count: 1, Packets forwarded: 5265, Packets received: 5265
RP-tree: Forwarding: 0/0/0/0, Other: 0/0/0
Source: 10.65.203.2/32, Forwarding: 5265/162/7/0, Other: 5265/0/0

Group: 224.0.1.39, Source count: 1, Packets forwarded: 0, Packets received: 0
Source: 10.10.10.10/32, Forwarding: 0/0/0/0, Other: 0/0/0

Group: 224.0.1.40, Source count: 0, Packets forwarded: 0, Packets received: 0

C891F#show ip mroute count
Use "show ip mfib count" to get better response time for a large number of mroutes.

IP Multicast Statistics
5 routes using 2310 bytes of memory
3 groups, 0.66 average sources per group
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kilobits per second
Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit etc)

Group: 239.193.5.51, Source count: 0, Packets forwarded: 0, Packets received: 0

Group: 224.0.1.39, Source count: 1, Packets forwarded: 0, Packets received: 14
Source: 10.10.10.10/32, Forwarding: 0/0/0/0, Other: 14/0/14

Group: 224.0.1.40, Source count: 1, Packets forwarded: 0, Packets received: 14
Source: 10.111.0.1/32, Forwarding: 0/0/0/0, Other: 14/0/14

Peter Paluch · ‎01-20-2016

Matthias,

Thanks for responding.

I am not suggesting to stop the EIGRP entirely. I am only suggesting to stop running it over the Tunnel interface. As I indicated before, I do not believe you want to run unicast traffic through the tunnel - the tunnel is there only to contain the multicast traffic, right? In that case, running EIGRP over the tunnel is counterproductive and could even be considered harmful. My strong recommendation is to keep your EIGRP running but declare the Tunnel interfaces as passive interface so that no EIGRP adjacency is established over the Tunnel.

Can you please repeat the scenario that does not work, i.e. Host-B sending traffic, Host-A receiving traffic (or not), and then perform show ip rpf 10.65.203.2 command on both routers and post the result here? I assume that 10.65.203.2 is the IP address of the Host-B sending multicast traffic.

Best regards,
Peter

schwarzenbachcyb · ‎01-20-2016

Yes 10.65.203.2 is Host-B

C3650#show ip rpf 10.65.203.2
RPF information for ? (10.65.203.2)
RPF interface: Vlan10
RPF neighbor: ? (10.65.203.2) - directly connected
RPF route/mask: 10.65.203.0/29
RPF type: multicast (connected)
Doing distance-preferred lookups across tables
RPF topology: ipv4 multicast base

C891F with "ip mroute 0.0.0.0 0.0.0.0 tunnel121"

C891F#show ip rpf 10.65.203.2
RPF information for ? (10.65.203.2)
RPF interface: Tunnel121
RPF neighbor: ? (10.111.0.1)
RPF route/mask: 0.0.0.0/0
RPF type: multicast (static)
Doing distance-preferred lookups across tables
RPF topology: ipv4 multicast base

C891F without "ip mroute 0.0.0.0 0.0.0.0 tunnel121"

C891F#show ip rpf 10.65.203.2
RPF information for ? (10.65.203.2)
RPF interface: Tunnel121
RPF neighbor: ? (10.111.0.1)
RPF route/mask: 10.65.203.0/29
RPF type: unicast (eigrp 100)
Doing distance-preferred lookups across tables
RPF topology: ipv4 multicast base, originated from ipv4 unicast base

Peter Paluch · ‎01-20-2016

Matthias,

Thank you very much - no wrong data here, either - and how about show ip igmp group from the C891 during the time when Host-A is subscribed and Host-B is sending data? (please not the show ip igmp snooping group)

Additional question - what is the size of the multicast packets? Is it anywhere near 1500 bytes? Is it possible to send packets that are significantly smaller, say, 1000 bytes? I want to make sure that there is no issue with possible fragmentation or MTU mismatch.

Best regards,
Peter

schwarzenbachcyb · ‎01-21-2016

Hi Peter, a new day, a new hope :)

C891F#show ip igmp group
IGMP Connected Group Membership
Group Address Interface Uptime Expires Last Reporter Group Accounted
239.193.5.51 Vlan1 00:04:35 00:02:45 10.202.1.2
224.0.1.40 Vlan1 16:39:19 00:02:46 10.202.1.1

I captured the packets with tcpdump and they told me, that the packet-length is 1356, that should be ok. To be sure, I tried now to send smaller packets, tcpdump sees now 980 size packets, but they still don't reach the C891F:

C891F#sh ip mroute 239.193.5.51 count
Group: 239.193.5.51, Source count: 0, Packets forwarded: 0, Packets received: 0

schwarzenbachcyb · ‎01-21-2016

I used Wireshark to sniff the packets between the two routers:

When no multicast stream is active (normal background traffic)

10.10.10.10 -> 224.0.1.39, Auto-RP, RP annoucement
10.111.0.2 -> 224.0.0.13, PIMv2, Hello
10.111.0.1 -> 224.0.0.13, PIMv2, Hello
10.111.0.1 -> 224.0.1.40, Auto-RP, RP mapping
10.111.0.2 -> 224.0.0.13, PIMv2, Prune (for group 224.0.1.39)

When I start the multicast stream that works

Source: 10.202.1.2
Receiver: 10.65.203.2

!start streamer on host 10.202.1.2
10.202.1.2 -> 239.193.5.51, PIMv2, Register
10.10.10.10 -> 10.111.0.2, PIMv2, Register-stop (for group 239.193.5.51, source 10.202.1.2)

!start client on host 10.65.203.2
10.111.0.1 -> 224.0.0.13, PIMv2, Join (for group 239.193.5.51, upstream-neighbor 10.111.0.2, IP 10.202.1.2)
10.202.1.2 -> 239.193.5.51, UDP, dst-port 2051 (stream-data)
->continue sending the UDP stream-data->

When I start the multicast stream that doesn't works

Source: 10.65.203.2
Receiver: 10.202.1.2

!start streamer on host 10.65.203.2
<nothing>

!start client on host 10.202.1.2
10.111.0.2 -> 224.0.0.13, PIMv2, Join (for group 239.193.5.51, upstream-neighbor 10.111.0.1, IP 10.10.10.10)
but no multicast stream-data

Jon Marshall · ‎01-21-2016

Matthias

I think that is to be expected because in the case that doesn't work the source is directly connected to the RP so there are no PIM register messages to be sent.

Jon

schwarzenbachcyb · ‎01-22-2016

Cisco confirmed me, that there is no multicast routing/PIM over GRE support for the 3650 platform.

Thanks anyone for helping me.

PIM Multicast Routing