C1921 bug? One way multicast and wrong interface info in "show i

richard-artevea · ‎11-06-2012

C1921, running version 15.1(4)M2, with licence for "IP base" feature set only.
Trying to pass multicast via a PPTP VPN from a Windows XP machine to work around a non multicast-aware WAN link

1. With the IP Base feature set I am able to create a plain PPTP VPN without any encryption; the Windows XP machine can bring it up and unicast data passes through it OK in both directions.

2. But when trying to send multicast, only one-way traffic is observed:
i. Windows XP host on far end of PPTP VPN and a local PC both running old Microsoft tool "MPING.EXE", sending and listening for traffic on the groiup 225.100.101.102
i. The distant host receives and echoes back the packets received from the local machine + sending its own (confirmed with Wireshark running at the far end)
ii. But the local machine directly connected to the C1921 router does not hear any packets from the far end; Wireshark shows only the ones it is sending.

3. Group status ("show ip igmp membership") as far as the C1921 is concerned shows both ends (192.168.50.10 (local end) and 192.168.50.201 (distant end via the PPTP VPN)) joined to the group

Site1-TNA#sh ip igmp membership
Flags: A - aggregate, T - tracked
       L - Local, S - static, V - virtual, R - Reported through v3
       I - v3lite, U - Urd, M - SSM (S,G) channel
       1,2,3 - The version of IGMP, the group is in
Channel/Group-Flags:
       / - Filtering entry (Exclude mode (S,G), Include mode (G))
Reporter:
       <mac-or-ip-address> - last reporter if group is not explicitly tracked
       <n>/<m>      - <n> reporter in include mode, <m> reporter in exclude

Channel/Group Reporter Uptime Exp. Flags Interface

*,239.255.255.250              192.168.50.20   00:44:53 02:23 2A     Vl1
*,239.255.255.250              192.168.0.132   00:45:23 02:28 2A     Gi0/0
*,239.255.255.100              192.168.0.2     00:44:51 02:13 1A     Vl1
*,239.255.255.100              192.168.0.2     00:44:53 02:11 1A     Gi0/0
*,225.100.101.102              192.168.50.10   00:00:20 02:39 2A     Vl1
*,225.100.101.102              192.168.50.201 00:42:42 02:09 2A     Vi2.1
*,224.0.1.60                   192.168.0.6     00:45:23 02:21 2A     Gi0/0
*,224.0.1.40                   192.168.50.224 00:45:24 02:39 2LA    Lo0

4. But "show ip mroute" for that group shows an error; for the source on the far end of the PPTP VPN (having the IP address 192.168.50.201), the source interface is incorrectly shown as GigabitEthernet0/0 (should be Virtual-Access2.1 for that PPTP VPN) and the outgoing interface is shown as Virtual-Access2.1 (should be Vlan1 - where the local PC having the IP address 192.168.50.10 is connected (to a HWIC-4ESW card in the router)

Site1-TNA#sh ip mroute 225.100.101.102
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report,
       Z - Multicast Tunnel, z - MDT-data group sender,
       Y - Joined MDT-data group, y - Sending to MDT-data group,
       V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 225.100.101.102), 00:44:53/stopped, RP 10.0.0.1, flags: SJC     <<== PIM-SM SPT - OK
Incoming interface: Null, RPF nbr 0.0.0.0
Outgoing interface list:
    Vlan1, Forward/Sparse-Dense, 00:02:31/00:02:21
    Virtual-Access2.1, Forward/Sparse-Dense, 00:44:53/00:02:54

(192.168.50.10, 225.100.101.102), 00:02:30/00:00:29, flags: T <<== Source 192.168.50.10 - sOK
Incoming interface: Vlan1, RPF nbr 0.0.0.0
Outgoing interface list:
Virtual-Access2.1, Forward/Sparse-Dense, 00:02:30/00:02:54

(192.168.50.201, 225.100.101.102), 00:44:52/00:02:02, flags: J      <<== Source 192.168.50.201 - not OK
Incoming interface: GigabitEthernet0/0, RPF nbr 192.168.0.1       <<== Wrong incoming interface reported
Outgoing interface list:
    Vlan1, Forward/Sparse-Dense, 00:02:31/00:02:21
    Virtual-Access2.1, Forward/Sparse-Dense, 00:44:52/00:02:54      <<== Bogus outgoing interface?

5. I have tried adding static mroutes and messing about with parameters for the virtual-template interface for the PPTP VPN, but the problem remains. And if I put another local PC onto a different Ethernet port of the router, the multicast traffic does flow both ways - so the issue is solely with the PPTP VPN.

After a week of head-scratching I am getting more and more convinced that it's a bug... but wonder if it is already-known, has a workaround, or a fix in newer firmware?

Regards,
Richard Culpan - Artevea Digital Ltd.

richard-artevea · ‎11-08-2012

Reult of some further digging...

Router is behaving as if there is an RPF failure (e.g. no known route back to the source) - even though that source - the PPTP VPN - is directly connected and is working for unicast traffic!

(a) "show ip mroute" lists the "incoming interface" for the source 192.168.50.201 (the PPTP VPN) as being GigabitEthernet0/0 because that is the interface leading to the gateway of last resort (the route to 0.0.0.0/0)

Site1-TNA#sh ip mroute 225.100.101.102
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report,
       Z - Multicast Tunnel, z - MDT-data group sender,
       Y - Joined MDT-data group, y - Sending to MDT-data group,
       V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 225.100.101.102), 16:01:31/stopped, RP 10.0.0.1, flags: SJC
Incoming interface: Null, RPF nbr 0.0.0.0
Outgoing interface list:
Vlan1, Forward/Sparse-Dense, 00:00:13/00:02:46
Virtual-Access2.1, Forward/Sparse-Dense, 15:43:45/00:02:13

(192.168.50.10, 225.100.101.102), 00:00:12/00:02:47, flags: T
Incoming interface: Vlan1, RPF nbr 0.0.0.0
Outgoing interface list:
Virtual-Access2.1, Forward/Sparse-Dense, 00:00:12/00:02:47

(192.168.50.201, 225.100.101.102), 00:00:14/00:02:45, flags: J
Incoming interface: GigabitEthernet0/0, RPF nbr 192.168.0.1      <<== Return path = gateway of last resort
Outgoing interface list:
    Vlan1, Forward/Sparse-Dense, 00:00:13/00:02:46
    Virtual-Access2.1, Forward/Sparse-Dense, 00:00:14/00:02:45

(b) "show ip mfib" lists the status as all packets from the source on the PPTP VPN 192.168.50.201 are being dropped due to RPF failure.

Site1-TNA#sh ip mfib 225.100.101.102
Entry Flags:    C - Directly Connected, S - Signal, IA - Inherit A flag,
                ET - Data Rate Exceeds Threshold, K - Keepalive
                DDE - Data Driven Event, HW - Hardware Installed
I/O Item Flags: IC - Internal Copy, NP - Not platform switched,
                NS - Negate Signalling, SP - Signal Present,
                A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,
                MA - MFIB Accept
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts:      Total/RPF failed/Other drops
I/O Item Counts:   FS Pkt Count/PS Pkt Count
Default
(*,225.100.101.102) Flags: C
   SW Forwarding: 0/0/0/0, Other: 0/0/0
   Tunnel1 Flags: A
   Virtual-Access2.1 Flags: F NS
     Pkts: 0/0
   Vlan1 Flags: F NS
     Pkts: 0/0
(192.168.50.10,225.100.101.102) Flags:
   SW Forwarding: 31/1/112/0, Other: 0/0/0
   Vlan1 Flags: A
   Virtual-Access2.1 Flags: F NS
     Pkts: 30/1
(192.168.50.201,225.100.101.102) Flags:
   SW Forwarding: 0/0/0/0, Other: 65/65/0       <<== Packets seemingly being dropped due to RPF failure
   Tunnel1 Flags: A
   Virtual-Access2.1 Flags: F
     Pkts: 0/0
   Vlan1 Flags: F NS
     Pkts: 0/0
   GigabitEthernet0/0 Flags: NS

(c) "show ip route" does correctly list the route to the host on the PPTP VPN, so the return path to the source 192.168.50.201 *should* be known...

Site1-TNA#sh ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 192.168.0.1 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 192.168.0.1
      10.0.0.0/32 is subnetted, 1 subnets
C        10.0.0.1 is directly connected, Loopback1
      192.168.0.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.0.0/24 is directly connected, GigabitEthernet0/0
L        192.168.0.4/32 is directly connected, GigabitEthernet0/0
S     192.168.40.0/24 [1/0] via 192.168.250.1
      192.168.50.0/24 is variably subnetted, 4 subnets, 2 masks
C        192.168.50.0/25 is directly connected, Vlan1
L        192.168.50.1/32 is directly connected, Vlan1
C        192.168.50.201/32 is directly connected, Virtual-Access2.1        <<== PPTP VPN
C        192.168.50.224/32 is directly connected, Loopback0
      192.168.250.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.250.0/24 is directly connected, GigabitEthernet0/1
L        192.168.250.2/32 is directly connected, GigabitEthernet0/1

(d) I have tried various flavours of "ip mroute" commands in an attempt to force the correct RPF (i.e. the "usual solution" to RPF failures where the unicast routing is maybe a little weird...).

i. Trying to set a default interface for that mroute (covering all the IP addresses in the local pool I have created for the PPTP VPN), such as:

ip mroute 192.168.50.200 255.255.255.248 Virtual-Template 1

is accepted but seemingly does nothing (wrong incoming interface for the source 192.168.50.201 still shows in "show ip mroute"

ii. Trying to use another router interface by name (e.g. Loopback 0 results in an error:
"Next hop IP address required for multi-access media."

iii. Trying to use another router interface by its IP address (e.g. 192.168.50.224 (the address of Loopback0)) results in an error:
"%Invalid RPF neighbor address (it's this router)"

iv. Defining a seemingly-silly mroute to the host itself with the next-hop as itself:

ip mroute 192.168.50.201 255.255.255.255 192.168.50.201

is accepted but seemingly does nothing (wrong incoming interface for the source 192.168.50.201 still shows in "show ip mroute")

(e) Tried disabling CEF on the Virtual-Template interface (incoming, outgoing or both) - e.g.
no ip mfib cef input
no ip mfib cef output

No improvement.

(f) Tried removing the static route entry for the gateway of last resort; the "show ip mroute" now shows the incoming interface for the source 192.168.50.201 as Null, with an RPF neighbour as 0.0.0.0 (no neighbour)

And, if I create any "narrower" static route that matches with the IP address 192.168.50.201, the incoming interface shown in "show ip mroute" is picked up from that route. Unfortunately, neither "Virtual-Template" nor "Virtual-Access" interfaces can be specified as the gateway interface in a static route.

So, IMO there is a bug - in that the multicast routing is not picking up the correct incoming interface from that (dynamically-created) connected host route when the PPTP VPN comes up, and is instead matching on any other existing route that happens to match that source's IP address.

And, I have so far found no way to fool it by creation of static routes or mroutes :-(

Any other suggestions?

Regards,
Richard

C1921 bug? One way multicast and wrong interface info in "show ip mroute"