Poor multicast over Tunnel

[KGrev]

Hi,

I have a cellular router (IR809G) with a tunnel setup. I'm trying to view a multicast video over that tunnel. It does appear but the video quality is very poor. I'm in a great reception area and speed tests from the router into the network hover around ~50meg. The video file is 480p mp4. Other paths from the core router(rp) can view this feed fine but the path through the tunnel is the issue. Are there any requirements for a tunnel connection for multicast to flow properly? Ive attached some details from the spoke and core routers. I hope it is helpful.

Is it interesting that the tunnel does not show a DR for pim? My other pim interfaces on the router do show DR (they are removed from the text)

Thank you for any advice.

[KGrev]

@Joseph W. DohertyHere are some images from wireshark. I changed my video stream to 1400, 1360, and 1300. I also pulled a different feed from an alternate source. This source is set to default 1500. Let me know if the images are not viewable. While these streams were being pulled the tunnel was set to ip mtu 1400/mss adjust 1360 and the cellular interface was set to the same.

Thank you for your help.

[Giuseppe Larosa]

Hello @KGrev ,

>> the tunnel was set to ip mtu 1400/mss adjust 1360 and the cellular interface was set to the same.

This doesn't look like correct the cellular interface should have a greater MTU , 24 Bytes more to accomodate for GRE header otherwise the fragmentation is not solved.

The cellular line is your physical interface so it must be able to support GRE packets

Hope to help

Giuseppe

[KGrev]

@Giuseppe Larosashould i try:
Cell 0
ip mtu 1440
no ip tcp mss-adjust

Tunnel 10

ip mtu 1400

ip tcp mss-adjust 1360

Or should the cell 0 have an mss-adjust?

[Joseph W. Doherty]

What I suspect is transit traffic is subject to both GRE tunnel and cellular IPSec tunnel.  If so, to avoid fragmentation, we want to reduce MTU to cover overhead for both.

I was hoping the 1400 would be enough of a reduction for both, but if not, we would go smaller, not larger.  Adust-mss, would be usually set MTU less 40.

Understand the forgoing unlikely to impact video packets because such packets usually don't set DF bit nor usually not TCP.

So, why adjust?

First, to avoid fragmentation of non-video traffic.  Second, to determine a MTU that doesn't need to be fragmented.

[KGrev]

@Joseph W. Doherty  I guess I should add that nothing is really "routed" over this tunnel. The router itself doesnt have static routes to send tcp traffic to the tunnel. It only has an mroute for 0.0.0.0 0.0.0.0 tunnel 10. Thats how the multicast is working. All other transmissions arent going through the tunnel. Its just for multicast.

[Joseph W. Doherty]

Ah, perhaps my light bulb just went on, dimly though.  ; )

All traffic transits the EasyVPN tunnel, correct?  This tunnel may be IPSec, so it's encapsulation overhead might be typical for such tunnels.  If so, an IP MTU of 1400 and a TCP adjust-mss 1360 may be very applicable.

Your GRE tunnel only carries the multicast traffic, correct?  Being basic GRE, its encapsulation overhead would be 24 bytes, but as it too would cross the EasyVPN tunnel (correct?), after GRE fragmentation, those packets could be fragmented again.  So, its IP MTU and TCP adjust-mss should allow for the GRE and EasyVPN overhead, BUT, again, I suspect your video packets don't set DF nor use TCP.  In theory, the only approach, to avoid fragmentation of you video packets, is to decrease the source's MTU.  Also in theory, do that, we don't need to concern ourselves with tunnel IP MTU and/or TCP adjust-mss settings, except for other traffic.  As GRE tunnel only carries video, we should be able to ignore those settings, but still important for your EasyVPN tunnel.

Assuming we preclude video fragmentation, even with the ideal EasyVPN MTU and adjust-mss settings, there can still be fragmentation on that tunnel (for non-TCP traffic that doesn't set DF).

Generally, it's a "good thing" to minimize fragmentation, and preclude it for your video, but fragmentation might not be the only issue causing your video issues across that path.

At this point, your next recourse might be to find an experienced A/V network engineer to "hands-on" analyze this issue.

[KGrev]

@Joseph W. Doherty @Giuseppe Larosa @Georg Pauwen  Are there any unique requirements to spanning multicast across the tunnel? Should I need mroutes in this sparse mode setup considering I now have a tunnel?  Here is an image with a few details if mroutes would be needed.

[KGrev]

Does "show ip traffic" contain information for multicast with the tunnel?
If I clear the counters and allow the multicast video to stream for a few minutes, its not showing any fragmentation at least from this perspective.

[KGrev]

@Giuseppe Larosa @Joseph W. Doherty @Georg Pauwen 

Does Cisco give out awards for annoyance of the month? I'm probably top contender.

I no longer think I have a fragmentation issue. I've tried all sorts of mtu and mss adjustments and wireshark always looks the same. It always shows the correct adjusted length also from the video source. I think i have been misunderstanding what wireshark is telling me this whole time.

If I pull the video stream from a local computer from the same switch as the host, wireshark looks pretty much identical as the pc on the cellular router. But the video is clear. Here is a picture of wireshark on the working computer. Looks similar to all the other pictures. The video source was set to 600 when I took this image but it looks the same at higher mtu sizes.

Meanwhile at that same time with mtu set to 600 on the source video, the pc across the tunnel looks terrible but the wireshark capture looks just like the pc back at the office.

Also if I force the video mtu to 1500 Then set my tunnel mtu too low, i can truly see an example of fragmentation. I can see the adjusted packet length followed by the fragments behind it.

With the tunnel set to the specs of your guidance it isn't fragmented like it is in this image.

I believe I just have a multicast video issue like @Joseph W. Doherty  mentioned before. But I am unsure what could be the issue here. The remote router only has an mroute of 0.0.0.0 0.0.0.0 tunnel 10 for multicast to establish.

Thanks for you guy's help so far. Guys like you keep the forums working.

[Torbjørn]

It would be interesting to see if you are experiencing any significant jitter in your multicast traffic across the tunnel. Could you test the performance using a tool like MultiCast hammer?

[KGrev]

Sorry @Torbjørn  I can't have third party software on the network. Is there a way to test this within cisco?

I am able to setup an ip sla responder.

My model doesn't show jitter but if I set the fequency low and refresh the command I can see the Latest RTT bounce back and forth from ~55 to ~350. Does this help?

[Torbjørn]

That does indicate that there is some jitter that could be problematic due to VLCs low default buffer size. Could you try to set the buffer to 1-2 seconds and see if that affects the issue? You can change this setting by doing the following: Tools > Preferences. Under "Show Settings" in the bottom left, press "all". Select "Stream Output" in the sidebar and change the value of "Stream Output Muxer Caching" to 1000 or 2000. If you see improvement you can play around with the buffer size to see how low you can go before issues appear.

VLC is a bit finnicky to test with in my experience, hence I usually stick with traffic-generator type applications for testing(MC Hammer, T-Rex, Ostinato etc.).

[Joseph W. Doherty]

55 to 350?!

So focused on fragmentation, overlooked possible other impairments to video quality, such as whether there would be benefit to bandwidth management and preferential treatment of the video multicast.

Does the cellular connection come with any specs like bandwidth guarantees?

[MHM Cisco World]

There are high over head go down to 1200 for tcp mss and 1280 for mtu.

[KGrev]

@MHM Cisco World  I adjusted the mtu and mss to your suggestions. No change.

@Torbjørn   I increased the buffer on the stream up to 5000 with no visible change in video quality.

Thank you guys for your help