GRE Tunnel streams

[wwwlstr0707]

This is very high level, wanting to know if this is possible.  We're the middle man.  I don't have access to or knowledge of equipment or configs on the other providers.  Currently we are handed off a GRE tunnel to a router.  We have an ipsec tunnel to another location, over a 50Mbps ethernet link from a local ISP, to another router at another location.  It works.   Like I mentioned, very high level.

We are at capacity on that 50Meg link.  We want a 200Meg link, we can't get that but they can give is four 50Mbps links and I've been tasked with "load balancing" it.  I put a wan switch in front of the routers, configured a four port Port-Channel, data still flows.  But monitoring the ports in the port channel you can see a single link is being utilized, if you shut that port it immediately starts flowing over another, so the port channel is working and failover works.  But the powers that be see this as a fail because we need to "load balance" over these 4 ports and have data passing over each individual link for this to be a success

First question, is there even a better way to do this, take the hand off from the other provider and "load balance" across 4 links we are given, knowing that hand off is a GRE tunnel?

Second question, if we're ultimately after a 200Meg link do we just fight that fight until we get it because that's the only way this will work?

Third question, more just on GRE that I'm not quite understanding...since the Port channel is sending that data over a single physical link in the port channel, does a GRE tunnel create just a single TCP conversation?  I've gone through several of the port channel load balancing algorithms and the result is just the same...

Looking for ideas or things to try or ammunition to argue with, so thoughts are welcome...and yes I realize this ask is a bit out of the norm, lol

[Joseph W. Doherty]

Yes, a GRE tunnel is seen as a single flow.

Possibly if you run four tunnels, you could get each across an Etherchannel, but it would difficult to do with Etherchannel.  Any possibility of making the links L3?

[Joseph W. Doherty]

With individual L3 links, you should be able to easily map one tunnel per link.

Or, run one tunnel across four L3 links, and use CEF packet-by-packet for routing.  You'll effectively obtain 200 Mbps.  Normally, round robin packet LB is bad because it often causes out of sequence delivery.  However, we're dealing with one flow and the tunnel endpoints might be adjusted to not be as sensitive to the out of order delivery.

Other options that come mind are MLPPP (200 Mbps is likely going to be an issue) or some hardware mux at both ends.

[wwwlstr0707]

In fact yes, that is a possibility.  So your next response mentions CEF, which I'm not as familiar with but I'm willing, and allowed actually, to explore all possibilities.  Got any good resources off the top of your head?    Thanks!

[Joseph W. Doherty]

---

@wwwlstr0707 wrote:

In fact yes, that is a possibility.  So your next response mentions CEF, which I'm not as familiar with but I'm willing, and allowed actually, to explore all possibilities.  Got any good resources off the top of your head?    Thanks!

---

https://www.cisco.com/c/en/us/td/docs/ios/12_4t/ip_switch/configuration/guide/tceflbs.html#wp1046328

Possibly also:

https://www.cisco.com/c/en/us/td/docs/ios/12_4t/ip_switch/configuration/guide/tceflbs.html#wp1053790

Couldn't find much info on the usefulness of the tunnel option.  One reference mentioned it might be for DMVPN tunnels.

BTW, to be clear, you'll need your links to be ECMP L3 for CEF to split packets across them.

[David Ruess]

Hello,

Looking at the problem initially since your traffic is encapsulated in a GRE tunnel the only thing the port channel will see is the source/destination of the GRE tunnel and not the internal traffic. This could be the reason none of the load balancing mechanisms work because the end result is the same. To test you could do as @Joseph W. Doherty said and run multiple tunnels to see if that works but I also agree in that it may be better off as a L3 port-channel.

-David

[wwwlstr0707]

Thanks, and yes, it would appear I'm trying to get a port channel to do something it's not designed for, with a single source/dest of the tunnel I was hoping there may be a load balance algorithm I was missing to compensate.   But mgmt is just seeing it as broken since it's over one physical link.  

What would be the difference with a L3 port channel though?  I'm not quite understanding

[David Ruess]

L3 Port-channel will have its own src/dst IP and it should be able to break it out over all the links. It sounds correct in theory but Id definitely have to test it.

[Joseph W. Doherty]

---

@David Ruess wrote:

L3 Port-channel will have its own src/dst IP and it should be able to break it out over all the links. It sounds correct in theory but Id definitely have to test it.

---

The breakout would be the same as for a L2 Etherchannel.  The only difference is port-channel interface has an IP instead of being an access or trunk port.

[Joseph W. Doherty]

---

@David Ruess wrote:

To test you could do as @Joseph W. Doherty said and run multiple tunnels to see if that works but I also agree in that it may be better off as a L3 port-channel.

No, you misunderstand, not a L3 port-channel, each member link a L3 p2p for ECMP.

A L3 port-channel will balance the same a a L2 port-channel.

[MHM Cisco World]

SW with L2 PO you need good hash for load balance 

make hash include scr/dest L4 port

MHM

[Joseph W. Doherty]

---

@MHM Cisco World wrote:

SW with L2 PO you need good hash for load balance 

make hash include scr/dest L4 port

MHM

---

Wouldn't a single tunnel use the same L4 ports?

[Giuseppe Larosa]

Hello @wwwlstr0707 ,

if all traffic is encapsulated within a single GRE tunnel the devices on the path see a single IP flow that has  a single IP source and single IP destination  so a single link  in the port-channel is used and a single provider link is used.

You would need 4 different GRE Tunnels with different sources and  destinations and you may  need to use PBR to force each GRE p2p  tunnel  over a single ISP link.

But for return traffic I don't think you can achieve the same.

Going to a 200 Mbps link is probably your best move, the only one that provides a secure performance boost effect.

Hope to help

Giuseppe

[Joseph W. Doherty]

---

@Giuseppe Larosa wrote:

You would need 4 different GRE Tunnels with different sources and  destinations and you may  need to use PBR to force each GRE p2p  tunnel  over a single ISP link.

But for return traffic I don't think you can achieve the same.

Going to a 200 Mbps link is probably your best move, the only one that provides a secure performance boost effect.

---

Yup, 4 tunnels to 4 p2p WAN links.  If a tunnel is bound to each p2p interface, you wouldn't need to use PBR, I believe.  If tunnel is bound to something like a loopback, then PBR would be needed.

As to return traffic, or even moving to making the links 4 p2p, cooperation will be needed from the other side.

Fully agree having a 200 Mbps link would be best choice, but you'll obtain most of the bandwidth boost if using CEF per-packet across the 4 links.  As each packet is round-robin, utilization should be pretty equal across the 4 links.

[wwwlstr0707]

Interestingly the wan switch is a 9300

ip load-sharing per-packet is not an option in interface config mode, but globally i can enter

ip cef load-sharing algorithm

And show ip cef shows entries....