Loadbalancing Traffic via two tunnels

[Joy3]

Hallo,

I have an ISR1100 LTE dual-sim router. The router has two VPN tunnels configured each using the cellular interfaces as the the sources. Failover when one sim card fails works perfect. I would however like to loadbalance the traffic between these two interfaces. I entered two static routes as below:

ip route vrf INTERNET 0.0.0.0 0.0.0.0 Cellular0/2/0
ip route vrf INTERNET 0.0.0.0 0.0.0.0 Cellular0/2/1

However, when both cellular interfaces are up, they get an ip address but the tunnels remain in up/down state. 

Cellular0/2/0 10.147.27.95 YES IPCP up up
Cellular0/2/1 10.154.74.68 YES IPCP up up
ATM0/3/0 unassigned YES NVRAM administratively down down
Ethernet0/3/0 unassigned YES NVRAM administratively down down
Loopback0 172.x.x.x YES NVRAM up up
Tunnel100 172.x.x.x YES NVRAM up down
Tunnel200 172.x.x.x YES NVRAM up down

Could anyone help point me in the right direction, Thanks

[balaji.bandi]

is the tunnel go other side same place ?

Instead of Load-balancing, i would  prefer with Load share between tunnels and failover if one of the tunnel fails to other. (is this works ?)

[MHM Cisco World]

You can use both cellular if you change the Tunnel source to be LoopBack.
this make Tunnel load balance between two cellular interface.

[Joy3]

@balaji.bandi No, the tunnels are going to different IPs. Could you perhaps show me an example of how the load sharing between the 2 tunnels will look like.

@MHM Cisco World I have just changed the tunnels' source bu the tunnels are still in up/down state. But what exactly is the logic behind that given that the tunnel interfaces have a differnet IP address like below:

Loopback0 x.x.255.98 YES NVRAM up up
Tunnel100 x.x.254.98 YES NVRAM up down
Tunnel200 x.x.253.98 YES NVRAM up down

[balaji.bandi]

@balaji.bandi No, the tunnels are going to different IPs. Could you perhaps show me an example of how the load sharing between the 2 tunnels will look like.

first basic you need to fix the Tunnel state up before you load balance the traffc.

post show run to understand the config.

[Joy3]

R1#sh run

!
hostname R1
vrf definition INTERNET
!

!
redundancy
mode none
!
!
!
crypto ikev2 keyring DMVPN-KEYRING-1
peer ANY
address 0.0.0.0 0.0.0.0
pre-shared-key -xxx
!
!
crypto ikev2 keyring DMVPN-KEYRING-2
peer ANY
address 0.0.0.0 0.0.0.0
pre-shared-key xxx
!

crypto ikev2 profile FVRF-IKEv2-IWAN-TRANSPORT-1
match fvrf INTERNET
match identity remote address 0.0.0.0
identity local address x.x.254.98
authentication remote pre-share
authentication local pre-share
keyring local DMVPN-KEYRING-1
dpd 40 5 on-demand
!
crypto ikev2 profile FVRF-IKEv2-IWAN-TRANSPORT-2
match fvrf INTERNET
match identity remote address 0.0.0.0
identity local address x.x.253.98
authentication remote pre-share
authentication local pre-share
keyring local DMVPN-KEYRING-2
dpd 40 5 on-demand

!

interface Loopback0
description ROUTER-MGMT
ip address 172.30.255.98 255.255.255.255
!
interface Tunnel100
ip address x.x.254.98 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication DMVPN-I1
ip nhrp network-id 100
ip nhrp nhs xxx nbma xxx multicast
ip nhrp registration timeout 60
ip nhrp redirect
zone-member security DMVPN
ip tcp adjust-mss 1360
delay 1000
cdp enable
if-state nhrp
tunnel source Cellular0/2/0
tunnel mode gre multipoint
tunnel key 100
tunnel vrf INTERNET
tunnel protection ipsec profile DMVPN-PROFILE-1
!
interface Tunnel200
ip address x.x.253.98 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication DMVPN-I2
ip nhrp network-id 200
ip nhrp nhs xxx nbma xxx multicast
ip nhrp registration timeout 60
ip nhrp redirect
zone-member security DMVPN
ip tcp adjust-mss 1360
delay 1000
cdp enable
if-state nhrp
tunnel source Loopback0
tunnel mode gre multipoint
tunnel key 200
tunnel vrf INTERNET
tunnel protection ipsec profile DMVPN-PROFILE-2

!

!
router bgp 65500
bgp router-id x.x.255.98
bgp log-neighbor-changes
neighbor USA-HUB peer-group
neighbor USA-HUB remote-as 65500
neighbor USA-HUB timers 20 60
neighbor CAN-HUB peer-group
neighbor CAN-HUB remote-as 65500
neighbor CAN-HUB timers 20 60
neighbor x.x.253.254 peer-group RTH-HUB
neighbor x.x.254.254 peer-group MUC-HUB
!
address-family ipv4
bgp redistribute-internal
redistribute connected route-map RM-REDIST-CONNECTED-TO-BGP
neighbor USA-HUB send-community
neighbor USA-HUB weight 50000
neighbor USA-HUB next-hop-self all
neighbor USA-HUB soft-reconfiguration inbound
neighbor CAN-HUB send-community
neighbor CAN-HUB weight 50000
neighbor CAN-HUB next-hop-self all
neighbor CAN-HUB soft-reconfiguration inbound
neighbor x.x.253.254 activate
neighbor x.x.253.254 weight 50000
neighbor x.x.253.254 activate
neighbor x.x.254.254 weight 50000
distance bgp 201 19 250
exit-address-family
!
ip forward-protocol nd
ip ftp source-interface Loopback0
ip ftp username CAN-USER
ip ftp password 7 xxx
no ip http server
ip http authentication local
no ip http secure-server
ip route vrf INTERNET 0.0.0.0 0.0.0.0 Cellular0/2/0
ip route vrf INTERNET 0.0.0.0 0.0.0.0 Cellular0/2/1
ip tacacs source-interface Loopback0
!

line con 0
exec-timeout 0 0
login authentication CONSOLE
stopbits 1
line vty 0 4

[MHM Cisco World]

I do this lab and I success load sharing between two GRE tunnel.

[_|brt.drml|_]

I would say, create a loopback and source both tunnel from that interface.

As suggested above. 

[Joy3]

Hi @MHM Cisco World. Thanks so much for the example. However, I am still unable to load balance. When I changed the configuration as you suggested:

1. Cellular interfaces no longer receive an IP address (they did before)

2. SInce the tunnel is in mode multi-GRE/IP, I am not able to enter the tunnel destination.

My setup is such that, a dual-SIM LTE router connects to Tunnel100 and Tunnel200 via Cellular0/2/0 and Cellular0/2/1, respectively (like a router-on-a-stick type of topology). 

Thanks. 

[MHM Cisco World]

only one Q are the Hub have two different ISP or only one with two tunnel shared one ISP??

[Joy3]

The two cellular interfaces in my ISR Dual SIM LTE Router are each connected to different ISPs (O2 and Vodafone). Then I had configured each of the tunnel interfaces to have one of the cellular interfaces as the source. Kindly check above, I pasted the running config.

[MHM Cisco World]

this config of Spoke I talking about the Hub are Hub use two different Public IP ??

[Joy3]

Yes, they both use public IPs

[MHM Cisco World]

I review the issue many many times, 
let explain why I could not find the solution here, 
as mention above we can use LO for load balance between two ISP, but GRE far end peer must know LO (reachable) that in network that run private is easy only config the static or routing protocol advertise private IP in far end GRE.
here in your case the LO must be Public IP and can reachable from Hub and I think this is solution is hard in your case.
other solution is what you already config two tunnel each tunnel use tunnel source (cellular interface, which get IP form ISP)

case:-
1- cellular both UP UP 
the Spoke with this case have tunnel UP UP to Hub and since you config BGP using both Hub GRE end then you will get route behind the Hub from both tunnel and BGP best path select depend on criteria.
here you can load balance using both tunnel by
**Outbound using LP (local preference), the Hub advertise the many prefix and by using LP for specific prefix the Spoke can select one tunnel as best path and hence all Outbound toward this specific tunnel will forward via tunnel we select.

**Inbound using AS-Path Prepend, this make Spoke send AS-Prepend to Hub and Hub as also run BGP will select the Lowest As-Path and hence the Hub will forward traffic via tunnel we select.

2-cellular both UP Down
the Spoke in this case have only one Tunnel UP to Hub and BGP now have one  path.

here there is no load balance all traffic will forward via the tunnel UP.

[Joy3]

Hallo @MHM Cisco World. Thanks so much for the detailed explanation. I did more research but I was unsuccessful, so I forwarded it to the 3rd level support. Still waiting for their response.