Re: DMVPN two mgre tunnels from one interface

Dmitry Podtikhov · ‎10-25-2010

Hello all

I want to understand dual hub dual cloud dmvpn setup.

I have several offices with two ISPs in each office. Offices are connected via mpls vpns with each other. We are not running any routing protocol between CE and PE routers. The idea is to implement robust routing with good failover between our sites.

I am looking at phase 3 DMVPN, but still have some questions.

First of all, we do not need encryption, so we just have pure mGRE tunnels with OSPF inside.

I want my setup to be as following:

Main ISP VPN - two hubs, each is separate DMVPN network, with corresponding OSPF cost. No encryption. Hubs are in different geographical locations.

Secondary ISP VPN - two hubs, each is separate DMVPN network, with corresponding OSPF cost. No encryption. Hubs are in different geographical locations.

There is no problem to set one hub in each isp network, but when i am trying to add second DMVPN network in each ISP cloud, OSPF neighbourship does not come up.

I suppose it is because two mgre tunnels are sourced from one physical interface (pointing to the ISP). According to the docs it is possible to source two tunnels from one interface with tunnel protection ipsec profile xxx shared keyword but what about my case with no crypto?

Leo Laohoo · ‎10-25-2010

As long as your tunnel destination IP addresses are different.

Dmitry Podtikhov · ‎10-25-2010

But my tunnel destination is gre multipoint

wzhang · ‎10-25-2010

Hi,

The "tunnel protection... shared" command is only applicable to crypto. It makes the SADB shared between the 2 tunnel interfaces sourced from the same physical interface since the you can't distinguish between the 2 tunnels when packets arrive on the physical interface prior to decryption. This command should not be needed if you are not using ipsec to protect the mGRE tunnels. If you are only running ospf in the overlay dmvpn network, then it shouldn't really matter if the 2 tunnels are sourced off of the same physical interface. So with your configuration, the 2 dmvpn tunnels work as far as ip goes (connectivity on both tunnels), but it's only a problem with OSPF neighbors coming up on both interfaces?

Thanks,

Wen

Dmitry Podtikhov · ‎10-26-2010

Thank you for your replies.

I'll try to explain further. I have a lab scenario in dynamips with two ISPs and four offices.

My goal is to reach good redundancy by setting up redundant DMVPN clouds with OSPF routing. Let's consider for simplicity that I have one ISP and I want to set up two hubs in my network for redundancy.

Here is the config of main hub mGRE tunnel

interface Tunnel0
description main cloud
ip address 10.7.0.1 255.255.255.0
no ip redirects
ip mtu 1476
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf cost 1000
tunnel source FastEthernet0/1
tunnel mode gre multipoint
tunnel key 0

Spokes are configured with their tunnels

interface Tunnel0
ip address 10.7.0.4 255.255.255.0
no ip redirects
ip mtu 1476
ip nhrp map multicast 10.5.174.1
ip nhrp map 10.7.0.1 10.5.174.1
ip nhrp network-id 1
ip nhrp nhs 10.7.0.1
ip nhrp shortcut
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf cost 1000
tunnel source FastEthernet0/1
tunnel mode gre multipoint
tunnel key 0

At this point all is going fine. I have one DMVPN cloud, OSPF neighbour relationship is forming ok.

Next, i add second hub router ( in the same mpls vpn, tunnel sourced from the same physical interface)

interface Tunnel1
description REZ HUB IN MAIN CLOUD
ip address 10.7.1.1 255.255.255.0
no ip redirects
ip mtu 1476
ip nhrp map multicast dynamic
ip nhrp network-id 2
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf cost 2000
tunnel source FastEthernet0/1
tunnel mode gre multipoint
tunnel key 1

and other routers are configured as the spokes, e.g

interface Tunnel1
ip address 10.7.1.3 255.255.255.0
no ip redirects
ip mtu 1476
ip nhrp map multicast 10.166.254.1
ip nhrp map 10.7.1.1 10.166.254.1
ip nhrp network-id 2
ip nhrp nhs 10.7.1.1
ip nhrp shortcut
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf cost 2000
tunnel source FastEthernet0/1
tunnel mode gre multipoint
tunnel key 1

Now let's look at the main hub

#sh ip ospf n
Neighbor ID     Pri   State           Dead Time   Address         Interface
10.175.0.1        0   FULL/ -        00:01:40    10.7.0.3        Tunnel0
192.168.74.1      0   FULL/ -        00:01:48    10.7.0.2        Tunnel0
10.166.254.1      0   FULL/ -        00:01:30    10.7.0.4        Tunnel0

I can see all spokes are neighbours with hub. All is fine

Looking at the secondary hub

#sh ip ospf n

Neighbor ID     Pri   State           Dead Time   Address         Interface
10.175.0.1        0   FULL/ -        00:01:33    10.7.1.2        Tunnel1
192.168.74.1      0   INIT/ -        00:01:41    10.7.1.3        Tunnel1
10.7.0.1          0   FULL/ -        00:01:30    10.7.0.1        Tunnel0

Well, we can see that it has formed an adjacency with main hub router (tunnel0). And with one spoke. But one spoke is stuck in the init state.

OSPF and mGRE configs are identical on the spokes (except the tunnel ip addresses).

From the spokes i can see that one spoke formed an adjacency with both hubs and second spoke only with main hub.

Also I am not able to ping secondary hub logical address from one spoke and able to ping from another.

Here are some more outputs:

From the working spoke:

#sh ip route 10.7.1.1
Routing entry for 10.7.1.1/32
Known via "ospf 1", distance 110, metric 2000, type intra area
Redistributing via ospf 59
Advertised by ospf 59 metric-type 1 subnets
Last update from 10.7.1.1 on Tunnel1, 00:04:28 ago
Routing Descriptor Blocks:
    10.7.1.1, from 10.166.254.1, 00:04:28 ago, via Tunnel1
      Route metric is 2000, traffic share count is 1
* 10.7.0.1, from 10.166.254.1, 00:04:28 ago, via Tunnel0
      Route metric is 2000, traffic share count is 1

#sh ip nhrp 10.7.1.1
10.7.1.1/32 via 10.7.1.1, Tunnel1 created 00:58:09, never expire
Type: static, Flags: used
NBMA address: 10.166.254.1

#sh ip cef 10.7.1.1
10.7.1.1/32, version 61, epoch 0, per-destination sharing
0 packets, 0 bytes
via 10.7.1.1, Tunnel1, 0 dependencies
    traffic share 1
    next hop 10.7.1.1, Tunnel1
    valid adjacency
via 10.7.0.1, Tunnel0, 0 dependencies
    traffic share 1
    next hop 10.7.0.1, Tunnel0
    valid adjacency
0 packets, 0 bytes switched through the prefix
tmstats: external 0 packets, 0 bytes
           internal 0 packets, 0 bytes

From the not working spoke:

#sh ip route 10.7.1.1
Routing entry for 10.7.1.1/32
Known via "ospf 1", distance 110, metric 2000, type intra area
Redistributing via ospf 43
Advertised by ospf 43 metric-type 1 subnets
Last update from 10.7.0.1 on Tunnel0, 00:02:47 ago
Routing Descriptor Blocks:
* 10.7.0.1, from 10.166.254.1, 00:02:47 ago, via Tunnel0
Route metric is 2000, traffic share count is 1

#sh ip nhrp 10.7.1.1
10.7.1.1/32 via 10.7.1.1, Tunnel1 created 00:59:46, never expire
Type: static, Flags: used
NBMA address: 10.166.254.1

#sh ip cef 10.7.1.1
10.7.1.1/32, version 49, epoch 0
0 packets, 0 bytes
via 10.7.0.1, Tunnel0, 0 dependencies
next hop 10.7.0.1, Tunnel0
valid adjacency

wzhang · ‎10-27-2010

Hi,

With ospf running over DMVPN, you want to configure the ospf network type to be "broadcast" instead of "point-to-multipoint" so that ospf won't try to install /32 host routes for all the spoke's tunnel addresses. I believe it's the same problem that you are seeing here. What you want to do is to change your configuration to broadcast, and use ospf priority to make sure only the hubs can become DR and BDRs. For more details, see http://www.cisco.com/en/US/tech/tk583/tk372/technologies_white_paper09186a008018983e.shtml#ospf.

Thanks,

Wen

Dmitry Podtikhov · ‎11-07-2010

Sorry for such long silence.

I have managed to set up the whole scenario.There were some mistakes in my configs, some tunnels don't have tunnel keys, several static routes were missing etc..

Two VPN providers - a pair of hubs in each VPN ( four dmvpn clouds totally), OSPF metric is used to switch over main/secondary hubs.

As for point-to-multipoint network type, I suppose it is a requirement for phase 3 DMVPN, and I think it is more scalable and perhaps more stable than phase 2 (e.g. hub is not overloaded by answering NHRP replies from the spokes by sending redirect messages)

But there is one thing that you have mentioned - OSPF installs /32 host routes for all the spoke's hub addresses.

So, suppose I have two mgre tunnels on one router, one to main hub, other for the secondary hub. I see /32 address of the secondary hub reachable via the main tunnel, so I suppose that all my nhrp conversations are going through my main mGRE tunnel. Is it normal behaiviour?

When main hub fails, router switches to the second tunnel normally (but i think there is an extra delay for routing protocol to converge, to rebuild CEF entries etc.)

For my scenario with four mgre tunnels, three of the hub addresses are reachable via the tunnel with best metric, it is a bit confusing.

wzhang · ‎11-08-2010

Hi,

You are right, with DMVPN phase3 you will no longer have to configure the OSPF network type to be broadcast given the NHRP shortcut capabilities. What you had observed with the /32 behavior is expected, and is probably something you want to avoid with a dual/multi DMVPN setup. You can use the OSPF prefix suppression feature to achieve that, see http://www.cisco.com/en/US/partner/docs/ios/12_4t/12_4t15/ht_osmch.html.

Thanks,

Wen