Solved: Help with OSPF and DMVPN

paul_shiner · ‎01-08-2013

Hi All,

Wonder if someone could help me or point me in the right direction. Basically I'm trying to setup and learn different VPNs and routing protocols but I'm stuck setting up OSPF over DMVPN. EIGRP over DMVPN worked fine for me but then I am aware it's easier to setup.

Ok so basically I have a simple setup in GNS3 with 4 3725 routers. They have serial links and then a DMVPN setup with ip range 10.255.253.0/24, they have they own IP "LAN" subnets.

With EIGRP it all just works but just setting up the first OSPF area 0 for DMVPN (10.255.253.0/24) network on all devices fails. I setup ospf process with network statement on DMVPN hub router and then one spoke and it works fine, but as soon as I add another spoke router to the OSPF process, it constantly flaps between setting up a neighborship between the spoke routers. I see this on the routers:

(PS: I've tried fiddling with router-id's and priorities but nothing helps - HUB router is 10.10.10.10, spoke 1 is 1.1.1.1 and spoke 2 is 2.2.2.2)

HUB:

R2(config-if)#

*Mar 1 17:19:04.051: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from LOADING to FULL, Loading Done

R2(config-if)#

*Mar 1 17:19:06.375: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:06.627: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from LOADING to FULL, Loading Done

R2(config-if)#

*Mar 1 17:19:10.123: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:10.223: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from EXSTART to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:10.243: %OSPF-4-NONEIGHBOR: Received database description from unknown neighbor 1.1.1.1

R2(config-if)#

*Mar 1 17:19:10.335: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from LOADING to FULL, Loading Done

R2(config-if)#

*Mar 1 17:19:13.707: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:13.975: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from LOADING to FULL, Loading Done

R2(config-if)#

*Mar 1 17:19:16.439: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:16.551: %OSPF-4-NONEIGHBOR: Received database description from unknown neighbor 1.1.1.1

R2(config-if)#

*Mar 1 17:19:16.623: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from LOADING to FULL, Loading Done

R2(config-if)#

*Mar 1 17:19:20.111: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:20.223: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from EXSTART to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:20.327: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from LOADING to FULL, Loading Done

R2(config-if)#

*Mar 1 17:19:23.719: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:19:23.863: %OSPF-4-NONEIGHBOR: Received database description from unknown neighbor 2.2.2.2

R2(config-if)#

*Mar 1 17:19:24.007: %OSPF-5-ADJCHG: Process 100, Nbr 1.1.1.1 on Tunnel100 from LOADING to FULL, Loading Done

Spoke1:

DMVPNSPOKE01(config-if)#

*Mar 1 17:18:39.067: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE01(config-if)#

*Mar 1 17:18:51.243: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE01(config-if)#

*Mar 1 17:19:01.239: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE01(config-if)#

*Mar 1 17:19:11.179: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE01(config-if)#

*Mar 1 17:19:21.263: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

Spoke2:

DMVPNSPOKE02(config-router)#

*Mar 1 17:18:50.799: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE02(config-router)#

*Mar 1 17:18:54.567: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE02(config-router)#

*Mar 1 17:19:00.831: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE02(config-router)#

*Mar 1 17:19:04.499: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE02(config-router)#

*Mar 1 17:19:10.835: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE02(config-router)#

*Mar 1 17:19:14.499: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from LOADING to FULL, Loading Done

DMVPNSPOKE02(config-router)#no network 10.255.253.0 0.0.0.255 area 0

DMVPNSPOKE02(config-router)#

*Mar 1 17:19:19.139: %OSPF-5-ADJCHG: Process 100, Nbr 10.10.10.10 on Tunnel100 from EXSTART to DOWN, Neighbor Down: Interface down or detached

and if it helps I enabled "debug ip ospf adj" on the hub and go this when two spoke routers are added to OSPF:

OSPF adjacency events debugging is on

R2(config-if)#

*Mar 1 17:01:46.875: OSPF: 192.168.1.1 address 10.255.253.1 on Tunnel100 is dead, state DOWN

*Mar 1 17:01:46.879: %OSPF-5-ADJCHG: Process 100, Nbr 192.168.1.1 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:01:46.935: OSPF: 192.168.2.1 address 10.255.253.2 on Tunnel100 is dead, state DOWN

*Mar 1 17:01:46.939: %OSPF-5-ADJCHG: Process 100, Nbr 192.168.2.1 on Tunnel100 from INIT to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:01:46.943: OSPF: 2 Way Communication to 192.168.1.1 on Tunnel100, state 2WAY

*Mar 1 17:01:46.943: OSPF: Send DBD to 192.168.1.1 on Tunnel100 seq 0x850 opt 0x52 flag 0x7 len 32

R2(config-if)#

*Mar 1 17:01:46.955: %OSPF-4-NONEIGHBOR: Received database description from unknown neighbor 192.168.2.1

*Mar 1 17:01:47.019: OSPF: 192.168.1.1 address 10.255.253.1 on Tunnel100 is dead, state DOWN

*Mar 1 17:01:47.023: %OSPF-5-ADJCHG: Process 100, Nbr 192.168.1.1 on Tunnel100 from EXSTART to DOWN, Neighbor Down: Adjacency forced to reset

R2(config-if)#

*Mar 1 17:01:47.027: OSPF: 2 Way Communication to 192.168.2.1 on Tunnel100, state 2WAY

*Mar 1 17:01:47.027: OSPF: Send DBD to 192.168.2.1 on Tunnel100 seq 0x3E7 opt 0x52 flag 0x7 len 32

*Mar 1 17:01:47.383: OSPF: Build router LSA for area 0, router ID 172.16.20.10, seq 0x80000036

R2(config-if)#

*Mar 1 17:01:51.931: OSPF: Rcv DBD from 192.168.2.1 on Tunnel100 seq 0x1808 opt 0x52 flag 0x7 len 32 mtu 1400 state EXSTART

*Mar 1 17:01:51.935: OSPF: NBR Negotiation Done. We are the SLAVE

*Mar 1 17:01:51.939: OSPF: Send DBD to 192.168.2.1 on Tunnel100 seq 0x1808 opt 0x52 flag 0x2 len 92

*Mar 1 17:01:51.975: %OSPF-4-NONEIGHBOR: Received database description from unknown neighbor 192.168.1.1

R2(config-if)#

*Mar 1 17:01:51.979: OSPF: Rcv DBD from 192.168.2.1 on Tunnel100 seq 0x1809 opt 0x52 flag 0x3 len 92 mtu 1400 state EXCHANGE

*Mar 1 17:01:51.983: OSPF: Send DBD to 192.168.2.1 on Tunnel100 seq 0x1809 opt 0x52 flag 0x0 len 32

*Mar 1 17:01:52.047: OSPF: Rcv DBD from 192.168.2.1 on Tunnel100 seq 0x180A opt 0x52 flag 0x1 len 32 mtu 1400 state EXCHANGE

*Mar 1 17:01:52.047: OSPF: Exchange Done with 192.168.2.1 on Tunnel100

*Mar 1 17:01:52.051: OSPF: Send LS REQ to 192.168.2.1 length 12 LSA count 1

*Mar 1 17:01:52.055: OSPF: Send DBD to 192.168.2.1 on Tunnel100 seq 0x180A opt 0x52 flag 0x0 len 32

*Mar 1 17:01:52.063: OSPF: Rcv LS REQ from 192.168.2.1 on Tunnel100 length 48 LSA count 2

*Mar 1 17:01:52.067: OSPF: Send UPD to 10.255.253.2 on Tunnel100 length 112 LSA count 2

*Mar 1 17:01:52.095: OSPF: Rcv LS UPD from 192.168.2.1 on Tunnel100 length 76 LSA count 1

*Mar 1 17:01:52.099: OSPF: Synchronized with 192.168.2.1 on Tunnel100, state FULL

*Mar 1 17:01:52.099: %OSPF-5-ADJCHG: Process 100, Nbr 192.168.2.1 on Tunnel100 from LOADING to FULL, Loading Done

*Mar 1 17:01:52.515: OSPF: 192.168.2.1 address 10.255.253.2 on Tunnel100 is dead, state DOWN

*Mar 1 17:01:52.519: %OSPF-5-ADJCHG: Process 100, Nbr 192.168.2.1 on Tunnel100 from FULL to DOWN, Neighbor Down: Adjacency forced to reset

*Mar 1 17:01:52.523: OSPF: 2 Way Communication to 192.168.1.1 on Tunnel100, state 2WAY

*Mar 1 17:01:52.523: OSPF: Send DBD to 192.168.1.1 on Tunnel100 seq 0x623 opt 0x52 flag 0x7 len 32

R2(config-if)#

*Mar 1 17:01:56.959: OSPF: Rcv DBD from 192.168.1.1 on Tunnel100 seq 0x2448 opt 0x52 flag 0x7 len 32 mtu 1400 state EXSTART

*Mar 1 17:01:56.963: OSPF: NBR Negotiation Done. We are the SLAVE

*Mar 1 17:01:56.967: OSPF: Send DBD to 192.168.1.1 on Tunnel100 seq 0x2448 opt 0x52 flag 0x2 len 92

*Mar 1 17:01:57.055: OSPF: Rcv DBD from 192.168.1.1 on Tunnel100 seq 0x2449 opt 0x52 flag 0x3 len 92 mtu 1400 state EXCHANGE

*Mar 1 17:01:57.059: OSPF: Send DBD to 192.168.1.1 on Tunnel100 seq 0x2449 opt 0x52 flag 0x0 len 32

*Mar 1 17:01:57.127: OSPF: Rcv DBD from 192.168.1.1 on Tunnel100 seq 0x244A opt 0x52 flag 0x1 len 32 mtu 1400 state EXCHANGE

*Mar 1 17:01:57.131: OSPF: Exchange Done with 192.168.1.1 on Tunnel100

*Mar 1 17:01:57.135: OSPF: Send LS REQ to 192.168.1.1 length 12 LSA count 1

*Mar 1 17:01:57.139: OSPF: Send DBD to 192.168.1.1 on Tunnel100 seq 0x244A opt 0x52 flag 0x0 len 32

*Mar 1 17:01:57.147: OSPF: Rcv LS REQ from 192.168.1.1 on Tunnel100 length 48 LSA count 2

*Mar 1 17:01:57.155: OSPF: Send UPD to 10.255.253.1 on Tunnel100 length 100 LSA count 2

*Mar 1 17:01:57.199: OSPF: Rcv LS UPD from 192.168.1.1 on Tunnel100 length 76 LSA count 1

*Mar 1 17:01:57.203: OSPF: Synchronized with 192.168.1.1 on Tunnel100, state FULL

*Mar 1 17:01:57.203: %OSPF-5-ADJCHG: Process 100, Nbr 192.168.1.1 on Tunnel100 from LOADING to FULL, Loading Done

*Mar 1 17:01:57.711: OSPF: Build router LSA for area 0, router ID 172.16.20.10, seq 0x80000037

*Mar 1 17:01:57.783: OSPF: Rcv LS UPD from 192.168.1.1 on Tunnel100 length 88 LSA count 1

As soon as I take off the network statement on the secondary spoke all comes back to life and stable for hub and spoke 1.

Please help if you can.

Thanks,

Paul

Peter Paluch · ‎01-08-2013

Hello Paul,

The DMVPN cloud interconnecting your OSPF routers is technically an NBMA network, and in addition, spokes usually have their relation created only towards the hub router, so the proper OSPF network type would be Point-to-Multipoint for me. How exactly did you configure the OSPF on your routers? Can you post the configuration of the Tunnel interface and of the OSPF process on each on your router?

Best regards,

Peter

View solution in original post

Giuseppe Larosa · ‎01-08-2013

Hello Paul,

interface tunnel 100

ip ospf network point-to-multipoint

do it on all routers

see

http://www.cisco.com/en/US/docs/ios-xml/ios/iproute_ospf/command/ospf-i1.html#GUID-A6B2AE48-8BCF-4BBC-BE31-6F3649D1D1E2

Hope to help

Giuseppe

View solution in original post

Peter Paluch · ‎01-12-2013

Hello Paul,

I hope this message still reaches you even though you consider the case to be solved.

I must correct myself - I was not correct with the suggested solution, or, to be more precise, I was correct within a particular environment but not in general - although, again, there is no general correct solution... Oh, all right, I know this is confusing Let me explain.

First of all, I was wrong in the assessment that OSPF will treat the GRE multipoint tunnel interface as a NBMA interface. While the GRE multipoint interface is technically a NBMA network, OSPF still considers the multipoint tunnel interface as a point-to-point OSPF type network. I've just replicated your configuration and voila:

R1-Hub(config-router)#do show ip ospf int tun1

Tunnel1 is up, line protocol is up

Internet Address 10.255.253.10/24, Area 0

Process ID 100, Router ID 10.10.10.10, Network Type POINT_TO_POINT, Cost: 11111

The intense flapping after the second spoke came up is logical, then: the hub expected just a single OSPF peer on the tunnel interface. After the second OSPF peer came up, the OSPF at the hub router got extremely confused - a second OSPF speak tries to establish an adjacency although there already is a full adjacency with another router, and there shall be only one adjacency on a point-to-point interface whatsoever! So the flapping is explained very easily here.

My second imprecise judgment was in my recommendation to configure all multipoint GRE interfaces as point-to-multipoint. This will take more time to explain.

As I explained earlier, the DMVPN "cloud" represented by a multipoint GRE tunnel interface between all the routers is technically a NBMA network, meaning that

multicasts have to be replicated by a node attached to this network as they will not be replicated by the network itself (non-broadcast)
you can reach multiple neighbors through a single interface (multiaccess)

OSPF has an identically called network type - non-broadcast - in which OSPF can run very nicely if a couple of prerequisites is met:

Full connectivity between all nodes is assumed. If partial connectivity exists (i.e. hub and spoke), the OSPF has to be tweaked extensively to work correctly (DR/BDR elections must be modified, next hop reachability must be solved because the routing tables as constructed by OSPF will assume each router can reach each other directly). In some cases of partial connectivity (namely, when no node has direct reachability to other nodes), this type of OSPF network cannot be used at all.
On NBMA networks, OSPF requires that neighbors are defined statically using neighbor commands. OSPF will not try to establish adjacencies over NBMA networks using multicast packets. All OSPF communication over NBMA network is unicast only.

The 1st prerequisite in DMVPN is met nicely if running DMVPN Phase2 (spoke-to-spoke tunnels) or Phase3 (spoke-to-spoke tunnels with hub-controlled redirects between spokes). However, the 2nd prerequisite is not what we like to see in DMVPN - it requires additional static configuration on all routers which goes contrary to the dynamic nature of DMVPN and makes things too tied.

Therefore, if running DMVPN Phase 2 (the version of DMVPN that supports spoke-to-spoke tunnels but the routing tables must be constructed so that routes to networks behind spokes are pointing to the appropriate spokes and not to the hub), the recommended OSPF network type is actually broadcast. This will work because of the ip nhrp map multicast command that allows spokes to send OSPF multicast traffic to hubs, and the hub to send OSPF multicast traffic to all spokes. In adition, OSPF priorities on all spokes must be set to 0 because the only router that can hear each other router is the hub router, so this is the only router allowed to be the Designated Router. There is no Backup Designated Router here. This is very similar to a hub-and-spoke configuration of OSPF over Frame Relay. By the way, your configuration is of the Phase2 type.

What I suggested to you earlier, namely the OSPF network type point-to-multipoint, works, however, there is a gotcha: in this network type, OSPF builds adjacencies only between routers that can hear each other's Hello packets. In DMVPN, the adjacencies will be built between the hub and spokes only, not spoke-to-spoke. The result of these adjacencies, however, is that on each spoke, the routing table points to the hub to reach other spokes and their networks. This routing table does not meet the fundamental requirement - seeing the other spoke's network using the next hop of that spoke - for the DMVPN Phase 2 to build spoke-to-spoke tunnels, and as a result, the spoke-to-spoke traffic will flow through the hub.

However, and this is where I need to get to, there is a so-called DMVPN Phase3. This DMVPN version actually improves the way spoke-to-spoke tunnels are created. In Phase3, the routing tables on spoke routers are actually supposed to point to the hub! However, in the moment a spoke sends a packet to another spoke through the hub, the hub will send a NHRP Redirect message to the originating spoke, informing it about the shortcut directly to the other spoke. This triggers the creation of the spoke-to-spoke tunnel. Note that for this setup, the OSPF network point-to-multipoint is ideal. So what I suggested at my very beginning - namely the point-to-multipoint network type - is the correct configuration for DMVPN Phase3 but not for DMVPN Phase2.

To sum the differences - the OSPF network type broadcast is suitable both for DMVPN Phase2 and Phase3 (though for Phase3, it is not recommended - but it still results into spoke-to-spoke tunnels). The OSPF network type point-to-multipoint is suitable for DMVPN Phase3 only, because in Phase2, it won't allow creation of spoke-to-spoke tunnels.

Read more about the DMVPN Phase3 here:

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6660/ps6808/prod_white_paper0900aecd8055c34e_ps6658_Products_White_Paper.html

http://www.cisco.com/en/US/docs/ios/12_4t/12_4t11/ht_nhrp.html

I apologize for the confusion I've created earlier. I must have had some brain blindness or something - I simply focused on one particular way of thinking, not realizing it is not appropriate for your particular situation...

In any way, I hope this post reaches you. Feel welcome to ask about anything I wasn't clear enough!

Best regards,

Peter

View solution in original post

Peter Paluch · ‎01-08-2013

Hello Paul,

The DMVPN cloud interconnecting your OSPF routers is technically an NBMA network, and in addition, spokes usually have their relation created only towards the hub router, so the proper OSPF network type would be Point-to-Multipoint for me. How exactly did you configure the OSPF on your routers? Can you post the configuration of the Tunnel interface and of the OSPF process on each on your router?

Best regards,

Peter

paul_shiner · ‎01-08-2013

Thank you for your responses (Peter and Giuseppe):

I think I might have issue with unwanted recursive routing as my hunch now Giuseppe has mentioned it, but here is more information on my setup:

Hub:

interface Tunnel100

ip address 10.255.253.10 255.255.255.0

no ip redirects

ip mtu 1400

ip nhrp map multicast dynamic

ip nhrp network-id 1

ip tcp adjust-mss 1360

ip ospf priority 255

tunnel source 80.242.73.2

tunnel mode gre multipoint

end

R2#sh run | begin router ospf 100

router ospf 100

router-id 10.10.10.10

log-adjacency-changes

network 10.255.253.0 0.0.0.255 area 0

network 20.20.20.0 0.0.0.255 area 0

!

Spoke 1:

interface Tunnel100

ip address 10.255.253.1 255.255.255.0

no ip redirects

ip mtu 1400

ip nhrp map 10.255.253.10 80.242.73.2

ip nhrp map multicast 80.242.73.2

ip nhrp network-id 1

ip nhrp nhs 10.255.253.10

ip tcp adjust-mss 1360

ip ospf priority 10

tunnel source 100.100.100.2

tunnel mode gre multipoint

end

DMVPNSPOKE01#sh run | begin router ospf 100

router ospf 100

router-id 1.1.1.1

log-adjacency-changes

network 10.255.253.0 0.0.0.255 area 0

network 192.168.0.0 0.0.7.255 area 0

!

Spoke 2 (network command removed for DMVPN due to issue):

interface Tunnel100

ip address 10.255.253.2 255.255.255.0

no ip redirects

ip mtu 1400

ip nhrp map 10.255.253.10 80.242.73.2

ip nhrp map multicast 80.242.73.2

ip nhrp network-id 1

ip nhrp nhs 10.255.253.10

ip tcp adjust-mss 1360

tunnel source 100.100.100.6

tunnel mode gre multipoint

end

DMVPNSPOKE02#sh run | begin router ospf 100

router ospf 100

router-id 2.2.2.2

log-adjacency-changes

network 192.168.0.0 0.0.7.255 area 0

!

Spoke 3 (again no network command):

interface Tunnel100

ip address 10.255.253.3 255.255.255.0

no ip redirects

ip mtu 1400

ip nhrp map 10.255.253.10 80.242.73.2

ip nhrp map multicast 80.242.73.2

ip nhrp network-id 1

ip nhrp nhs 10.255.253.10

ip tcp adjust-mss 1360

tunnel source 100.100.100.10

tunnel mode gre multipoint

end

DMVPNSPOKE03#sh run | begin router ospf 100

router ospf 100

router-id 3.3.3.3

log-adjacency-changes

network 192.168.0.0 0.0.7.255 area 0

!

The setup is acutally more complicated that just 4 routers so here is the topology attached.

The DMVPN HUB (R2) is actually two routers away amougst some other tunnels and routing I was setting up. Although disabling the GRE tunnel between R1 and R2 didn't help. Note - the picture is from working EIGRP setup so ingore some things on the drawing...

Oh and the subnets for connections between R1 and spokes are 100.100.100.0/30 .. 100.100.100.4/30 and 100.100.100.8/30

Thanks for your help guys.

Giuseppe Larosa · ‎01-08-2013

Hello Paul,

>> The DMVPN HUB (R2) is actually two routers away amougst some other tunnels and routing I was setting up.

I would simplify the network topology avoiding the p2p GRE tunnel between R1 and R2 the DMVPN hub.

Try to add a direct link between R1 and R2 to make things simpler have on R2 static routes or EIGRP 1 routes for the spoke public addresses 100.100.100.x

have EIGRP1 or static routes on DMVPN spoke routers for 80.242.73.2 address.

This is to avoid to have mGRE traffic to be encapsulated again in GRE packets, this is a very peculiar setup.

However, what is really strange is that if a true GRE over GRE encapsulation problem would be happening here, I would expect also the EIGRP based DMVPN to fail.

Instead in your case you see instability only when using OSPF and when adding the second spoke.

Your mGRE tunnels configuration and OSPF configuration look like fine and I don't see the potential for recursive routing that was my first guess.

I would try with the changes proposed above.

Hope to help

Giuseppe

paul_shiner · ‎01-08-2013

I'll response to this at the bottom as this thread is getting muddled up

Giuseppe Larosa · ‎01-08-2013

Hello Paul,

in addition to Peter's notes the behaviour you have described looks like the effects of unwanted recursive routing, that is it is likely that the third device is advertising in OSPF over the p2mp GRE tunnel the "public" IP addresses, that are the addresses to be used for GRE encapsulation and managed by NHRP.

If this happens the tunnel dies because it tries to use the tunnel to reach the tunnel destination address (in this case provided by NHRP resolution) and OSPF flaps down. As OSPF adjacency fails the tunnel connectivity is restored and then again when OSPF is up the tunnel fails and this becomes a loop of events.

As Peter has noted post a network diagram and the configurations of the three routers.

With three routers you should be able to use OSPF over DMVPN even with default network type broadcast.

So in my opinion it is not the network type that is causing what you see.

Hope to help

Giuseppe

paul_shiner · ‎01-08-2013

Thanks for response giuseppe, please see my response above.

paul_shiner · ‎01-08-2013

Yea my next step was going to simplify but I started of with simple site to site that then moved into a GRE for dymanic routing, and was just learning a few things. I wanted to expand and make it a little more complicated so I could mess around with sub optimal routing etc. As it worked with EIGRP I didn't expect OSPF not to work AT ALL lol

I turned off the GRE tunnel between R1 and R2 so can't be double encapsulation of GRE packets can it?

I forgot to mention that the R1, ISP and R2 routers are running RIPv2 for basic routing between each other... could that be causing an issue?

Thanks again for your replies.

paul_shiner · ‎01-08-2013

OK an update - Removing ISP, RIP etc and just running a serial link between R1 and R2 with some basic default static routes ..... It STILL doesn't work and flaps as soon as you add another OSPF neighbor. (Exact same symptoms)

Any other ideas? Have I hit some sort of IOS/GNS3 bug?

Cheers,

Paul

Giuseppe Larosa · ‎01-08-2013

Hello Paul,

what IOS image is running on the emulated C3725 devices?

>> Have I hit some sort of IOS/GNS3 bug?

There might be a chance for this

I have tested DMVPN in the past with real network devices and even without changing the OSPF network type I haven't faced a similar issue.

Hope to help

Giuseppe

paul_shiner · ‎01-08-2013

C:\Users\shipa01\GNS3\Images\c3725-adventerprisek9-mz.124-12.image

What network type should I change to just for testing really...

Giuseppe Larosa · ‎01-08-2013

Hello Paul,

interface tunnel 100

ip ospf network point-to-multipoint

do it on all routers

see

http://www.cisco.com/en/US/docs/ios-xml/ios/iproute_ospf/command/ospf-i1.html#GUID-A6B2AE48-8BCF-4BBC-BE31-6F3649D1D1E2

Hope to help

Giuseppe

paul_shiner · ‎01-09-2013

This resolved it completely. So many thanks to Giseppe and Peter who mentioned that would be the network type for him originally.

Definitely worth remembering this one

Peter Paluch · ‎01-12-2013

Hello Paul,

I hope this message still reaches you even though you consider the case to be solved.

I must correct myself - I was not correct with the suggested solution, or, to be more precise, I was correct within a particular environment but not in general - although, again, there is no general correct solution... Oh, all right, I know this is confusing Let me explain.

First of all, I was wrong in the assessment that OSPF will treat the GRE multipoint tunnel interface as a NBMA interface. While the GRE multipoint interface is technically a NBMA network, OSPF still considers the multipoint tunnel interface as a point-to-point OSPF type network. I've just replicated your configuration and voila:

R1-Hub(config-router)#do show ip ospf int tun1

Tunnel1 is up, line protocol is up

Internet Address 10.255.253.10/24, Area 0

Process ID 100, Router ID 10.10.10.10, Network Type POINT_TO_POINT, Cost: 11111

The intense flapping after the second spoke came up is logical, then: the hub expected just a single OSPF peer on the tunnel interface. After the second OSPF peer came up, the OSPF at the hub router got extremely confused - a second OSPF speak tries to establish an adjacency although there already is a full adjacency with another router, and there shall be only one adjacency on a point-to-point interface whatsoever! So the flapping is explained very easily here.

My second imprecise judgment was in my recommendation to configure all multipoint GRE interfaces as point-to-multipoint. This will take more time to explain.

As I explained earlier, the DMVPN "cloud" represented by a multipoint GRE tunnel interface between all the routers is technically a NBMA network, meaning that

multicasts have to be replicated by a node attached to this network as they will not be replicated by the network itself (non-broadcast)
you can reach multiple neighbors through a single interface (multiaccess)

OSPF has an identically called network type - non-broadcast - in which OSPF can run very nicely if a couple of prerequisites is met:

Full connectivity between all nodes is assumed. If partial connectivity exists (i.e. hub and spoke), the OSPF has to be tweaked extensively to work correctly (DR/BDR elections must be modified, next hop reachability must be solved because the routing tables as constructed by OSPF will assume each router can reach each other directly). In some cases of partial connectivity (namely, when no node has direct reachability to other nodes), this type of OSPF network cannot be used at all.
On NBMA networks, OSPF requires that neighbors are defined statically using neighbor commands. OSPF will not try to establish adjacencies over NBMA networks using multicast packets. All OSPF communication over NBMA network is unicast only.

The 1st prerequisite in DMVPN is met nicely if running DMVPN Phase2 (spoke-to-spoke tunnels) or Phase3 (spoke-to-spoke tunnels with hub-controlled redirects between spokes). However, the 2nd prerequisite is not what we like to see in DMVPN - it requires additional static configuration on all routers which goes contrary to the dynamic nature of DMVPN and makes things too tied.

Therefore, if running DMVPN Phase 2 (the version of DMVPN that supports spoke-to-spoke tunnels but the routing tables must be constructed so that routes to networks behind spokes are pointing to the appropriate spokes and not to the hub), the recommended OSPF network type is actually broadcast. This will work because of the ip nhrp map multicast command that allows spokes to send OSPF multicast traffic to hubs, and the hub to send OSPF multicast traffic to all spokes. In adition, OSPF priorities on all spokes must be set to 0 because the only router that can hear each other router is the hub router, so this is the only router allowed to be the Designated Router. There is no Backup Designated Router here. This is very similar to a hub-and-spoke configuration of OSPF over Frame Relay. By the way, your configuration is of the Phase2 type.

What I suggested to you earlier, namely the OSPF network type point-to-multipoint, works, however, there is a gotcha: in this network type, OSPF builds adjacencies only between routers that can hear each other's Hello packets. In DMVPN, the adjacencies will be built between the hub and spokes only, not spoke-to-spoke. The result of these adjacencies, however, is that on each spoke, the routing table points to the hub to reach other spokes and their networks. This routing table does not meet the fundamental requirement - seeing the other spoke's network using the next hop of that spoke - for the DMVPN Phase 2 to build spoke-to-spoke tunnels, and as a result, the spoke-to-spoke traffic will flow through the hub.

However, and this is where I need to get to, there is a so-called DMVPN Phase3. This DMVPN version actually improves the way spoke-to-spoke tunnels are created. In Phase3, the routing tables on spoke routers are actually supposed to point to the hub! However, in the moment a spoke sends a packet to another spoke through the hub, the hub will send a NHRP Redirect message to the originating spoke, informing it about the shortcut directly to the other spoke. This triggers the creation of the spoke-to-spoke tunnel. Note that for this setup, the OSPF network point-to-multipoint is ideal. So what I suggested at my very beginning - namely the point-to-multipoint network type - is the correct configuration for DMVPN Phase3 but not for DMVPN Phase2.

To sum the differences - the OSPF network type broadcast is suitable both for DMVPN Phase2 and Phase3 (though for Phase3, it is not recommended - but it still results into spoke-to-spoke tunnels). The OSPF network type point-to-multipoint is suitable for DMVPN Phase3 only, because in Phase2, it won't allow creation of spoke-to-spoke tunnels.