Re: debugging DMVPN

mwertejuk · ‎09-28-2005

Hi,

I've got a problem with developing a DMVPN network using the following setup:

I've got two routers, router1 is a 2821 using IOS 12.4(1a), router2 is a 1760 using IOS 12.3(somewhat).

router1 is our DMVPN hub.

router2 uses pppoe to establish an adsl connection to the internet, the internal lan of router2 is 10.128.0.0/24 and router2's internal ip address is 10.128.0.16.

The DMVPN tunnel uses 10.121.0.0/24 (so router1 is 10.121.0.1, router2 is 10.121.0.2) and users who connect to router1 using easyvpn and the cisco vpn client software get ips assigned starting at 10.121.0.5.

Now all crypto sessions are established, both routers use eigrp to announce their internal networks to each other and the eigrp tables look reasonable filled with routes.

When a easyvpn user connects (to router1) it can ping 10.128.0.16 (internal ip of router2) through the vpn but when I try to ping 10.128.0.17 (a server on router2's internal lan) I've struggled into problems.

According to tcpdump on the pinged server it receives echo requests from 10.121.0.5 and replies to them but these replies never arrive at the easyvpn client.

Which commands will be helpful to debug where the data get's lost? tcpdump for ios would be nice but I don't know of such a command.

Any suggestions regarding debugging this issue or maybe hints to solve this issue (maybe it's a common config error?) would be very nice.

Thank you.

Shawn Lebbon · ‎09-29-2005

Why is your DMVPN tunnel network using the same range of IPs as your Remote-Access users?

As for why your pings are being lost; is the default gateway of the 'server being pinged' 'router2'? If not, and it doesn't have a static route for the DMVPN tunnel/remote access network range, the machine has no idea where to send them!

mwertejuk · ‎09-29-2005

Thanks for your reply Shawn.

The remote-access users use the same ip space because I didn't knew why to use another address space and having a single space for remote networks and users seemed reasonable to me at first, but I'll try to use separate address spaces on a second try soon.

Regarding the default route issue - I'm pretty sure I've set all routes correctly, i.e. on the pinged server I've set the route to the 10.121.0.0/24 network via 10.128.0.16 router2's internal address. I think the server wouldn't reply to pings anyway if it wouldn't know where to send the replies to.

Maybe I should setup a new test environment and try again.

Shawn Lebbon · ‎09-30-2005

I suppose it would work to use the same IP space, but it could get confusing. Especially if scaling the network at all, because what if (in your example) you were to get 5 spokes? Then you'd have DMVPN addresses boing .1 to .4 users on .5 to .n and then more DMVPN addresses on .n+1, etc. By seperating it, it gives you flexability to move things later on, without reconfiguring your network as much. And it's a real pain to bring down DMVPN sites to change their IP addresses, especially considering they're usually remote sites.

Well, if you have a defined route it should forward OK, just be sure that there aren't any ACLs blocking things too. If the machine doesn't have a specific route back to the pinging server, then it will simply forward the response to it's default gateway. Capturing traffic from the pingED machine will show it responding, even if it doesn't know how to get to it. If it's default gateway knows, it should send a redirect to the client machine, and forward the packet on to it's destination. Note that WindowsXP by default now ignores redirects, so you have to turn them back on in the firewall.

If you get it all setup again, let me know if you still end up having trouble. I know as long as the routes are set right on our network, and firewalls/ACLs are set properly, we can ping inside addresses of the DMVPN tunnel.

mwertejuk · ‎10-04-2005

Hey Shawn,

I've got new problems with my new test setup ;)

First of all, it worked, basically, but only for a short time and I don't know why it stopped working.

While it worked I was able to ping from a vpn client to a server on another site and vice versa and every other ping src/dst combination through the vpn worked, but some minutes later it stopped working and I don't know why.

Some details from router1:

r1#show ip eigrp topology

IP-EIGRP Topology Table for AS(65000)/ID(10.121.0.1)

Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,

r - reply Status, s - sia Status

P 10.121.0.0/24, 1 successors, FD is 2816000

via Connected, Tunnel0

P 10.128.0.0/24, 1 successors, FD is 2841600

via 10.121.0.2 (2841600/281600), Tunnel0

--- You can see r1 knows 10.128.0.0/24 is behind 10.121.0.2 (that's r2)

r1#ping 10.128.0.16

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.128.0.16, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 32/34/36 ms

--- 10.128.0.16 is r2's internal address.

r1#ping 10.128.0.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.128.0.1, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

--- 10.128.0.1 is a server on r2 side, have a look at tcpdump on this server

18:11:09.543742 10.121.0.1 > 10.128.0.1: icmp: echo request

18:11:09.544171 10.128.0.1 > 10.121.0.1: icmp: echo reply

18:11:11.550561 10.121.0.1 > 10.128.0.1: icmp: echo request

18:11:11.551028 10.128.0.1 > 10.121.0.1: icmp: echo reply

18:11:13.560307 10.121.0.1 > 10.128.0.1: icmp: echo request

18:11:13.560839 10.128.0.1 > 10.121.0.1: icmp: echo reply

18:11:15.570275 10.121.0.1 > 10.128.0.1: icmp: echo request

18:11:15.571052 10.128.0.1 > 10.121.0.1: icmp: echo reply

18:11:17.570230 10.121.0.1 > 10.128.0.1: icmp: echo request

18:11:17.570698 10.128.0.1 > 10.121.0.1: icmp: echo reply

--- have a look at netstat -rn on this server

Routing tables

Internet:

Destination Gateway Flags Refs Use Netif Expire

default 10.128.0.16 UGS 0 57278 sis0

10.1.1/24 link#1 UC 0 0 sis0

10.2.1/24 link#4 UC 0 0 an0

10.128/24 link#1 UC 0 0 sis0

10.128.0.16 00:04:dc:0c:b5:20 UHLW 1 8467 sis0 1134

127.0.0.1 127.0.0.1 UH 0 72 lo0

Any further debugging ideas?

Shawn Lebbon · ‎10-04-2005

Since you're using eigrp, I'd double check you have the proper commands entered in the tunnel config for it as well.

Double check that you have the following configured on the HUB DMVPN tunnel interface:

no ip next-hop-self eigrp ###

no ip split-horizon eigrp ###

where ### is the eigrp group number.

There are no SPOKE DMVPN tunnel interface eigrp commands necessary.

Otherwise post a sample config and I'll take a look and see if I find anything obvious...

mwertejuk · ‎10-04-2005

Hey Shawn, I found the culprit.

I tried your settings without success and then decided to stop using eigrp which didn't help either.

Then I've checked the configuration again and noticed cisco express forwarding was enabled on router 2.

I haven't yet really understood cef but disabling it makes the setup work fine.

Shawn Lebbon · ‎10-05-2005

Mostly those settings are required to allow the dynamic spoke-to-spoke tunnels to work with eigrp. Obviously you won't see that with only a single hub and spoke. There are other commands for other routing protocols like OSPF.

So CEF was the culprit though huh? I don't think i've disabled it on my routers, and they're working fine...of course it's probably on on all of them too.

I'm not all that well versed in it's operation, but it should make things faster in general. I don't have any idea why it would break it, unless for some reason being configured on only one router caused some sort of failure. But I didn't think CEF talked to other devices...

Anyway glad to hear it's working.