topic Re: Losing EIGRP adjacencies w/ DMVPN in Network Security

Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Fri, 21 Feb 2020 07:49:42 GMT

I am trying a new code, 12.3.11T2 on our routers and every night when traffic is low, the spoke router with the new code on it would lose it EIGRP adjacencies. All the other spoke that are running 12.3.6a run fine. And the only way to get the neighbors back is to reload the router. Even reconfiguring the EIGRP process does not bring it back. Any insight would be appreciated.

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Tue, 28 Dec 2004 03:44:36 GMT

I have had this exact same problem. I am running a single DMVPN with two 2691 routers running 12.3(12). I opened a case with TAC because the spoke was loosing its routes after anywhere from 8 to 15 hours and a reload would only fix the problem. It turned out the input queue on the tunnel interface was incrementing and once it reached the max would drop all other traffic including eigrp routes. Here is the interface stats.

Tunnel0 is up, line protocol is up

Hardware is Tunnel

Internet address is 10.255.255.2/24

MTU 1514 bytes, BW 1000 Kbit, DLY 10000 usec,

reliability 255/255, txload 6/255, rxload 6/255

Encapsulation TUNNEL, loopback not set

Keepalive not set

Tunnel source 193.28.89.9 (FastEthernet0/0), destination UNKNOWN

Tunnel protocol/transport multi-GRE/IP, key 0x186A0, sequencing disabled

Checksumming of packets disabled, fast tunneling enabled

Tunnel protection via IPSec (profile "SDM_Profile1")

Last input 00:00:00, output 00:00:02, output hang never

Last clearing of "show interface" counters never

Input queue: 186/1024/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/0 (size/max)

5 minute input rate 27000 bits/sec, 20 packets/sec

5 minute output rate 27000 bits/sec, 18 packets/sec

7122069 packets input, 1729019848 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

6496026 packets output, 1314657726 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 output buffer failures, 0 output buffers swapped out

KLDN2691#

TAC suggested I change the queue limit to 1024 with the command "hold-queue 1024 in" under the Tunnel interface config. This will cause the interface to take a much longer period based on traffic to max out. I am working on grabbing an engineering release from TAC now. Hope that helps.

-Jake Gibb

jgibb@krollworldwide.com

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Tue, 28 Dec 2004 17:52:59 GMT

Thanks for the info. I'll give that a shot. Please let me know when you get that release.

Quan Nguyen

qnguyen@cci.edu

Re: Losing EIGRP adjacencies w/ DMVPN

damarcus.richards — Thu, 30 Dec 2004 17:52:38 GMT

I ran into the same problem on this code as well. I downgraded to 12.3.8T5 on the 831. Did you try bouncing the Tunnel interface on the hub side? Is the Hub having any CPU EIGRP issues? They 2 may not be related, but I was able to stablize EIGRP w/ DMVPN. Also, one thing to look at is to see if the router running the 12.3.11T2 code 'eigrp topology is showing'. If it has another interface as a better route than the Tunnel interface, that can cause the routing table to send eigrp updates out of the other interface for the Tunnel subnet & cause the RTO to be quite high.

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Thu, 30 Dec 2004 19:32:03 GMT

I was using 12.3.8T5 before and because I am also using NAT route maps with multiple links and CBAC I started seeing stack traces which lead to a known bug. The engineer suggested that I get a non-public release 12.3(11.6)T which has been stable on the 1712 so far. For the 2600's I am still awaiting the final image from engineering.

-Jake Gibb

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Fri, 07 Jan 2005 18:59:06 GMT

Which code are you running on your routers now? I'm still having problems with 12.3.12. Tunnel input queue still increments. NAT not working correctly. It's not expiring old translations and I have to manually clear the translation tables. Thanks

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Fri, 07 Jan 2005 18:59:09 GMT

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Fri, 07 Jan 2005 19:45:38 GMT

I am still running 12.3.12 but just yesterday I was given access to an interrim release c2691-advsecurityk9-mz.123-12.10. I have uploaded it to my 2691 and will reload with the new image later today. I hope this will solve the issue with the 2691 with the encryption module. What has me confused is my head end 2691 WITHOUT the encryption module does not have this problem. I have mentioned this to Cisco but have not received a response. Do you have an encryption card on your router??

-Jake

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Fri, 07 Jan 2005 21:54:59 GMT

I have a 7204VXR at the head running 12.3.6a. That works fine. One of my spokes is one of the new 2821. Code is very limited on these, only offering 12.3.8T and 12.3.11T versions. They both have VPN accelerator cards. The hub is running fine and my other spokes running older 12.3.6a code run fine also. It's just the 2821 with new code not working right.

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Sun, 09 Jan 2005 15:16:03 GMT

I just loaded 12.3(6c) on the 2691 router and will be watching it to see if it has the same issues as 12.3(12). So far the input queue is 0 and it appears to be fine. The image I got from Cisco crashes and I'm waiting for their response.

-Jake

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Mon, 10 Jan 2005 18:35:19 GMT

Did TAC give you a bug ID for this? Trying to work with TAC on this end. What's terrible is the 2800s only have 12.3.8T or 12.3.11T. They both have the same problem.

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Mon, 10 Jan 2005 20:33:43 GMT

Yep.CSCsa43492.I am having the image posted again because I think the MD5 sum did not match and that is what caused the router to crash when trying to load it.

-Jake

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Wed, 12 Jan 2005 18:47:16 GMT

PLEASE READ - Cry engine problem.

I've tested what I think may be a fix and help isolate the problem. I noticed that on the 2691 routers with the cry engine accelerator card the input queue problem was noticed whereas the routers with the cry engine card it was not. I disabled the cry engine with the command no cry engine accel and the input queue stopped incrementing!! This has been confirmed on two routers (1712 and 2691) both with accelerator cards installed. Can someone please try this using the 12.11(12) IOS revision and report back to this conversation?

-Jake

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Wed, 12 Jan 2005 22:14:43 GMT

TAC posted up 12.3.12(10) for me but, it worked like crap. All my routers have VPN AIM mods in them. You can also disable CEF and the queue will stop incrementing. I tried it last night and it hasn't incremented today. TAC says the 12.3.13 code due out in March is suppose to fix these bugs.

Re: Losing EIGRP adjacencies w/ DMVPN

gfullage — Tue, 18 Jan 2005 00:10:02 GMT

Sorry to come in on this a little late, I'll try and help out as much as I can.

First off, CSCsa43492 was a definate major bug with very high priority, and should be resolved in 12.3(12.10) mainline code. This is an interim release that will have to be specially posted for you. It will then be integrated into the 12.3(13) release which according to the current schedule, is due out around mid-Feb.

If noc@krollworldwide.com upgrades to the interim version on his/her routers then the problem should go away, and you should be able to enable your encryption card. We don't want you to run without the card since this will put extra load on your CPU. You mention that it crashed when you laoded it but you think it wasn't downloaded correctly, can you confirm that for us please.

If qnguyen8 only has 28xx's, then we'll need to get that bug integated into the T train, which currently it doesn't seem to be. The bug for this same issue on the T train is CSCsa45724, so let me investigate with the developers what's going on and I'll get back to you here. Can you also expand further on "it worked like crap", what exactly was crap?

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Tue, 18 Jan 2005 01:17:44 GMT

I actually got all this stabalized with TAC. The 12.3.(12.10) code was posted for me twice. The first one had a bad checksum. The second one had had some "bad magic number" boot up errors and while it was up, had "CHUNKBADMAGIC" errors, TACACs also didn't work.

The workaround was to disable cef on the tunnel interface. This worked but, it would be nice to have cef up.

Eventually went with 12.3.11T2 and changed the IP MTU on the tunnel interface to 1400. The input queue wedging stopped with this code. We also used 12.3.8T6 which worked but if you are running ip inspect or ip ips, you will run into CSCef58083.

So in short, upgrade to 12.3.11T2 and change ip mtu on tunnel interface to 1400. It has been stable without input queue increasing for the last 72 hours.

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Tue, 18 Jan 2005 14:24:42 GMT

Thank you everyone for posting on this topic. I have been running 12.3(12.10) on the 2691 and it does appear to be fine. The only downside is the tunnel interface has to be shut then no shut to bring it up upon a router reload. The engineer I am working with is trying to find the fix for this problem now. I'm tempted to back down to 12.3.11T2 until Cisco can resolve these issues. Can anyone tell me if there are running the advipservices version or advsecurity and what the difference may be? Also I have a Cisco 1712 experiencing the same problem with input queue wedge but I am using CBAC and NAT route maps on the router so I do come across the bug mentioned in previous messages. Is there going to be a fix for this as well? Thanks again.

-Jake Gibb (noc@krollworldwide.com)

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Tue, 18 Jan 2005 14:56:08 GMT

I did a compare of the two images 12.3.11T2 advsecurity and advipservices. It appears they both support standard VPN, QOS, IDS, etc. but ipservices includes support for BGP, VoIP, and MPLS functions. Hope that helps anyone that may have been curious as I was. Thanks.

-Jake Gibb

Re: Losing EIGRP adjacencies w/ DMVPN

noc — Tue, 18 Jan 2005 17:43:44 GMT

I have just reloaded my 2691 with 12.3.11T2 and disabled IP CEF. The input queue wedge problem did stop on the tunnel interface. How much of an impact does this pose on the router and passing traffic? Is it substantial?

Re: Losing EIGRP adjacencies w/ DMVPN

qnguyen8 — Tue, 18 Jan 2005 18:01:43 GMT

I didn't see much of a performance hit but, you can enable cef. You just need to take your IP MTU size down to 1400 on both hub and spoke tunnel interfaces and that should stop the wedging problem.