DMVPN NHRP error indication logs

[the-lebowski]

I am seeing a ton of these entries one of my spoke routers.. the hub router IP is 10.1.2.1 and all the dst IP's are not in use so I can't figure out why the spoke router is sending packets to 10.1.2.3, 10.1.2.44, 10.1.2.47 and so-o and those IP's are not in use in my environment. 

Its dropping packets across the tunnel constantly and I can't figure out why. Does anyone have an idea of what is happening? 

Oct 29 20:05:13.544 IST: %NHRP-3-PAKERROR: Received Error Indication from 10.1.2.1, code: protocol generic error(7), (trigger src: 10.1.2.192 (nbma:28.28.45.45) dst: 10.1.2.3), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 58 F3 2C 00 34

Oct 29 20:05:18.572 IST: %NHRP-3-PAKERROR: Received Error Indication from 10.1.2.1, code: protocol generic error(7), (trigger src: 10.1.2.192 (nbma:28.28.45.45) dst: 10.1.2.44), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 58 F2 F2 00 34

Oct 29 20:05:23.777 IST: %NHRP-3-PAKERROR: Received Error Indication from 10.1.2.1, code: protocol generic error(7), (trigger src: 10.1.2.192 (nbma:28.28.45.45) dst: 10.1.2.47), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 58 F2 ED 00 34

Oct 29 20:05:30.181 IST: %NHRP-3-PAKERROR: Received Error Indication from 10.1.2.1, code: protocol generic error(7), (trigger src: 10.1.2.192 (nbma:28.28.45.45) dst: 10.1.2.46), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 58 F2 EC 00 34

Oct 29 20:05:36.638 IST: %NHRP-3-PAKERROR: Received Error Indication from 10.1.2.1, code: protocol generic error(7), (trigger src: 10.1.2.192 (nbma:28.28.45.45) dst: 10.1.2.7), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 58 F3 26 00 34

Oct 29 20:05:42.842 IST: %NHRP-3-PAKERROR: Received Error Indication from 10.1.2.1, code: protocol generic error(7), (trigger src: 10.1.2.192 (nbma: 28.28.45.45) dst: 10.1.2.34),

This is the tunnel config on the spoke itself, granted I have another tunnel (different ISP) on this spoke going to another hub and I do not see these error messages on that tunnel.

[David Tomkinson]

Hello,

I am experiencing something very similar, did you find an explanation for this?

Thanks

Dave

[the-lebowski]

Yeah, I had to add this command on the hub:

ip nhrp server-only

Once i did that those error messages went away on the spoke itself.    

[David Tomkinson]

Thanks for the swift response!

I will try this out

Dave

[Frank DeNofa]

All,

Here's a quick blurb from a document I've been working on which helps to explain this specific error message:

An error code of 7 will be returned by the NHS when an error occurs when processing the packet which is not associated with any of the other NHRP error codes. According to RFC2332, triggers for the error code include invalid version numbers, invalid protocol types, and failed checksums. This error is commonly seen if the NHS receives a Resolution Request for an IP address which it does not have an entry for in its NHRP cache. For example, if a DMVPN spoke tries to send traffic to a spoke IP address which is not registered with the hub, the hub will return an NHRP Error Indication with the Protocol Generic Error specified.

To troubleshoot this condition, you should collect the following on both the hub and spoke routers:

show ip nhrp

debug nhrp

debug nhrp packet

Collecting the debugs will show you the exact NHRP packets which are being sent and received, which may give you an indicate as to what is prompted these errors to be return. Keep in mind that the debugs can be quite chatty and significant NHRP traffic may cause the debugs to impact the router performance.

HTH,

Frank

[Justin Reeve]

This happened to me when I was trying to ping an address that didn't exist, so NHRP couldn't resolve it:

*Jul 1 16:13:50.067: %NHRP-3-PAKERROR: Received Error Indication from 155.1.0.5, code: protocol generic error(7), (trigger src: 155.1.0.2 (nbma: 169.254.100.2) dst: 155.1.0.8), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 58 D5 FC 00 34 

155.1.0.8 - doesn't exist, it was a typo.

Cheers

J

[Fayssal KASSA]

Hi all,

Somes IOS (n° 12/15, or types IPServices, AdvEnt, etc) implements differently the DMVPN 

Somes IOS needs the cmd: ip nhrp network-id X, others, needs tunnel key Y, others need the both of them.

Without theses cmd(s), Spokes can't registrate in the HUB, so the HUB can't frwd the response when onr spoke needs resolution, because of his empty cache (sh ip nhrp), so he generate the msg error: 

 NHRP: Encapsulation failed for destination @private@ out Tunnel0

(M) error code: protocol generic error(7)

[Frank DeNofa]

To elaborate a bit further on this, regardless of your version of IOS code "ip nhrp network-id" is required in order to enable NHRP on your tunnel interface. The tunnel key configuration is separate from NHRP, but allows a router to properly processed GRE packets on the correct mGRE interface. From this guide:

"The tunnel source, tunnel destination, and tunnel key (triplet) must be unique for all tunnel interfaces on a device. For a multipoint GRE interfaces where tunnel destination is not configured, the pair (tunnel source and tunnel key) must be unique. Incoming GRE packets are also matched to point-to-point GRE tunnels first; if there is not a match, they are matched to mGRE tunnels."

[Fayssal KASSA]

Hi Frank,

Yes, i'm agree with this. But for my labs, i used two versions of IOS

DMVPN cloud composed of  five routers,

each router have only one tunnel interface, so "tunnel key" is not mandatory.

v12.2, just using ip nhrp network-id ==> DMVPN OK

v15.2, just using ip nhrp network-id ==> DMVPN NOK

v15.2, just using ip nhrp network-id + tunnel key ==> DMVPN OK

Regards,

Fay ONIS

[Frank DeNofa]

Fay,

I'm surprised that your second situation would not work if this was the only tunnel configured on each of the routers. The purpose of the tunnel key is so that the routers can properly classify the GRE packets on the correct tunnel; if there is only one tunnel, then by that logic it should not be possible to improperly classify a packet. That said, much has changed in IOS code for every feature which we implement, so it's possible that this could have something to do with the change in behavior. As a best practice, it is a good idea to use tunnel keys on all mGRE tunnels.

Frank

[Fayssal KASSA]

Hello Frank,

Many thanks for your response,

I'll use the "tunnel key" whatever the number of mGRE tunnels on the router.

Have a nice weekend,

Fay ONIS

[casanavep]

I get the same error with both of those items on all tunnels, hubs and spokes.  This is with ISR4K spokes and ASR 1000 series dual-hubs running 15.5 (3.15 XE).   Also seeing an old message from Cisco Bugs of the past creeping back:

%NHRP-3-PAKREPLY: Receive Resolution Reply packet with error - insufficient resources(5)

ISR4Ks have great performance, but you get to fight lots of code bugs and TAC engineers who don't yet understand them.  Of course, my configs follow the Cisco "Validated" design guides exactly, to include IOS versions (but the constantly change IOS versions in their guide, good luck keeping up).  The joy of new toys :)

[Valdir Malaquias]

I found this bug for the same problem.

Cisco Bug: CSCuo25655 - %NHRP-3-PAKERROR on spoke-to-spoke tunnel in DMVPN Phase 3 deployment

Last Modified

May 05, 2016

Product

Cisco IOS

Known Affected Releases

15.2(4)M6

Description (partial)

Symptom:
On a DMVPN Phase 3 deployment where hubs are load-balanced and consequently hubs don't hold nhrp entries for all spokes, the following error is encountered on spoke to spoke tunnels:
NHRP-3-PAKERROR Received Error Indication from 10.0.0.1, code: protocol generic error(7), (trigger src: 10.0.0.2 (nbma: 20.0.0.2) dst: 10.0.0.3), offset: 0, data: <hex data>

Here 10.0.0.1 is the complaining hub (it is complaining because it does not have the NHRP entry for the registered spoke 10.0.0.3) and the spoke that is making the NHRP Resolution request is 10.0.0.2 with NBMA address 20.0.0.2. This can happen if the spoke 10.0.0.3 is registered to another hub and hence the complaining hub has no awareness of the NHRP entry of the spoke.

Conditions:
a. ISR G2 routers running 15.2(4)M release 
B. DMVPN Phase 3 deployment where hubs are load-balanced and consequently hubs don't hold nhrp entries for all spokes
https://quickview.cloudapps.cisco.com/quickview/bug/CSCuo25655

I hope it help everyone.

[asdasdmvdfd2030]

Do not use the interface name for the tunnel source. You must use the client IP address for the tunnel source. In this case the problem will be fixed and you will no longer receive an error !

[olly-ray]

make sure the ospf network type is set to point-to multipoint on the spoke on the tunnel interface. I hope this helps