Decapsulated inner packet doesn't match... but it looks ok?

[Turkey Twizzler]

Hello,

I have a site-site VPN between a Cisco ASA & a non-cisco device.  The 'sites' are sensors, which only have a couple of IPs so they're on /29 networks.  I needed to add more sites, so copied the original VPN settings using the ASDM.

The sites are in several 192.168.205.xxx /29 (255.255.255.248) ranges, which map to 192.168.105.xx /29 interfaces on the ASA.

However, other than the original site, I'm seeing the following error;

The decapsulated inner packet doesn't match the negotiated policy in the SA.
The packet specifies its destination as 255.255.255.255, its source as 192.168.205.37, and its protocol as udp.
The SA specifies its local proxy as 192.168.105.32/255.255.255.248/ip/0 and its remote_proxy as 192.168.205.32/255.255.255.248/ip/0.

From what I can work out from the error;

I am sending a packet from my laptop at address 192.168.205.37 to somewhere else (broadcast in this case, have also seen other addresses I ping, such as 8.8.8.8). This is correct.

The local address is 192.168.105.32/29 This is correct.

The remote site address is 192.168.205.32/29  This is also correct.

What am I missing?  There are a couple of things I don't understand;

"The SA specifies" - where is the SA config defined on the SA?

"....the local proxy..." maybe I miss-understand what is meant by the proxies?

Thank you for any advice.

I haven't uploaded any configs as they're absolutely huge.  If I need to show anything, let me know the relevant bits...

Thanks!

[Rob Ingram · Accepted solution]

When you setup the VPN you would define a crypto map to define the interesting traffic to encrypt and tunnel over the VPN, this would create an IPSec Securtity Association (SA). The networks (local and remote) defined within the ACL are referred to as the proxy IDs. You will see these SA and proxy IDs in the output of "show crypto ipsec sa".

Only traffic defined in the ACL (which is then referenced in the crypto map configuration) will be encrypted and tunnel. So I would not expect 8.8.8.8 (in your example) to be tunneled. Is traffic between 192.168.105.32/29 and 192.168.205.32/29 actually being sent over the tunnel?

You should make sure you are accurate with the ACL configuration on peer firewalls, this is a common cause for VPN issues.

Another common issue is NAT, usually over a VPN tunnel you should define a conditional NAT (no-NAT) rule is configuration, to ensure that traffic is not natted.

If you are still having issues, please upload the crypto, nat, access-list configuration and the output of "show nat".

HTH

[Turkey Twizzler]

Thank you both for your assistance, I now know where to start looking and am confident you've got me on the right track. 

The crypto ACL list only contains the 192.168.x.x addresses, so you're right, 8.8.8.8 isn't something it would know about.  You've also made me realise that I haven't seen the decapsulation error between devices in the 192.168.105 & 205 ranges, so I must almost have it working!  Devices still can't communicate between those ranges (even though the log says it's built the connection and not denied or nat error), but I'm close! Maybe I have it natting one way but not the other. A couple more days of staring at the manual & trying it out & I think I'll have it working as intended.  It's very slowly starting to make sense.

Thanks again!

[Rahul Govindan]

To add to what @Rob Ingram mentioned, this is almost certainly an issue with the remote end setting. Sometimes, third party devices do route based VPN instead of policy based. This means that they negotiate a local and remote proxy as 0.0.0.0. If this is the case and the ASA is the one initiating the tunnel, the ASA would propose the right proxies, but the remote end still installs 0.0.0.0 in their SA database. ASA obviously creates the SA's with the right proxies. The ASA also has a security check (which many 3rd party devices do not), to check to see if the inner traffic (or actual data through tunnel) matches what we negotiated for in the first place. That is why you are seeing this error message. See if there is a setting to do policy based VPN on the 3rd party VPN device. 

[eduardo.silva@fr]

Thanks for the great explanation.
It helped me investigate the same error, however I still can't find my problem.
I have a local network already working on the inside of my ASA5505, which is 10.70.10/24. I'm trying to add another local network. I've added it to the tunnel group using the CISCO ASDM app and it has automatically updated the Access-list to include both local networks which looks good:

access-list outside_cryptomap extended permit ip object-group DM_INLINE_NETWORK_11 object Remote_network

object-group network DM_INLINE_NETWORK_11
 network-object 10.70.10.0 255.255.255.0
 network-object object NewNetwork

object network NewNetwork
 subnet 10.90.0.0 255.255.0.0

object network Remote_network
 subnet 10.0.0.0 255.255.0.0

However I still get this error:

IPSEC: Received an ESP packet (SPI= 0xFA375B94, sequence number= 0x6CEEE) from xx.xx.xx.xx (user= xx.xx.xx.xx) to xx.xx.xx.xx.  The decapsulated inner packet doesn't match the negotiated policy in the SA.  The packet specifies its destination as 10.90.90.90, its source as 10.0.10.210, and its protocol as icmp.  The SA specifies its local proxy as 10.70.10.0/255.255.255.0/ip/0 and its remote_proxy as 10.0.0.0/255.255.0.0/ip/0.

It doesn't look like the SA has taken into consideration the change on the access-list

ASA01# show ipsec sa
interface: outside
    Crypto map tag: outside_map, seq num: 3, local addr: xx.xx.xx.xx

      access-list outside_cryptomap extended permit ip 10.70.10.0 255.255.255.0 10.0.0.0 255.255.0.0
      local ident (addr/mask/prot/port): (10.70.10.0/255.255.255.0/0/0)
      remote ident (addr/mask/prot/port): (10.0.0.0/255.255.0.0/0/0)
      current_peer: xx.xx.xx.xx

Are the local ident negotiated at the tunnel handshake and maybe I need to re-stablish to consider the ACL change?
Appreciate any help.
Thanks

[Jon Marshall]

To answer your specific question the SA in terms of IPs is defined by the access list you use in the IPSEC settings on your firewall. 

Local proxy is the local subnet from the firewall's perspective and remote proxy is the subnet at the other end of the tunnel again all from the perspective of the firewall you are on. 

Jon