Disjointed IPSEC FLOW traffic - Cisco Community

2126

Views

10

Helpful

12

Replies

I have an issue with a Site-to-Site VPN where the traffic isn't flowing over the tunnel correctly. I cannot find anywhere online where someone even remotely has the same symptom as I am seeing.

This is from the command show crypto session remote pu.bl.ic.ip2 detail:

Crypto session current status

Code: C - IKE Configuration mode, D - Dead Peer Detection K - Keepalives, N - NAT-traversal, T - cTCP encapsulation X - IKE Extended Authentication, F - IKE Fragmentation R - IKE Auto Reconnect, U - IKE Dynamic Route Update S - SIP VPN

Interface: Tunnel15

Uptime: 00:32:10

Session status: UP-ACTIVE

Peer: pu.bl.ic.ip2 port 4500 fvrf: (none) ivrf: (none)

Phase1_id: pu.bl.ic.ip2

Desc: (none)

Session ID: 0

IKEv1 SA: local pu.bl.ic.ip1/4500 remote pu.bl.ic.ip2/4500 Active

Capabilities:N connid:1615 lifetime:23:27:48

IPSEC FLOW: permit ip host xxx.xxx.xxx.1 10.11.12.13/255.255.255.0

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 2 drop 0 life (KB/Sec) 4175560/2045

Outbound: #pkts enc'ed 0 drop 0 life (KB/Sec) 4175561/2045

IPSEC FLOW: permit ip host xxx.xxx.xxx.2 10.11.12.13/255.255.255.0

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 2 drop 0 life (KB/Sec) 4196348/1985

Outbound: #pkts enc'ed 0 drop 0 life (KB/Sec) 4196349/1985

IPSEC FLOW: permit ip host xxx.xxx.xxx.3 10.11.12.13/255.255.255.0

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 3537 drop 0 life (KB/Sec) 4222116/1669

Outbound: #pkts enc'ed 3474 drop 0 life (KB/Sec) 4221503/1669

IPSEC FLOW: permit ip host xxx.xxx.xxx.4 10.11.12.13/255.255.255.0

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 0 drop 0 life (KB/Sec) 4323580/1669

Outbound: #pkts enc'ed 0 drop 0 life (KB/Sec) 4323580/1669

IPSEC FLOW: permit ip 0.0.0.0/0.0.0.0 0.0.0.0/0.0.0.0

Active SAs: 2, origin: crypto map

Inbound: #pkts dec'ed 0 drop 0 life (KB/Sec) 4297362/1726

Outbound: #pkts enc'ed 8 drop 0 life (KB/Sec) 4297361/1726

Notice how the traffic for xxx.xxx.xxx.3 is flowing correctly(in my opnion) through the same IPSEC FLOW, but the other traffic is flowing in a disjointed manner where it is coming in over the correct IPSEC FLOW but it is going outbound over the default route(or whatever it is called) IPSEC flow.

I only see it doing this when something is wrong. I can't find anyone that mentions this situation online(that's long for google). Anyone of you all ever see this issue and know the cause? I have actually seen a case where it was working in this disjointed manner, but very rarely.

12 Replies 12

You've got multiple SAs with what appears to be different source IP addresses going to the same destination, although the netmask seems incorrect?

And you've also got what appears to be a tunnel interface (0.0.0.0 0.0.0.0).

It would be better if we saw your configuration, please provide your configuration for review.

Do you have NAT configured, if so have you excluded traffic over the VPN from being translated?

I am using VTIs. I have the following route that should sent the traffic through the tunnel:

ip route 10.11.12.13 255.255.255.0 Tunnel15

The only NAT involved is a 1-to-1 of another set of IPs that go over the same tunnel:

ip nat inside source static xxx.xxx.xxx.xx2 xxx.xxx.xxx.xx3

Tunnel interface 15 has ip nat outside, the LAN interface on the router has ip nat inside.

I also have an in access-list applied to int tunnel 15, but that should affect this since the issue is the return traffic. I also removed the access-list, bounced the tunnel and it made no difference.

@mcunetworking have you got a VTI or Multi SA (tunnel interface with an ACL defining interesting traffic)? You shouldn't have multiple SAs if you were using a VTI. Or do you have a legacy crypto map configuration left over that should be removed?

Provide the configuration, even it is it only the pertinent configuration for this tunnel would help speed things up.

Masking table definition:

1.1.1.1 = my public IP

1.1.1.2 = my public IP's gateway

2.2.2.2 = remote peer Public IP

10.0.0.1 = some internal IP 1

10.0.0.2 = some internal IP 2

10.0.0.3 = some internal IP 3

10.0.0.4 = some internal IP 4

172.16.0.0/24 = Remote internal subnet

Config:

no crypto isakmp default policy
crypto isakmp policy 15
encr aes 256
hash sha512
authentication pre-share
group 14
!

!

crypto ipsec transform-set ProblemTunnel esp-aes 256 esp-sha512-hmac
mode tunnel
!

!

crypto ipsec profile ProblemTunnel
set transform-set ProblemTunnel
set pfs group14
!

!

interface Tunnel15
description ProblemTunnel
ip address 10.0.15.1 255.255.255.0
ip access-group ProblemTunnel in
ip nat outside
ip virtual-reassembly in
tunnel source 1.1.1.1
tunnel mode ipsec ipv4
tunnel destination 2.2.2.2
tunnel protection ipsec profile ProblemTunnel

interface GigabitEthernet0/1
description LAN
ip address 10.0.0.5 255.255.255.0
ip nat inside
ip virtual-reassembly in
duplex auto
speed auto
!

!

ip nat inside source static 10.0.0.3 10.0.0.4

ip route 0.0.0.0 0.0.0.0 1.1.1.2

ip route 172.16.0.0 255.255.255.0 Tunnel15

!

!

ip access-list extended ProblemTunnel
permit tcp 172.16.0.0 0.0.0.255 host 10.0.0.1 eq 1024
permit tcp 172.16.0.00 0.0.0.255 host 10.0.0.3 eq 1024
permit tcp 172.16.0.0 0.0.0.255 host 10.0.0.4 eq 1024
permit tcp 172.16.0.0 0.0.0.255 host 10.0.0.2 eq 443
deny ip any any

Let me know if I am missing something in the configuration.

I will say, and this could be the problem, the PEER side has configured 4 separate tunnels(they are not using a Cisco ISR), my side essentially has one. It has pretty much always worked, just occasionally gone down. Could this be a situation where the traffic that works, works because it was the first traffic to go over the newly created tunnel(s) and/or newly re-negotiated SAs and the rest of the traffic is what breaks? Sort of like a first come first serve? The traffic to the 10.0.0.1 server is the primary traffic that goes over this tunnel, the 10.0.0..2-4 traffic is for testing. I am currently researching the best way to split my side up into 4 tunnel too, although that might be improbable since they have the same peers.

Edit: I just tested and there are situations where I can bounce the tunnel and the traffic for 10.0.0.1 doesn't even work, even if it is the first traffic that tries to traverse the tunnel. Took me bouncing the tunnel 7 times to correct it. The traffic for 10.0.0.2 still doesn't traverse correctly though.

Reading this line in a cisco "IPsec Virtual Tunnel Interfaces" document makes me think the different SA's is what is causing the issue:

"Static VTIs (SVTIs) support only a single IPsec SA that is attached to the VTI interface." Link here: Link.

I am able to create a separate VTI with the same peer IP address, but now I have to figure out how to route the same destination IP address over 2 different VTIs depending on the source IP. I might be able to do a plain jane policy route with a route-map but I am still testing.

@mcunetworking yes, like I said you shouldn't have multiple SAs if you are using a VTI. You should have 0.0.0.0/0.0.0.0 as the proxy ID. It's the routing that determines what traffic is sent over the tunnel.

It sounds like your peer has configured a policy based VPN and you've got a Route Based VPN, determine what configuration your peer has and mirror their configuration or get them to mirror yours with a VTI.

You don't need to create another VTI to the same destination, you create multiple static routes via the tunnel interface.

The problem is that they are using a GUI that isn't Cisco, and they just add another config like it is a completely separate tunnel.

For example, they say "Oh now we need to connect to 10.0.0.5, just create another tunnel". So their tunnels are destination based.

The other end has 3 or 4 IP addresses that contact me, lets say:

172.16.0.1

172.16.0.2

172.16.0.3

172.16.0.4

They are trying to contact the following on my side(as a recap), and it could be any of the above connecting to any of the below:

10.0.0.1

10.0.0.2

10.0.0.3

10.0.0.4

So your comment "create multiple static routes via the tunnel interface", are you saying instead of having one routed subnet:

ip route 172.16.0.0 255.255.255.0 Tunnel15

I should instead have this:

ip route 172.16.0.1 255.255.255.0 Tunnel15
ip route 172.16.0.2 255.255.255.0 Tunnel15
ip route 172.16.0.3 255.255.255.0 Tunnel15
ip route 172.16.0.4 255.255.255.0 Tunnel15

I tried creating PBRs, and used one ACL to say anything coming from 10.0.0.1 go to Tunnel15 and everything coming from 10.0.0.2 go to Tunnel16 in order to try and match their config, but it didn't work. After the fact I was thinking is can't really since I don't know which tunnel they cam to me on. So I kind of see what you mean about one VTI, but I don't quite get how to get get the traffic to merge over one SA.

I also came across Cisco's "Configure Multi-SA Virtual Tunnel Interface on IOS-XE router" webpage. Which makes it look like I would need to upgrade to IOS 16, change a couple of things and my issue would be fixed. I am currently on 15.0(1r)M16.

@mcunetworking if the peer does not want to change to a route based VPN, then change your configuration to policy based VPN using a crypto map. In the ACL to define the interesting traffic just define your local ip address(es) as the source, with the destination as their IP addresses. Essentially you need to mirror each others configuration in regards to the local/remote networks that define the interesting traffic.

I came across more Cisco documentation that says VTI is not compatible with 3rd parties. I am trying to revert back to a crypto map setup for them.

I was finally able to get the crypto map setup, which I now remember why I went away from them. I will just have to upgrade the IOS in order to get Multiple-SA VTIs.

https://www.cisco.com/c/en/us/support/docs/security-vpn/ipsec-negotiation-ike-protocols/214728-configure-multi-sa-virtual-tunnel-interf.html

according to this doc.

please config the tunnel with ipv4 policy and config reverse-route under IPSec profile.

Learn, share, save

Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.

New here? Get started with these tips. How to use Community New member guide

Log in to Community