topic Re: FTD VTI tunnels limit in Network Security

FTD VTI tunnels limit

NUSFETLEN — Thu, 18 May 2023 22:00:28 GMT

I use FTD (FPR1010) for VTI tunnels to remote sites. When I have 10 tunnels, they are all stable, however, as soon as I add new tunnel it works but flopping periodically. As more new tunnels are added, they flop more frequently. At the same time, the original 10 tunnels are still stable (only new tunnels are flopping). It looks weird to me. Does the FPR1010 have a limit on number of VTI tunnels? Should I switch to the next version (FPR1012) to have more than 10 tunnels (actually I need around 50 tunnels from the FTD)? With ASA 5506 I could have around 25 VTI tunnels without any issue.

Re: FTD VTI tunnels limit

MHM Cisco World — Thu, 18 May 2023 23:37:35 GMT

the Firepower 1010 support up to 300 mbps vpn throughput.

Re: FTD VTI tunnels limit

Sheraz.Salim — Fri, 19 May 2023 09:08:34 GMT

You can create up to 100 VTIs per source interface, and a maximum of 1000 VTIs.

https://www.cisco.com/c/en/us/td/docs/security/firepower/670/fdm/fptd-fdm-config-guide-670/fptd-fdm-s2svpn.html

you 1010 only support up to 75 vpn tunnel and if you requirement is more then this in that case you might have to consider to upgrade your FTD unit. the way you describe your issue it seems the FTD 1010 can not handel the traffic flow it might doing more cpu processes on the other part (either control plane/mgmt plane) doing the IPS inspection. so yes could be this is the issue. did you check when you had these issues on the vpn what is the processes utlization on the box?

what is the presentation of your Internet line? was it the same internet line you used for the ASA too?

Re: FTD VTI tunnels limit

NUSFETLEN — Fri, 19 May 2023 19:49:49 GMT

I've disabled IPS inspection, but it doesn't help. The 10 VTI tunnels work perfect. As soon as I add a new VTI tunnel, it doesn't work stable and also affect one existing VTI tunnel.

Data Plane and System memories are the same as before while Snort memory has increased from 24% to 38%. The Control Plane CPU has increased from 0.3 to 0.7.

BTW, the FTD version is 7.0.1 while FMC version is 7.1.0.

Re: FTD VTI tunnels limit

NUSFETLEN — Sat, 20 May 2023 16:00:38 GMT

Probably I should open a ticket with Cisco TAC on this.

Re: FTD VTI tunnels limit

Sheraz.Salim — Sat, 20 May 2023 16:54:00 GMT

I am sure the unit FTD 1010 is definitely capable to support more than 10 VTI. it could be some how software bug playing up with this. But if you have cisco TAC support please open case with them.

Your CPU processor look normal in given circusmstance. worth giving tac a shout on this.

Re: FTD VTI tunnels limit

MHM Cisco World — Sat, 20 May 2023 16:57:22 GMT

It can we look in different direction'

It can that the vti effect fpr routing and effect all other vti.

You say that over 10 vti' disable old vti and enable new vti and see if the fpr show same issue or not.

Re: FTD VTI tunnels limit

NUSFETLEN — Mon, 22 May 2023 04:02:40 GMT

10 VTIs work perfect. As soon as I add the 11th VTI, it doesn't work stable (it stays Up for around 20 seconds and drops). Also, one of the old VTIs stars flopping as soon as the 11th VTI is added. As soon as the 11th VTI is deleted, the original 10 VTIs become stable again.

I'll try to add the 11th VTI and remove one of the original VTIs and see if the new 10 VTIs are stable.

Re: FTD VTI tunnels limit

NUSFETLEN — Mon, 22 May 2023 04:43:26 GMT

OK, let me provide some updates. It seems FPR1010 doesn't like more than one VTI with FPR1010 on far end. 9 of 10 VTIs are with ASA 5506 on far end with only one VTI with FPR1010 on far end. I've removed all my 9 VTIs with ASA (while keeping one VTI with FPR1010) and added one VTI with FPR1010 on far end. They don't work. New VTI runs for around 20 seconds and drops while the original VTI doesn't come up (it was working before I added new VTI).

My local FRP1010 runs version 7.0.1 while all far end FPR1010 runs version 7.1.0. I'll try to upgrade the local FPR1010 to the version 7.1.0 and check if it helps.

Re: FTD VTI tunnels limit

Sheraz.Salim — Mon, 22 May 2023 08:44:26 GMT

This is very Interesting. 2xFTD 1010 the VTI does not work. Are they managed FTD through FMC?

Re: FTD VTI tunnels limit

MHM Cisco World — Mon, 22 May 2023 09:23:31 GMT

Can I see how you config the VTI

Re: FTD VTI tunnels limit

NUSFETLEN — Mon, 22 May 2023 17:12:22 GMT

Yes, they are all managed by FMCs, separate FMC for each FTD so far.

Re: FTD VTI tunnels limit

NUSFETLEN — Mon, 22 May 2023 17:18:50 GMT

Interface IP-Address OK? Method Status Protocol
Tunnel1 x.x.x.x YES CONFIG up up
Tunnel3 x.x.x.x YES CONFIG up up
Tunnel10 x.x.x.x YES CONFIG up up
Tunnel16 x.x.x.x YES CONFIG up up
Tunnel22 x.x.x.x YES CONFIG up up
Tunnel29 x.x.x.x YES CONFIG up up
Tunnel30 x.x.x.x YES CONFIG up up
Tunnel32 x.x.x.x YES CONFIG up up
Tunnel34 x.x.x.x YES CONFIG up up
Tunnel35 x.x.x.x YES CONFIG up up
Tunnel38 x.x.x.x YES CONFIG down down
Tunnel41 x.x.x.x YES CONFIG down down

interface Tunnel35
nameif Jefferson
ip address x.x.x.x 255.255.255.252
tunnel source interface outside
tunnel destination y.y.y.y
tunnel mode ipsec ipv4
tunnel protection ipsec profile FMC_IPSEC_PROFILE_3
!
interface Tunnel41
nameif Baker
ip address 169.254.10.154 255.255.255.252
tunnel source interface outside
tunnel mode ipsec ipv4
#The respective IPSEC tunnel is removed
!
interface Tunnel38
nameif Macon_1
ip address 169.254.10.142 255.255.255.252
tunnel source interface outside
tunnel mode ipsec ipv4
#The respective IPSEC tunnel is removed

Re: FTD VTI tunnels limit

NUSFETLEN — Mon, 22 May 2023 19:19:20 GMT

I've upgraded the FTD from version 7.0.1 to 7.1.0, but it doesn't help. The issue is still there.

Re: FTD VTI tunnels limit

MHM Cisco World — Mon, 22 May 2023 19:27:54 GMT

this same major subnet 169.254.10.x 255.255.255.x
check if there is conflict in IP
also this IP is 169.x is used in case of system not get ip from DHCP server, why you use it?

Re: FTD VTI tunnels limit

Sheraz.Salim — Mon, 22 May 2023 20:02:19 GMT

does this VTI flap happens on both FTD units or its only happen to your side of FTD? (if you have access to Both FTD run the below command on both FTD/s as it will give more insight details to us)

even after upgrading the software does not fix the issue in that cause next step is we need to collection some debug from your side of the FTD unit. If i remember the issue is at your side (apologies for that).

you need to SSH into the FTD command line. once you in the command line. To enter Privileged EXEC mode use system support diagnostic -cli command. once in there. follow these command and display the output here. Ideally, we need to understand what causing this issue.

debug crypto condition peer X.X.X.X (Public IP address of the remote FTD) debug crypto ikev2 platform 255 debug crypto ikev2 protocol 255 debug crypto ipsec ! capture VTI type ikv2 interface outside match ip host x.x.x.x host y.y.y.y

The capture VTI as mentioned above you can configure these capture from the FMC that would be more better and easily off-load on to the wireshark.

(Note:I assume you running IKEV2?)

Re: FTD VTI tunnels limit

NUSFETLEN — Sat, 27 May 2023 18:52:50 GMT

I've found a fix of the problem. For each tunnel with FTD at the far end, I've added a unique Local Identity Configuration Key ID. After that all tunnels are Up and stable. It seems the reason was that IP address of each remote FTD's outside interface behind NAT is the same on all remote FTDs, and each FDT uses this IP address as ID by default. This explains why only one tunnel at a time was up for about 20 seconds until another tunnel with the same ID was coming Up.

Re: FTD VTI tunnels limit

MHM Cisco World — Sat, 27 May 2023 19:21:29 GMT

happy ending in end
have a nice day
MHM