Help Setting or Troubleshooting the Default Gateway on FTD

[Lucas Phelps]

I've got 20+ Cisco 5506s deployed with the FirePower Threat Defense (FTD) 6.2.2.1 code.   They are all managed by a single FMC server.  When I go into Devices > Device Management, several show up as green/online, but I'm not able to ping them from my FMC.

On a few of my remote FTD boxes, they don't seem to have a default gateway defined in the config and I don't know how to set one on them.  I'm not able to ping a few of them from my FMC, so I'm not sure why they are reporting as online.  

Any thoughts on how to fix the default gateway/routing issue on these boxes?  I can access these boxes fine if I try to SSH from a PC on their local subnet, so that's why I think its got to be a GW issue.

[Francesco Molino]

Hi

What IP are you trying to ping? Management or inside?

The green shows up because FMC uses management interface to access FTDs.

To allow icmp on other interfaces except management you'll need to configure a platform policy settings under Devices menu.

What gateway are you talking about? For management or other interfaces?

The management interface can be modified tough cli by using the command configure network ipv4 manual

For other interfaces you'll need to add static routes or dynamic routing configuration of you have any on your lan side.

[Lucas Phelps]

I'm trying to ping from my HQ to the management interface of my FTDs. Most work, but I've got a few that don't.  I can only ping the management interface of these few devices from the router at that office (on the local management subnet).  Attempting to ping and SSH to management IP, and it fails from my HQ or anywhere other than the local subnet.

Does the 'configure network ipv4 manual' configure the management gateway or the gateway for the other interfaces?

[Francesco Molino]

This is for management not other interfaces.
Also after doing what Mohammed said, you can verify that all parameters are configured correctly on your management interface by issuing the command:
cat /ngfw/etc/sysconfig/network-devices/ifcfg-management0

[Mohammed al Baqari]

Access the unit using CLI from a local PC and get the following:

- *show network*
- from expert mode change to sudo su then get *netstat -rn*
- from expert mode, get *cat
/ngfw/etc/sysconfig/network-devices/ifcfg-internal-route*
- from expert mode, get *ls -la /ngfw/var/common*

[Lucas Phelps]

I issued the 'configure network ipv4 manual 1.1.4.2 255.255.255.0 1.1.4.1' command but it didn't seem to help.  Still no IPv4 Default Gateway listed on the show network.

Sure looks like the default gateway isn't there.  The correct gateway for my management interface should be 1.1.4.1 which is the local router at my branch office.

> show network
===============[ System Information ]===============
Hostname : 04-FirePower
Domains : contoso.com
DNS Servers : 172.16.1.1
10.16.1.1
Management port : 8305

======================[ br1 ]=======================
State : Enabled
Channels : Management & Events
Mode : Non-Autonegotiation
MDI/MDIX : Auto/MDIX
MTU : 1500
----------------------[ IPv4 ]----------------------
Configuration : Manual
Address : 1.1.4.2
Netmask : 255.255.255.0
Broadcast : 1.1.4.255
----------------------[ IPv6 ]----------------------
Configuration : Disabled

===============[ Proxy Information ]================
State : Disabled
Authentication : Disabled

#########################################################

root@04-FirePower:/home/admin# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 169.254.1.1 0.0.0.0 UG 0 0 0 tap_nlp
1.1.4.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
127.0.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br0
127.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 tap0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 tun1
169.254.1.0 0.0.0.0 255.255.255.248 U 0 0 0 tap_nlp
#
#
#
cat /ngfw/etc/sysconfig/network-devices/ifcfg-int
# automatically generated on Fri Jul 7 17:18:55 UTC 2017

INTERNAL_ROUTE_ENABLED=1
INTERNAL_GATEWAY_DEVICE=tap_nlp
INTERNAL_GATEWAY=169.254.1.1
INTERNAL_GATEWAY_V6=fd00:0:0:1::1/64
#
#
root@04-FirePower:/home/admin# ls -la /ngfw/var/common
total 48856
drwxrwxr-x 2 root detection 4096 Oct 19 08:25 .
drwxr-xr-x 13 root root 4096 Nov 23 03:33 ..
-rw-r--r-- 1 root root 50016691 Oct 19 08:25 results-10-19-2017--71701.tar.gz

[Lucas Phelps]

Here is the output from a properly WORKING FTD:

Show Network

===============[ System Information ]===============
Hostname : 07-FirePower
Domains : contoso.com
DNS Servers : 172.16.1.1
10.16.1.1
Management port : 8305
IPv4 Default route
Gateway : 1.1.7.1

======================[ br1 ]=======================
State : Enabled
Channels : Management & Events
Mode : Non-Autonegotiation
MDI/MDIX : Auto/MDIX
MTU : 1500
MAC Address : 00:62:EC:93:F7:83
----------------------[ IPv4 ]----------------------
Configuration : Manual
Address : 1.1.7.2
Netmask : 255.255.255.0
Broadcast : 1.1.7.255
----------------------[ IPv6 ]----------------------
Configuration : Disabled

===============[ Proxy Information ]================
State : Disabled
Authentication : Disabled

##########################################################

root@07-FirePower:/home/admin# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 1.1.7.1 0.0.0.0 UG 0 0 0 br1
1.1.7.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
127.0.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br0
127.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 tap0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 tun1
169.254.1.0 0.0.0.0 255.255.255.248 U 0 0 0 tap_nlp
#
#
#
root@07-FirePower:/home/admin# cat /ngfw/etc/sysconfig/network-devices/ifcfg-int
cat: /ngfw/etc/sysconfig/network-devices/ifcfg-int: No such file or directory
#
#
root@07-FirePower:/home/admin# ls -la /ngfw/var/common
total 49196
drwxrwxr-x 2 root detection 4096 Oct 19 08:21 .
drwxr-xr-x 13 root root 4096 Nov 21 02:01 ..
-rw-r--r-- 1 root root 50366078 Oct 19 08:21 results-10-19-2017--71231.tar.gz

[Lucas Phelps]

Mohammed, I did the 'configure network ipv4 manual 1.1.4.2 255.255.255.0 1.1.4.1' but it still did not set the IPv4 Default Route in the 'show network'

Broken FTD:

> show network
===============[ System Information ]===============
Hostname : 04-FirePower
Domains : contoso.com
DNS Servers : 172.16.1.1
10.16.1.1
Management port : 8305

======================[ br1 ]=======================
State : Enabled
Channels : Management & Events
Mode : Non-Autonegotiation
MDI/MDIX : Auto/MDIX
MTU : 1500
MAC Address : 70:DF:2F:CD:0C:C3
----------------------[ IPv4 ]----------------------
Configuration : Manual
Address : 1.1.4.2
Netmask : 255.255.255.0
Broadcast : 1.1.4.255
----------------------[ IPv6 ]----------------------
Configuration : Disabled

===============[ Proxy Information ]================
State : Disabled
Authentication : Disabled

##############################################################


root@04-FirePower:/home/admin# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 169.254.1.1 0.0.0.0 UG 0 0 0 tap_nlp
1.1.4.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
127.0.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br0
127.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 tap0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 tun1
169.254.1.0 0.0.0.0 255.255.255.248 U 0 0 0 tap_nlp
##############################################################

cat /ngfw/etc/sysconfig/network-devices/ifcfg-int
# automatically generated on Fri Jul 7 17:18:55 UTC 2017

INTERNAL_ROUTE_ENABLED=1
INTERNAL_GATEWAY_DEVICE=tap_nlp
INTERNAL_GATEWAY=169.254.1.1
INTERNAL_GATEWAY_V6=fd00:0:0:1::1/64

#####################################################

root@04-FirePower:/home/admin# ls -la /ngfw/var/common
total 48856
drwxrwxr-x 2 root detection 4096 Oct 19 08:25 .
drwxr-xr-x 13 root root 4096 Nov 23 03:33 ..
-rw-r--r-- 1 root root 50016691 Oct 19 08:25 results-10-19-2017--71701.tar.gz

[Lucas Phelps]

Here is a WORKING FTD:

> show network
===============[ System Information ]===============
Hostname : 07-FirePower
Domains : contoso.com
DNS Servers : 172.16.1.1
10.16.1.1
Management port : 8305
IPv4 Default route
Gateway : 1.1.7.1  << Notice the gateway

======================[ br1 ]=======================
State : Enabled
Channels : Management & Events
Mode : Non-Autonegotiation
MDI/MDIX : Auto/MDIX
MTU : 1500
MAC Address : 00:62:EC:93:F7:83
----------------------[ IPv4 ]----------------------
Configuration : Manual
Address : 1.1.7.2
Netmask : 255.255.255.0
Broadcast : 1.1.7.255
----------------------[ IPv6 ]----------------------
Configuration : Disabled

===============[ Proxy Information ]================
State : Disabled
Authentication : Disabled

################################################################

root@07-FirePower:/home/admin# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 1.1.7.1 0.0.0.0 UG 0 0 0 br1
1.1.7.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
127.0.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br0
127.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 tap0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 tun1
169.254.1.0 0.0.0.0 255.255.255.248 U 0 0 0 tap_nlp

################################################################

root@07-FirePower:/home/admin# cat /ngfw/etc/sysconfig/network-devices/ifcfg-int
cat: /ngfw/etc/sysconfig/network-devices/ifcfg-int: No such file or directory

################################################################

root@07-FirePower:/home/admin# ls -la /ngfw/var/common
total 49196
drwxrwxr-x 2 root detection 4096 Oct 19 08:21 .
drwxr-xr-x 13 root root 4096 Nov 21 02:01 ..
-rw-r--r-- 1 root root 50366078 Oct 19 08:21 results-10-19-2017--71231.tar.gz

[Mohammed al Baqari]

Here is the problem.

cat /ngfw/etc/sysconfig/network-devices/ifcfg-int
# automatically generated on Fri Jul 7 17:18:55 UTC 2017

*INTERNAL_ROUTE_ENABLED=1*
INTERNAL_GATEWAY_DEVICE=tap_nlp
INTERNAL_GATEWAY=169.254.1.1
INTERNAL_GATEWAY_V6=fd00:0:0:1::1/64

This means that FTD will ignore your default gateway. If you know vi editor
in linux, use it to change this to '0'.

vi /ngfw/etc/sysconfig/network-devices/ifcfg-int

Then change INTERNAL_ROUTE_ENABLED=*0*

After that reboot the box. Otherwise open a tac case and they will do it
for you.

To give some context on this parameter. This parameter will change to '1'
if FTD default gateway flaps intermittently for any reason. In that case
FTD will set this parameter to '1' to ignore the default gateway and avoid
flapping. Now the next question why this changed to '1'. I think its a bug
in 6.2.2 as I have seen it in multiple FTDs once upgrading to this version.
But this could be for many other reasons.

What I suggest is to fix it now and monitor. If its changed again to '1'
then we need to look at the logs and see what is happening.

[Lucas Phelps]

Thanks, I'll give this a shot tonight.

On a side note, is there a reason that my working Firepower '07-FirePower' shows no file exists when running the 'cat /ngfw/etc/sysconfig/network-devices/ifcfg-int' command?    Is that a problem?

[Lucas Phelps]

I tried two different fixes. First, I used vi to create the file /ngfw/etc/sysconfig/network-devices/ifcfg-int and entered the text below, saved, and rebooted..no improvement.

root@04-FirePower:/home/admin# cat /ngfw/etc/sysconfig/network-devices/ifcfg-int
INTERNAL_ROUTE_ENABLED=0
INTERNAL_GATEWAY_DEVICE=tap_nlp
INTERNAL_GATEWAY=169.254.1.1
INTERNAL_GATEWAY_V6=fd00:0:0:1::1/64

Second, I tried to delete the file completely and reboot (as the file isn't found on my working FTDs) but that didn't help either.

[tazevedo · Accepted solution]

Hi there,

I just had the same issue and fixed.

i think you're editing the wrong file:

i noticed you're working with the /ngfw/etc/. Try the /etc/sysconfig as below.

:/etc# cat /etc/sysconfig/network-scripts/ifcfg-internal-route
# automatically generated on Wed Feb 28 10:14:59 UTC 2018

INTERNAL_ROUTE_ENABLED=0
INTERNAL_GATEWAY_DEVICE=tap_nlp
INTERNAL_GATEWAY=169.254.1.1

INTERNAL_GATEWAY_V6=fd00:0:0:1::1/64

After editing the file i just reconfigured the management interface (to do a interface reload) and it worked.

I would advise to reboot the FTD after to validate if the config sticks.(didnt have a chance to do it yet).

Please let me know.

Thanks

[Lucas Phelps · Accepted solution]

Nice find!   That did the trick.  I did vi /etc/sysconfig/network-scripts/ifcfg-internal-route and changed the INTERNAL_ROUTE_ENABLED=1 to INTERNAL_ROUTE_ENABLED=0

Then I saved the file and exited Expert mode.  Then issued a configure network ipv4 manual 1.1.4.2 255.255.255.0 1.1.4.1 to reconfigure the management IP.

Worked just fine. The show network command now shows the gateway.

[tazevedo]

Great news!

Did you reboot the FTD to see if the configuration sticks?

Thanks