NEW Firepower 4110 Very Slow Throughput

[matty-boy]

Hi all,

I'm with a customer trying to figure out why their new FPR4110 is exhibiting slow throughput (about 500Kbps). The FPR was installed and set up by another company.

Internet access via the FPR is very slow. Pages load eventually but something that should take about 1 second currently takes 10-20 seconds!

We are using jperf for testing. If we route through the FPR, jperf reports around 500Kbps. If we change routing in both directions to bypass the FPR, we get about 50Mbps which is what we would expect.

Everything looks fine with the LAN and the FPR configuration. The FPR is effectively routing on a stick through a single port-channel consisting of 2 10Gig links with multiple sub-interfaces.

We tried a prefilter rule (to match any traffic from our test machine to any destination) - no joy.

We tried shutting down each member of the port-channel in turn - no joy.

Tried replacing fibre patches and SFPs - no joy.

Failed over to secondary FPR - no joy.

The ONLY thing I have noticed is that the FXOS chassis manager is reporting that it is not properly licensed. Could this cause the chassis to throttle the throughput?

Thanks,

Matt.

[Marvin Rhoads]

That's definitely odd. I have customers running FTD on the lower end 2100 series and they are getting the full 1 Gbps that their provider pipe allows.

Throughput should not be throttled due to licensing.

Given what you mentioned about it being used as a single armed deployment, I'd look closely at something at the layer 1 (physical) or layer 2 (the etherchannel and trunking subinterfaces) domains.

[matty-boy]

Hi Marvin,

Thank you for the response.

One more observation: pinging through the FPR, results in 5 or 6 successes and then a drop, then 5 or 6 successes and then a drop (and so it goes on....).

Any clues??

[Marvin Rhoads]

Regarding the observed icmp packet loss, check if your Firepower is running 6.3.0. If so, then you may be hitting this bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvo80715/?reffering_site=dumpcr

The fix is to disable the SSL hardware offload feature as follows:

>system support ssl-hw-offload disable
(This command will reboot the FTD)

The fixed release for the defect was 6.3.0.3.
Of course, that might not be it - it may be something related to the root problem that's causing the performance problem. The behavior almost sounds like it's arp-related.

Is/was the etherchannel perchance going to multiple internal devices like a VSS or VPC cluster?

[matty-boy]

Hi Marvin,

Thank you again for your input.

The 4110s are running FXOS 2.3 and Firepower 6.2.3.10 so the issue does not appear to be related to the bug you mention.

The Etherchannel is indeed going to multiple devices: A pair of Cisco 6807 in VSS.

That said, we did try shutting down one member of the Etherchannel but this made no difference.

Thanks,
Matt.

[Marvin Rhoads]

Have you tried testing with the interface not being an Etherchannel at all (even single member)?

Your 4110 is in routed mode- correct?

Would it be possible to open a TAC case on the issue and have an engineer work with you directly?

[matty-boy]

Hi Marvin,

We didn't try removing the Etherchannel completely but we did try shutting down one member of the EC (and then the other).

We do have a TAC case open but I wanted to bring this issue up with with wonderful people of the community and see if anyone had any bright ideas :)

Cheers!

Matt.

[VijayKate6703]

Did you find any possible cause on this issue. I looks to have similar issue in our environment