Download slows down then stops

[dannyh]

Hello,

For some time now we have been experiencing an issue on the network for a while. I haven't been able to figure it out.

When downloading a large file (usually 500 mb or higher) the download starts strong, then slows down, coming to a complete stop eventually. I've tested this at the router by plugging a laptop directly into our router. The download is fine there. But from the router, it goes downstream to our ASA then to our Core Switch. I tried the download directly out of the Core Switch as well and it seems like the problem starts when I am in the internal part of the network (ASA and downstream). I have been looking into QoS as other engineers have suggested. I am the only Network Engineer and I've been stuck on this problem for a while now. Can anyone please provide some insight to what may be happening on our network?

[balaji.bandi]

what ASA  Model and what code running.

can you post the configuation of ASA

show interface x/x (output of inside and outside interface)

also conencted switch port same information.

[dannyh]

Our firewall is ASA5516

Cisco Adaptive Security Appliance Software Version 9.8(1)
Firepower Extensible Operating System Version 2.2(1.47)
Device Manager Version 7.8(1)

ASA5516# show interface gi1/6
Interface GigabitEthernet1/6 "INSIDE", is up, line protocol is up
Hardware is Accelerator rev01, BW 1000 Mbps, DLY 10 usec
Auto-Duplex(Full-duplex), 1000 Mbps(1000 Mbps)
Input flow control is unsupported, output flow control is off
MAC address 7070.8b67.cc26, MTU 1500
IP address x.x.x.x, subnet mask x.x.x.x
115536804314 packets input, 28594720419160 bytes, 0 no buffer
Received 25 broadcasts, 0 runts, 0 giants
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
0 pause input, 0 resume input
0 L2 decode drops
130542297106 packets output, 48742919724822 bytes, 0 underruns
0 pause output, 0 resume output
0 output errors, 0 collisions, 0 interface resets
0 late collisions, 0 deferred
0 input reset drops, 5484 output reset drops
input queue (blocks free curr/low): hardware (1988/1820)
output queue (blocks free curr/low): hardware (2046/1544)
Traffic Statistics for "INSIDE":
115536351136 packets input, 26475939649120 bytes
130542297107 packets output, 46356548851548 bytes
12499626 packets dropped
1 minute input rate 4945 pkts/sec, 2527874 bytes/sec
1 minute output rate 4810 pkts/sec, 2027400 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 4862 pkts/sec, 2504546 bytes/sec
5 minute output rate 5126 pkts/sec, 2494771 bytes/sec
5 minute drop rate, 0 pkts/sec

ASA5516# show interface gi1/8
Interface GigabitEthernet1/8 "OUTSIDE", is up, line protocol is up
Hardware is Accelerator rev01, BW 1000 Mbps, DLY 10 usec
Auto-Duplex(Full-duplex), Auto-Speed(1000 Mbps)
Input flow control is unsupported, output flow control is off
MAC address 7070.8b67.cc28, MTU 1500
IP address x.x.x.x, subnet mask x.x.x.x
125099670710 packets input, 47663013544937 bytes, 0 no buffer
Received 1123 broadcasts, 0 runts, 0 giants
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
0 pause input, 0 resume input
0 L2 decode drops
109525961608 packets output, 27387825283606 bytes, 0 underruns
0 pause output, 0 resume output
0 output errors, 0 collisions, 0 interface resets
0 late collisions, 0 deferred
0 input reset drops, 3046 output reset drops
input queue (blocks free curr/low): hardware (1971/1819)
output queue (blocks free curr/low): hardware (2047/1515)
Traffic Statistics for "OUTSIDE":
125087585980 packets input, 45375182138145 bytes
109525961609 packets output, 25377800839868 bytes
554704543 packets dropped
1 minute input rate 5740 pkts/sec, 2419281 bytes/sec
1 minute output rate 5472 pkts/sec, 2130663 bytes/sec
1 minute drop rate, 14 pkts/sec
5 minute input rate 4861 pkts/sec, 2485024 bytes/sec
5 minute output rate 4573 pkts/sec, 2493384 bytes/sec
5 minute drop rate, 18 pkts/sec

CoreSwitch#show interface ten1/0/24
TenGigabitEthernet1/0/24 is up, line protocol is up (connected)
Hardware is Ten Gigabit Ethernet, address is 802d.bf53.bec0 (bia 802d.bf53.bec0)
Description: Link_To_ASA
Internet address is x.x.x.x/x
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 4/255, rxload 7/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:04:03, output 00:00:02, output hang never
Last clearing of "show interface" counters never
Input queue: 0/375/12414/0 (size/max/drops/flushes); Total output drops: 47544
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 29831000 bits/sec, 5998 packets/sec
5 minute output rate 18299000 bits/sec, 4778 packets/sec
21001904274 packets input, 9883857237083 bytes, 0 no buffer
Received 139 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
17613305792 packets output, 5297076469714 bytes, 0 underruns
Output 2 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 3 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

[balaji.bandi]

i see in ASA  and Switch side Drops :

ASA side :

0 input reset drops, 3046 output reset drops

Switch side : (can you let us know what switch mode and IOS code running)

Input queue: 0/375/12414/0 (size/max/drops/flushes); Total output drops: 47544

Note : also please confirm on switch or ASA do you have any QoS configured (if possible share show run from both ASA and Switch)

[dannyh]

Core Switch:

ROM: IOS-XE ROMMON
BOOTLDR: System Bootstrap, Version 17.6.1r[FC2], RELEASE SOFTWARE (P)

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 50 C9500-40X 16.12.08 CAT9K_IOSXE INSTALL

As for the show runs I cannot provide the whole thing. I am able to provide portions of the show run if you can specify the places that you need to see.

As for the QoS, no we don't have any configured. It is a small network and it wasn't implemented even before I joined the team (I am the only Network Engineer here). I tried configuring the interfaces that connect from Core Switch to the ASA with speed limits (1 gb bandwidth from ISP so I set the interfaces' limit to 1000 mbps).

[balaji.bandi]

Look at the ASA  throughputs ;

https://www.cisco.com/c/en/us/products/collateral/security/asa-firepower-services/datasheet-c78-742475.html

ASA  9.8 is old code, so upgrade to the latest, I believe it is 9.12 or 9.14, Does this ASA -5516-X  have SFR Module?

when you connect PC to switch and transfer you get more speeds ? (I think this was the test you did)

[dannyh]

What commands would I use for to see if we have SFR Module? I am not seeing any command references in the document you provided.

[balaji.bandi]

#show module

[dannyh]

Thank you. I am not sure what I am looking at, but it looks like it does have the sfr module.

Output: sfr FirePOWER Services Software Module ASA5516

[balaji.bandi]

why i was asked, if you have any policy going via SFR, you may see some download speed issue.

[dannyh]

so how can I disable this?

[balaji.bandi]

i would first check what is installed rather than disable and monitor

start with below guide :

https://www.cisco.com/c/en/us/support/docs/security/asa-firepower-services/118644-configure-firepower-00.html

[dannyh]

So we found something funny going on. The when connecting to the Wi-Fi we see that the downloads are working just fine. It is only on the ethernet connection that is causing this.

[balaji.bandi]

you need to troublesheet, how wifi flows compare to ethernet connection flows.

[dannyh]

Thank you. I'll take a look into WireShark. Seems like we are narrowing down closer thanks to your input and through what we are finding. Really appreciate it.