09-17-2014 04:55 PM - edited 03-11-2019 09:46 PM
Our ASA 5505 is running very slowly, and causing slow response times from the servers sitting behind it. I'm seeing cpu usage of ~80% for the most part, and show processes cpu-hog looks like this:
Process: ssh_init, PROC_PC_TOTAL: 2, MAXHOG: 11, LASTHOG: 11
LASTHOG At: 17:18:27 EDT Sep 3 2014
PC: 8063875 (suspend)
Process: ssh_init, NUMHOG: 2, MAXHOG: 11, LASTHOG: 11
LASTHOG At: 17:18:27 EDT Sep 3 2014
PC: 8063875 (suspend)
Call stack: 8063875 8135e4b 9281694 929a5e8 9296265 9283a7e 92837ff
92835d0 9272adf 926252c 928286a 92826e1 9271826 92622dc
Process: ssh_init, PROC_PC_TOTAL: 17, MAXHOG: 14, LASTHOG: 11
LASTHOG At: 20:15:05 EDT Sep 11 2014
PC: 8bc05ad (suspend)
Process: ssh_init, NUMHOG: 17, MAXHOG: 14, LASTHOG: 11
LASTHOG At: 20:15:05 EDT Sep 11 2014
PC: 8bc05ad (suspend)
Call stack: 8bc05ad 8bcd34d 8bcb29e 8bcb448 8bcc4d1 8bc5dc4 80626e3
Process: Dispatch Unit, PROC_PC_TOTAL: 999, MAXHOG: 55, LASTHOG: 13
LASTHOG At: 17:22:42 EDT Sep 17 2014
PC: 81aba19 (suspend)
Process: Dispatch Unit, NUMHOG: 999, MAXHOG: 55, LASTHOG: 13
LASTHOG At: 17:22:42 EDT Sep 17 2014
PC: 81aba19 (suspend)
Call stack: 81aba19 80626e3
Process: ssh_init, PROC_PC_TOTAL: 38638, MAXHOG: 15, LASTHOG: 13
LASTHOG At: 17:43:02 EDT Sep 17 2014
PC: 8bc05fc (suspend)
Process: ssh_init, NUMHOG: 38638, MAXHOG: 15, LASTHOG: 13
LASTHOG At: 17:43:02 EDT Sep 17 2014
PC: 8bc05fc (suspend)
Call stack: 8bc05fc 8bcd34d 8bcb29e 8bcb448 8bcc4d1 8bc5dc4 80626e3
I can't find any documentation that explains what the output of cpu-hog actually means, nor can I find anything explaining what ssh_init is (I can guess) and what it means if I see a number of them suspended. sh resource usage also shows:
Resource Current Peak Limit Denied Context
SSH 2 5 5 162 System
Syslogs [rate] 542 3127 N/A 0 System
Conns 5180 9256 25000 0 System
Xlates 223 1254 N/A 0 System
Hosts 3543 6839 N/A 0 System
Conns [rate] 238 3001 N/A 0 System
Inspects [rate] 19 226 N/A 0 System
Which seems like a lot of denied ssh sessions. sh processes cpu usage sorted shows:
PC Thread 5Sec 1Min 5Min Process
08c1215b d5183090 25.3% 2.0% 0.4% ssh_init
081ab744 d51ab260 21.3% 19.7% 20.4% Dispatch Unit
092cc2d4 d51a0718 2.5% 3.3% 3.3% esw_stats
08bf68e6 d51a53e0 1.0% 1.0% 1.0% Logger
Lastly, I'm seeing a quickly increasing number of dropped packets on both my inside and outside interfaces with anywhere from 10-20 packets per second being dropped, which I suppose explains why everything is so slow on the network. I was wondering if I could clear those ssh_init sessions in some way to regain some cpu and see if that helps. Is there a way to do that other than rebooting?
Solved! Go to Solution.
09-18-2014 01:31 PM
Hello;
If the hogs are actually causing an issue, you should see overruns on the interfaces, I mean a lot. Are you seeing those? Have you take a capture and compare the time when it gets and and how much it takes for the packet to leave the ASA?
Mike Rojas
09-18-2014 01:31 PM
Hello;
If the hogs are actually causing an issue, you should see overruns on the interfaces, I mean a lot. Are you seeing those? Have you take a capture and compare the time when it gets and and how much it takes for the packet to leave the ASA?
Mike Rojas
09-19-2014 07:23 AM
Hi,
I see. No, I have 0 overruns. It appears I was making an erroneous assumption that if my cpu usage is high, it has to be some sort of hogging process. Instead maybe it's simply the ASA is getting more traffic than it can handle.
I have not done a packet capture like the one you described. I'll Google it and figure out how to do one. Will it add any additional load on an already bogged down ASA to do so?
09-19-2014 09:52 AM
It should cause a bit latency, but almost unnoticeable.
The capture would be stored on RAM, here is a link that you can use:
https://supportforums.cisco.com/document/69281/asa-using-packet-capture-troubleshoot-asa-firewall-configuration-and-scenarios
If you think you are having an oversubscription, you can sum up the the bytes per-second, input and output on each interface from the output "show traffic" excluding the internal Data interfaces. That way you would be able to see the throughput of the firewall.
Mike.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide