topic CPU 100% during FTP in Network Security

CPU 100% during FTP

stuart.jones — Mon, 11 Mar 2019 09:52:05 GMT

Hi,

I have an issue with my Pix, during an FTP transfer the CPU goes upto 100% and stays there during the period of the transfer. When the transfer ends it goes back to 0%.

The Pix is a 535 with 512MB, the two interfaces transfering the data are both gigabit and are running at gigabit.

The only connection in the 'sh conn' is the one in progress the FTP so its not being hammered from all over yet the CPU spikes upto 100% and stays there then drops back to 0%.

This started because the server guys were complaining about slow transfers and were pointing at the FW, i didnt think for a second that one transfer would cause a Pix 535 with gigabit interfaces to flatline like it does.

Is there something obvious wrong ? or something i could do to identify the problem or a workaround ?

Any input would be great, much head scratching going on here.

Thanks

Stu

Re: CPU 100% during FTP

vitripat — Tue, 27 Mar 2007 04:27:14 GMT

Hey Stuart,

Certainly we need to look into this. A single connection should ideally not spike the CPU to 100% and keep it constantly there.

First off, Id like to know if any changes were made on PIX due the complain from the server guys for slow transfer.

What code is running on PIX? Is it possible to take output of "show proc", "sh cpu usage" & "sh mem" just before the start of FTP transfer and then during the FTP transfer. Try keeing the time difference between the two outputs to 10-15 seconds. This might tell us which process is actually consuming CPU so much. Also, if you could provide the "sh interface" and "show run" output, it would be of great help.

Regards,

Vibhor.

Re: CPU 100% during FTP

zubairjalal — Tue, 27 Mar 2007 09:38:37 GMT

What you are seeing is basically a sort of DOS attack. The server guys must be using a third party software to do the FTP. That software must be opening simenltaneous streams to do the ftp...just like a download acclerator. When the ftp process is going on and see the output of show conn count. If the current and the maximum values are almost same then it is a DOS attack..maybe not an intentional dos attack. Possibly you should try to limit the number of connections that the server guys can open..this you can do if you have a self static in place.

regards

Zubair

Re: CPU 100% during FTP

stuart.jones — Tue, 27 Mar 2007 21:57:43 GMT

Zubair,

Thanks for your reply.

This is an output of the sh conn when i had the problem.

------------------ show conn count ------------------

2 in use, 198 most used

So i dont think it ties in with your description.

Thanks

Stu

Re: CPU 100% during FTP

abinjola — Wed, 28 Mar 2007 00:14:01 GMT

at least give us the info of version you are running on FW..??

Re: CPU 100% during FTP

stuart.jones — Wed, 28 Mar 2007 01:50:50 GMT

Vibhor,

No changes have been made to the Pix that i am aware of.

I am unable to get snapshot until the next transfer is run, but attached is output from when the situation was occurring.

Hope this helps, will get a snapshot of status before next transfer and during as you suggested when i can.

Thanks

Stu

Re: CPU 100% during FTP

abinjola — Wed, 28 Mar 2007 03:27:54 GMT

heyy Stuart..i missed the output that you posted earlier..now I see you are running 6.3.4

has the issue been there right from day one.?

is the issue only with "FTP" transfers ?

when the ftp transfer is complete does the cpu usage go down to normal ?

Is there any IP Audit commands configured on the fw ?

There is one more things that I notice that the broadcasts on your inside Interface

997532413 packets input, 11075500019630 bytes, 6271442 no buffer

Received 106715356 broadcasts, 0 runts, 0 giants

Currently the Broadcasts are around 10%... the above is an instantanious value...which might go up if the Broadcast storm is heavy ...and this would definitely help us to shoot up the cycles..:-)

Re: CPU 100% during FTP

stuart.jones — Wed, 28 Mar 2007 03:36:29 GMT

abinjola,

Not sure i believe it might have but has only been highlighted to me in the last few days.

Not sure about the FTP transfer either though i did check last night when there were 24 connection and the CPU was only around 60%, though the transfers last night were not going through the two gig interfaces, it was just using one of them.

As for the CPU usage returning to normal yes, please see the snapshot of the PDM i posted.

Hadnt noticed the broadcasts i will keep an eye on that, i was worried about the 'no buffers'

Thanks

Stu

Re: CPU 100% during FTP

abinjola — Wed, 28 Mar 2007 04:12:59 GMT

Cool..24 connections and CPU was still 60 %......that too without tunnels ...no failover..no SNMP..dont ya think its still high..?

The other thing that I noticed in the sh proc is that the these processes are consuming too much of run time

i82543_timer

557poll are interface

1942469178 0567401c 6252/8192 pix/intf0

now the i82543_timer and the 557poll are interface card's driver polling threads corresponding two types of interfaces in

your pix 535: gb-ethernet (i82543) and ethernet (i82559

Soo much of broadcast again supports this explanation...

how much of CPU would you think is normal ? or in the pleasant days how much CPU usage did you observe ?

Is there any ways to control this traffic and secondly how feasible is it for you to upgrade to 6.3.5 (to rule out the possibility of any bug hitting us )