topic Re: Processing Load Percentage = 100 in Network Security

Processing Load Percentage = 100

Diego Armando Cambronero Arias — Sun, 10 Mar 2019 12:03:42 GMT

Hello,

Guys I have been working with 3 IPS 2 4260 and a 4270 since yesterday I have noticed that the Inspection LOAD is RED. On the 4260 the inspection load is.

SJDetec1# sh statistics virtual-sensor | inc Load

Processing Load Percentage = 99

On the 4270!!!!!!!

DCDetect1# sh statistics virtual-sensor | inc Load

Processing Load Percentage = 100

DCDetect1# sh statistics analysis-engine

Analysis Engine Statistics

Number of seconds since service started = 174759

The rate of TCP connections tracked per second = 0

The rate of packets per second = 4711

The rate of bytes per second = 8402

Receiver Statistics

Total number of packets processed since reset = 823334516

Total number of IP packets processed since reset = 822979042

Transmitter Statistics

Total number of packets transmitted = 823478816

Total number of packets denied = 0

Total number of packets reset = 0

Fragment Reassembly Unit Statistics

Number of fragments currently in FRU = 0

Number of datagrams currently in FRU = 0

TCP Stream Reassembly Unit Statistics

TCP streams currently in the embryonic state = 0

TCP streams currently in the established state = 0

TCP streams currently in the closing state = 0

TCP streams currently in the system = 0

TCP Packets currently queued for reassembly = 0

The Signature Database Statistics.

Total nodes active = 16115

TCP nodes keyed on both IP addresses and both ports = 3438

UDP nodes keyed on both IP addresses and both ports = 29

IP nodes keyed on both IP addresses = 1715

Statistics for Signature Events

Number of SigEvents since reset = 153308490

For example in the 4270 we are passing almost nothing through the sensor... ANd its working in promiscuos mode. Why is the Inspection Load that High? In the 4260 is the same.. It is working in promiscuos mode.. There are alarms for missed packets as well.

I have been seeing several Discutions for the same reason but none has a fix. The Issue with the inspection load is random. DUring the day sometimes it high and sometimes is low.

Cisco Intrusion Prevention System, Version 7.0(2)E4

Signature Update S499.0

Any advice will be really appreciated.

Diego.

Re: Processing Load Percentage = 100

Justin Teixeira — Wed, 14 Jul 2010 14:03:57 GMT

Hi Diego,

The first thing to try would be to restore all your signatures to default (provided they're tuned), and disable all custom signatures and see if you still experience the problem. Overtuning of signatures (for example, enabling every signature on the device) is the most likely reason for the processing load being that high.

Best Regards,

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 14:10:51 GMT

Hi I have been tunning the signatures since the las t months. I have tunned arround 900 signatures. Or at least the action is what I have tunned. If I set them all to their default values I will loose all the work done right?

In the 4270 the traffic passed is very very little. For example. In the last 10 hours an Only have 96 events. So it's nothing for a 4270.

Something else to try?

Re: Processing Load Percentage = 100

Christopher Dreier — Wed, 14 Jul 2010 15:56:57 GMT

Diego,

What is your traffic profile? Can you provide packet captures during a time of high processing load along with a show tech? What changed in your network when the issue started?

You are seeing quite a few signature events:

Analysis Engine Statistics

Number of seconds since service started = 174759

...

Statistics for Signature Events

Number of SigEvents since reset = 153308490

From the limited amount of information we have, it appears that you may be experiencing traffic spikes throughout the day that lead to many events firing which cause the IPS to become oversubscribed. There is quite a bit of data that we will need to examine to validate this. Please open a TAC case and we can begin to investigate.

Thank you,
Blayne Dreier
Cisco TAC IDS Team

**Please check out our Podcast**
TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

Re: Processing Load Percentage = 100

Justin Teixeira — Wed, 14 Jul 2010 16:31:32 GMT

Hi Diego,

You can backup your configuration with the copy "current-config" before restoring the signatures to default. You can then restore them from the backup later.

Best Regards,

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 16:39:55 GMT

Hi,

Just did it. Let's see if the INpection load goes high Again. The Missed packet % is still in a 5 % and it is a Yellow Alarm in the health status.

Thank you very much. I will keep u updated.

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 17:03:41 GMT

I's Seeing the following in the Error Events.

Several messages of:

Inline data bypass has started

Hundreds of:

transmitPacket: Error TX Queue full, no lost buf yet if=7

And some of:

A global correlation update failed: Failed to open a TLS connection to HTTP proxy server at 10.1.4.5:8002 : TLS connection failed

Messages, like this one, in the category - Reputation update failure - were logged 24 times in the last 7200 seconds. name=errUnclassified

So the IPS is actually doing the Bypass. The Traffic being sent to this IPS 4270 is MINIMAL.. So i do not understand the reason why it has this problem.. And It is a 4270!!! even if the traffic is huge it must handle it without any problem...that is just imposible with the traffic that Im sending to this IPS it is no tpossible.

Thanks

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 17:08:22 GMT

Gest what.. ANOTHER BUG

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtc18038

Cisco cisco cisco

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 17:12:04 GMT

However it is fixed in the version 7.0(2)E4

The same that I have.

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 17:18:52 GMT

Inspection Load is at 100% Again.

If someone can help me, I would really appreciate it. Maybe someone from the TAC checkout the TOPIC maybe there is something there. Thank you Guys

Re: Processing Load Percentage = 100

Justin Teixeira — Wed, 14 Jul 2010 17:24:27 GMT

Hi Diego,

I would go ahead and open a TAC case at this point for us to take a look at it.

Best Regards,

Re: Processing Load Percentage = 100

Diego Armando Cambronero Arias — Wed, 14 Jul 2010 17:52:13 GMT

Im going to open de case. I hope that someone from Costa Rica can take a look to this. Thanks