High latency ICMP SNMP tool to ASA 5540

[Nelson Neto]

Hello,

I'm having trouble with high response times with icmp via CheckMK tool for equipment that is housed behind a DMZ in a Cisco ASA 5540.
 
I am in doubt if it is a problem of channel capacity or packet processing.
 

The response time presented in the monitoring tool is from 950ms to 995ms.

The monitor is on the side of the interface "inside" and the monitors are behind of the "DMZ_ASA" interface

Interface Configuration:

!
interface GigabitEthernet0/1 <<<<<<<<<<< Where is ICMP monitoring Check_MK
nameif inside
security-level 100
ip address x.x.x.x x.x.x.x standby x.x.x.x
!
Interface GigabitEthernet0/1 "inside", is up, line protocol is up
Hardware is i825xxGB rev03, BW 1000 Mbps, DLY 10 usec
Auto-Duplex(Full-duplex), Auto-Speed(1000 Mbps)
Input flow control is unsupported, output flow control is off
MAC address xxxx.xxxx.xxxx, MTU 1500
IP address x.x.x.x, subnet mask x.x.x.x
40873611552 packets input, 24161952752164 bytes, 0 no buffer
Received 584349 broadcasts, 0 runts, 0 giants
20675385 input errors, 0 CRC, 0 frame, 20675385 overrun, 0 ignored, 0 abort
0 pause input, 0 resume input
0 L2 decode drops
41475878351 packets output, 21624031950460 bytes, 20368959 underruns
0 pause output, 0 resume output
0 output errors, 0 collisions, 2 interface resets
0 late collisions, 0 deferred
0 input reset drops, 0 output reset drops, 0 tx hangs
input queue (blocks free curr/low): hardware (255/230)
output queue (blocks free curr/low): hardware (255/0)
Traffic Statistics for "inside":
40873402031 packets input, 23385913357886 bytes
41496247310 packets output, 20856791981497 bytes
97146495 packets dropped
1 minute input rate 4727 pkts/sec, 2791783 bytes/sec
1 minute output rate 4621 pkts/sec, 1688935 bytes/sec
1 minute drop rate, 7 pkts/sec
5 minute input rate 5123 pkts/sec, 3136113 bytes/sec
5 minute output rate 5060 pkts/sec, 2327510 bytes/sec
5 minute drop rate, 6 pkts/sec
Control Point Interface States:
Interface number is 4
Interface config status is active
Interface state is active
--------

!
interface GigabitEthernet0/2.100 <<<<<<<<<< Where are the monitored equipment
vlan 100
nameif DMZ_ASA
security-level 100
ip address x.x.x.x x.x.x.x standby x.x.x.x
ospf priority 50
!
Interface GigabitEthernet0/2.100 "DMZ_ASA", is up, line protocol is up
# Attention: This interface is located in a PCI-e x0 slot. For #
# optimal throughput, install the interface in a PCI-e x42 slot #
# if one is available. Refer to 'show controller slot'. #
Hardware is i825xxGB rev03, BW 1000 Mbps, DLY 10 usec
VLAN identifier 100
MAC address xxxx.xxxx.xxxx, MTU 1500
IP address x.x.x.x, subnet mask x.x.x.x
Traffic Statistics for "DMZ_ASA":
33826024689 packets input, 23584008836284 bytes
31201234329 packets output, 20069081816413 bytes
37069218 packets dropped
Control Point Interface States:
Interface number is 54
Interface config status is active
Interface state is active
Control Point Vlan10 States:
Interface vlan config status is active
Interface vlan state is UP
--------

I reviewed the "service-policy" and I do not think it's a QOS problem, but feel free to check it out.

Global policy:
Service-policy: global_policy
Class-map: inspection_default
Inspect: dns dns_map, packet 668826650, drop 91080, reset-drop 0
message-length maximum client auto, drop 0
message-length maximum 512, drop 0
dns-guard, count 331717534
protocol-enforcement, drop 266
nat-rewrite, count 0
Inspect: ftp, packet 540211, drop 0, reset-drop 0
Inspect: h323 h225 _default_map_h323map, packet 23737905, drop 0, reset-drop 76
tcp-proxy: bytes in buffer 0, bytes dropped 4086
h245-tunnel-block drops 0 connection
Inspect: h323 ras _default_map_h323_map, packet 127823, drop 0, reset-drop 0
h245-tunnel-block drops 0 connection
Inspect: ip-options _default_ip_options_map, packet 5590, drop 0, reset-drop 0
Router Alert: allow 5590, clear 0
Inspect: netbios, packet 826680417, drop 0, reset-drop 0
Inspect: rsh, packet 1761, drop 30, reset-drop 0
Inspect: sqlnet, packet 143704426, drop 0, reset-drop 0
Inspect: sunrpc, packet 21261, drop 3, reset-drop 716
tcp-proxy: bytes in buffer 0, bytes dropped 84
Inspect: tftp, packet 6947842, drop 20, reset-drop 0
Inspect: sip , packet 175775173, drop 1, reset-drop 0
tcp-proxy: bytes in buffer 0, bytes dropped 90734
Inspect: xdmcp, packet 24, drop 3, reset-drop 0
Inspect: rtsp, packet 691, drop 0, reset-drop 0
tcp-proxy: bytes in buffer 0, bytes dropped 0
Inspect: icmp, packet 737287799, drop 1961433, reset-drop 0

-------------

I also checked the processing and it seems normal.

------------------ show memory ------------------

Free memory: 787486032 bytes (37%)
Used memory: 1359997616 bytes (63%)
------------- ----------------
Total memory: 2147483648 bytes (100%)

------------------ show conn count ------------------

8795 in use, 95265 most used

------------------ show xlate count ------------------

3312 in use, 8853 most used

------------------ show cpu usage ------------------

CPU utilization for 5 seconds = 44%; 1 minute: 45%; 5 minutes: 44%

Please inform me if necessary any further configuration or testing.

Regards,

[Nelson Neto]

I wonder if anyone has an answer to this problem I'm having.

[Marvin Rhoads]

Ping (icmp echo request / reply) is notoriously inaccurate if there's any congestion at all on the path. The ASA does appear to be showing a fairly high number of packet drops.

Is it possible to use any tcp-based monitoring like WMI or even tcping?

[Nelson Neto]

Hello @Marvin Rhoads

Thank you for your attention in this case.
I am studying the feasibility of implementing the TCP monitoring you requested.

More information about this case:

1 - Only ICMP traffic is affected in this case? Or is all the communication from the "inside" interface to the "DMZ_ASA" interface slow?

Test from another computer on the same network and I did not have any losses.

2 - SNMP CheckMK only has high MS problems for the computers behind the "DMZ_ASA" interface, or are there teams behind other interfaces that are also with the MS high?

I got access to the front end of the tool (not the server) and so I observe the status of losses is not at all times.
The image that we shared the high response times are 900ms with 60% loss. What I'm noticing is that there are packet losses at some times of the day and not every day, with no apparent pattern of behavior and this seems to happen for the DMZs behind the inside
 

3 - ICMP test of other equipment in the same network as the CheckMK monitoring server for the computers that are in the interface network "DMZ_ASA" also has the MS high?

As mentioned above, test from another computer in the same network without problems with packet loss or high response times.
I have the suspicion that happens at some time but without finding an apparent relationship.

4 - Has any change been made recently?

I have seen that since almost the end of September the level of CPU usage has increased from 25% to 50% approx. I have a minor change implemented on that date.

Would you have any comments on that?

Regards,

[Nelson Neto]

Some information, this only happens with ICMP and happens every day at random times.

Does anyone know if there is any configuration in ASA that prioritizes ICMP traffic?
I know that for example Nexus 5k does not prioritize ICMP by default, does this also happen with ASA?

Can someone help me with this case?