03-15-2019 03:39 AM
I am observing ARP input Queue full and dropping packets.
At the same time Observing CPU high due to PID 9 which belongs to ARP Input. Kindly suggest solution, customer is dropping connections to their servers.
007152: Mar 15 12:03:32.816: %SYS-1-CPURISINGTHRESHOLD: Threshold: Total CPU Utilization(Total/Intr): 91%/26%, Top 3 processes(Pid/Util): 9/46%, 211/7%, 267/2%
9 107058250 495013478 216 0.47% 0.43% 0.44% 0 ARP Input
tnp_ctt_ctp34_01_sw_01#show ip traffic | in input
Drop due to input queue full: 1860186
Kindly suggest a solution
03-15-2019 04:33 AM
Hello,
Can you please issue the following commands and give the outputs:
show ver
show ip traffic
show int | i Input queue|line protocol is up
Remi Astruc
03-15-2019 05:18 AM
Show ver :
Cisco IOS Software, C3750 Software (C3750-IPSERVICESK9-M), Version 12.2(55)SE3, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2011 by Cisco Systems, Inc.
Compiled Thu 05-May-11 16:29 by prod_rel_team
Image text-base: 0x01000000, data-base: 0x02F00000
ROM: Bootstrap program is C3750 boot loader
BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(44)SE5, RELEASE SOFTWARE (fc1)
tnp_ctt_ctp34_01_sw_01 uptime is 45 weeks, 3 days, 2 hours, 23 minutes
System returned to ROM by power-on
System restarted at 11:45:29 SAST Tue May 1 2018
System image file is "flash:c3750-ipservicesk9-mz.122-55.SE3.bin"
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
If you require further assistance please contact us by sending email to
export@cisco.com.
cisco WS-C3750G-24TS-1U (PowerPC405) processor (revision H0) with 131072K bytes of memory.
Processor board ID FOC1446V3Z9
Last reset from power-on
4 Virtual Ethernet interfaces
28 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.
512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address : 08:17:35:41:E6:00
Motherboard assembly number : 73-10219-09
Power supply part number : 341-0098-02
Motherboard serial number : FOC14464P62
Power supply serial number : AZS143912PF
Model revision number : H0
Motherboard revision number : A0
Model number : WS-C3750G-24TS-S1U
System serial number : FOC1446V3Z9
Top Assembly Part Number : 800-26859-03
Top Assembly Revision Number : B0
Version ID : V05
CLEI Code Number : COMB600BRA
Hardware Board Revision Number : 0x09
Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 28 WS-C3750G-24TS-1U 12.2(55)SE3 C3750-IPSERVICESK9-M
Configuration register is 0xF
Show Ip Traffic:
IP statistics:
Rcvd: 278594504 total, 208859258 local destination
0 format errors, 0 checksum errors, 0 bad hop count
0 unknown protocol, 69735246 not a gateway
0 security failures, 0 bad options, 1123742 with options
Opts: 0 end, 0 nop, 0 basic security, 0 loose source route
0 timestamp, 0 extended security, 0 record route
0 stream ID, 0 strict source route, 1123742 alert, 0 cipso, 0 ump
0 other
Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble
0 fragmented, 0 couldn't fragment
Bcast: 197910334 received, 0 sent
Mcast: 0 received, 0 sent
Sent: 12600531 generated, 0 forwarded
Drop: 364355 encapsulation failed, 0 unresolved, 0 no adjacency
0 no route, 0 unicast RPF, 0 forced drop
0 options denied, 0 source IP address zero
ICMP statistics:
Rcvd: 0 format errors, 0 checksum errors, 1 redirects, 27980 unreachable
1235104 echo, 117634 echo reply, 33 mask requests, 0 mask replies, 0 quench
0 parameter, 11 timestamp, 0 info request, 11 other
0 irdp solicitations, 0 irdp advertisements
Sent: 0 redirects, 5895 unreachable, 117735 echo, 1235104 echo reply
0 mask requests, 0 mask replies, 0 quench, 11 timestamp
0 info reply, 0 time exceeded, 0 parameter problem
0 irdp solicitations, 0 irdp advertisements
TCP statistics:
Rcvd: 797085 total, 0 checksum errors, 446 no port
Sent: 1045663 total
UDP statistics:
Rcvd: 206681337 total, 10 checksum errors, 185575835 no port
Sent: 10196153 total, 0 forwarded broadcasts
BGP statistics:
Rcvd: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh, 0 unrecognized
Sent: 0 total, 0 opens, 0 notifications, 0 updates
0 keepalives, 0 route-refresh
EIGRP-IPv4 statistics:
Rcvd: 0 total
Sent: 0 total
PIMv2 statistics: Sent/Received
Total: 0/0, 0 checksum errors, 0 format errors
Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0, Hellos: 0/0
Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0
Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
State-Refresh: 0/0
IGMP statistics: Sent/Received
Total: 0/0, Format errors: 0/0, Checksum errors: 0/0
Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0
DVMRP: 0/0, PIM: 0/0
OSPF statistics:
Rcvd: 0 total, 0 checksum errors
0 hello, 0 database desc, 0 link state req
0 link state updates, 0 link state acks
Sent: 0 total
0 hello, 0 database desc, 0 link state req
0 link state updates, 0 link state acks
ARP statistics:
Rcvd: 635583950 requests, 3023824 replies, 335149 reverse, 0 other
Sent: 1965 requests, 22512202 replies (0 proxy), 0 reverse
Drop due to input queue full: 1863016
#show int | i Input queue|line protocol is up
Vlan1 is up, line protocol is up
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Vlan2 is up, line protocol is up
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Vlan300 is up, line protocol is up
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/1 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/2 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/3 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/4 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/5 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/6 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/7 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
GigabitEthernet1/0/25 is up, line protocol is up (connected)
Input queue: 1/75/0/0 (size/max/drops/flushes); Total output drops: 200
GigabitEthernet1/0/26 is up, line protocol is up (connected)
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Loopback0 is up, line protocol is up
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
03-15-2019 06:54 AM
Hi,
The problem is well related to ARP traffic.
During the problem, please run few times the 2 commands below, to get a sampled base:
sh clock
sh ip traffic | b ARP
By the way, your platform is older than Gandalf.
Remi Astruc
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide