02-07-2012 06:34 AM - edited 03-07-2019 04:46 AM
We are experiencing with high CPU input due to ARP input between 20:30 and 22:30 every day
At this time we have a lot of backup operations. When I look the netflow report, I can't see anything anormal.
We are changing our backup server's NIC card from 1gig to 10Gig. The backup operation's traffic is high (approx 2Gbps level) but 6509 has to be handle this size of traffic.
We are using two 6509E in VSS mode
and our image version is s72033-adventerprisek9_wan-mz.122-33.SXJ.bin
What the problem cause may be?
20:00
show ip arp summary
--------------
2588 IP ARP entries, with 166 of them incomplete
show process cpu sorted
--------------
CPU utilization for five seconds: 11%/3%; one minute: 9%; five minutes: 10%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
3 124 130 953 1.67% 0.13% 0.02% 6 SSH Process
9 7713920 452047 17064 1.19% 0.47% 0.37% 0 Check heaps
306 45535792 118795990 383 0.79% 0.73% 0.73% 0 Earl NDE Task
327 18046456 124766366 144 0.71% 0.54% 0.50% 0 IP Input
558 9185532 35575478 258 0.71% 0.36% 0.35% 0 Port manager per
12 263195052 123093541 2138 0.47% 0.52% 0.55% 0 ARP Input
20:30
show ip arp summary
--------------
2587 IP ARP entries, with 165 of them incomplete
show process cpu sorted
--------------
CPU utilization for five seconds: 88%/29%; one minute: 82%; five minutes: 83%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
12 263956908 123363086 2139 42.39% 39.49% 40.28% 0 ARP Input
397 26557084 28759093 923 4.71% 4.33% 4.32% 0 XDR mcast
9 7721604 452477 17065 3.11% 0.52% 0.38% 0 Check heaps
306 45582404 118903810 383 2.79% 2.38% 2.00% 0 Earl NDE Task
3 168 130 1292 1.67% 0.13% 0.02% 6 SSH Process
21:00
show ip arp summary
--------------
2589 IP ARP entries, with 167 of them incomplete
show process cpu sorted
--------------
CPU utilization for five seconds: 81%/27%; one minute: 83%; five minutes: 79%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
12 265046164 123738558 2141 37.91% 39.18% 37.08% 0 ARP Input
397 26664908 28858941 923 5.43% 4.55% 4.26% 0 XDR mcast
306 45645772 119037173 383 3.11% 2.09% 1.95% 0 Earl NDE Task
3 176 130 1353 1.67% 0.13% 0.02% 6 SSH Process
327 18089292 124960978 144 1.11% 0.71% 0.70% 0 IP Input
21:30
show ip arp summary
--------------
2589 IP ARP entries, with 168 of them incomplete
--------------
show process cpu sorted
--------------
CPU utilization for five seconds: 98%/40%; one minute: 83%; five minutes: 79%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
12 266025804 124084184 2143 41.67% 37.30% 36.26% 0 ARP Input
397 26760700 28950445 924 4.07% 3.98% 3.92% 0 XDR mcast
24 18692320 127459398 146 1.75% 1.07% 0.83% 0 IPC Seat Manager
3 160 130 1230 1.67% 0.13% 0.02% 6 SSH Process
327 18110704 125057205 144 1.27% 0.78% 0.70% 0 IP Input
22:00
show ip arp summary
--------------
2586 IP ARP entries, with 164 of them incomplete
--------------
show process cpu sorted
--------------
CPU utilization for five seconds: 23%/9%; one minute: 65%; five minutes: 72%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
12 266948752 124452798 2144 7.27% 28.77% 32.71% 0 ARP Input
3 144 130 1107 1.67% 0.13% 0.02% 6 SSH Process
306 45782092 119315110 383 1.59% 2.04% 2.08% 0 Earl NDE Task
397 26863336 29062080 924 1.11% 3.87% 4.44% 0 XDR mcast
327 18132460 125159162 144 0.79% 0.66% 0.69% 0 IP Input
02-13-2012 12:01 AM
Hi,
We solved the problem.. The problem is faulty NIC card of backup server. We changed the backup server's NIC card, after then the CPU problem is gone.
Thanks for your hep Dan, it was very helpful to find problem source..
Regards..
02-13-2012 02:15 AM
Hi eek ,
I'm glad that your problem was solved , and also that I could help.
Also you can mark as answered your question.
Regards
Dan
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide