02-25-2015 07:10 PM - edited 03-07-2019 10:51 PM
We have a 7010 core with a WLC attached to it. We have been noticing high CPU and high broadcasts throughout the network for the past couple days. While running "ethanalyzer local interface mgmt limit-captured-frames 100" on the admin VDC, I'm seeing the 1.1.2.1 address continuously hitting the CPU. This is the virtual IP address for the WLC. I'm also seeing a very high level of L3 gleans.
Anyone know why I'd be seeing the VIP hitting the CPU and why the L3 gleans would be so high? I look to be averaging about 100 L3 gleans per second.
2015-02-25 21:58:50.220459 1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51600 [ACK] Seq=1 Ack=1 Win=87 Len=0 TSval=15893329 TSecr=171981758
2015-02-25 21:58:50.220585 1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51586 [ACK] Seq=1 Ack=1 Win=87 Len=0 TSval=15893329 TSecr=171975329
2015-02-25 21:58:50.220602 1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51585 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893329 TSecr=171975308
2015-02-25 21:58:50.220708 1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51584 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893329 TSecr=171975286
2015-02-25 21:58:50.220836 1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51579 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893329 TSecr=171971703
2015-02-25 21:58:50.224466 1.1.2.1 -> 10.254.166.206 TCP 66 snpp > 49006 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893330 TSecr=10003475
2015-02-25 21:58:50.228705 1.1.2.1 -> 10.254.201.27 TCP 60 snpp > 47895 [RST] Seq=1 Win=0 Len=0
2015-02-25 21:58:50.229713 1.1.2.1 -> 10.254.201.27 TCP 60 snpp > 42253 [RST] Seq=1 Win=0 Len=0
2015-02-25 21:58:50.233845 1.1.2.1 -> 10.254.201.27 TCP 60 snpp > 43352 [RST] Seq=1 Win=0 Len=0
2015-02-25 21:58:50.235340 10.134.252.2 -> 224.0.0.2 HSRP 62 Hello (state Active)
2015-02-25 21:58:50.236202 10.134.252.3 -> 224.0.0.2 HSRP 62 Hello (state Standby)
2015-02-25 21:58:50.242831 1.1.2.1 -> 10.254.209.236 TCP 66 snpp > 58677 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893335 TSecr=143055633
2015-02-25 21:58:50.252582 1.1.2.1 -> 10.254.224.210 TCP 66 snpp > 51082 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15962133 TSecr=74711841
2015-02-25 21:58:50.264445 1.1.2.1 -> 10.254.21.88 TCP 66 snpp > 41789 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893340 TSecr=23110136
**** High L3 Gleans ****
bmh-dc-csw-1-corevdc# show hardware rate-limiter module 1
Units for Config: packets per second
Allowed, Dropped & Total: aggregated since last clear counters
rl-1: STP and Fabricpath-ISIS
rl-2: L3-ISIS and OTV-ISIS
rl-3: UDLD, LACP, CDP and LLDP
rl-4: Q-in-Q and ARP request
rl-5: IGMP, NTP, DHCP-Snoop, Port-Security, Mgmt and Copy traffic
Module: 1
Rate-limiter PG Multiplier: 1.00
R-L Class Config Allowed Dropped Total
+------------------+--------+---------------+---------------+-----------------+
L3 mtu 500 0 0 0
L3 ttl 500 223050505 119193 223169698
L3 control 10000 0 0 0
L3 glean 100 5240691570 5396697578 10637389148
L3 mcast dirconn 3000 16242363 0 16242363
L3 mcast loc-grp 3000 0 0 0
L3 mcast rpf-leak 500 0 0 0
L2 storm-ctrl Disable
access-list-log 100 6 0 6
copy 30000 10485178997 0 10485178997
receive 30000 3596301609 203297 3596504906
L2 port-sec 500 0 0 0
L2 mcast-snoop 10000 1120442375 0 1120442375
L2 vpc-low 4000 0 0 0
L2 l2pt 500 0 0 0
L2 vpc-peer-gw 5000 0 0 0
L2 lisp-map-cache 5000 0 0 0
L2 dpss 100 0 0 0
L3 glean-fast 100 0 0 0
L2 otv 100 0 0 0
L2 netflow 500 0 0 0
03-12-2015 02:36 AM
Hi,
what are the other ip addresses, 10.254.97.48, 10.254.201.27 etc.
Probably you should consider capturing packets on inbound rather mgmt to see what is coming to cpu.
Please post the output of
sh processes cpu sort | ex 0.00%
sh processes cpu history
Thanks,
Madhu
,
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide