cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
634
Views
0
Helpful
1
Replies

High CPU, broadcasts, and L3 gleans

campbech1
Level 1
Level 1

We have a 7010 core with a WLC attached to it. We have been noticing high CPU and high broadcasts throughout the network for the past couple days. While running "ethanalyzer local interface mgmt limit-captured-frames 100" on the admin VDC, I'm seeing the 1.1.2.1 address continuously hitting the CPU. This is the virtual IP address for the WLC. I'm also seeing a very high level of L3 gleans.

Anyone know why I'd be seeing the VIP hitting the CPU and why the L3 gleans would be so high? I look to be averaging about 100 L3 gleans per second.

2015-02-25 21:58:50.220459      1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51600 [ACK] Seq=1 Ack=1 Win=87 Len=0 TSval=15893329 TSecr=171981758
2015-02-25 21:58:50.220585      1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51586 [ACK] Seq=1 Ack=1 Win=87 Len=0 TSval=15893329 TSecr=171975329
2015-02-25 21:58:50.220602      1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51585 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893329 TSecr=171975308
2015-02-25 21:58:50.220708      1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51584 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893329 TSecr=171975286
2015-02-25 21:58:50.220836      1.1.2.1 -> 10.254.97.48 TCP 66 snpp > 51579 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893329 TSecr=171971703
2015-02-25 21:58:50.224466      1.1.2.1 -> 10.254.166.206 TCP 66 snpp > 49006 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893330 TSecr=10003475
2015-02-25 21:58:50.228705      1.1.2.1 -> 10.254.201.27 TCP 60 snpp > 47895 [RST] Seq=1 Win=0 Len=0
2015-02-25 21:58:50.229713      1.1.2.1 -> 10.254.201.27 TCP 60 snpp > 42253 [RST] Seq=1 Win=0 Len=0
2015-02-25 21:58:50.233845      1.1.2.1 -> 10.254.201.27 TCP 60 snpp > 43352 [RST] Seq=1 Win=0 Len=0
2015-02-25 21:58:50.235340 10.134.252.2 -> 224.0.0.2    HSRP 62 Hello (state Active)
2015-02-25 21:58:50.236202 10.134.252.3 -> 224.0.0.2    HSRP 62 Hello (state Standby)
2015-02-25 21:58:50.242831      1.1.2.1 -> 10.254.209.236 TCP 66 snpp > 58677 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893335 TSecr=143055633
2015-02-25 21:58:50.252582      1.1.2.1 -> 10.254.224.210 TCP 66 snpp > 51082 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15962133 TSecr=74711841
2015-02-25 21:58:50.264445      1.1.2.1 -> 10.254.21.88 TCP 66 snpp > 41789 [ACK] Seq=1 Ack=1 Win=103 Len=0 TSval=15893340 TSecr=23110136

 

**** High L3 Gleans ****

bmh-dc-csw-1-corevdc# show hardware rate-limiter module 1

Units for Config: packets per second
Allowed, Dropped & Total: aggregated since last clear counters
rl-1: STP and Fabricpath-ISIS
rl-2: L3-ISIS and OTV-ISIS
rl-3: UDLD, LACP, CDP and LLDP
rl-4: Q-in-Q and ARP request
rl-5: IGMP, NTP, DHCP-Snoop, Port-Security, Mgmt and Copy traffic

Module: 1

Rate-limiter PG Multiplier: 1.00

  R-L Class           Config           Allowed         Dropped            Total
 +------------------+--------+---------------+---------------+-----------------+
  L3 mtu                   500               0               0                 0
  L3 ttl                   500       223050505          119193         223169698
  L3 control             10000               0               0                 0
  L3 glean                 100      5240691570      5396697578       10637389148
  L3 mcast dirconn        3000        16242363               0          16242363
  L3 mcast loc-grp        3000               0               0                 0
  L3 mcast rpf-leak        500               0               0                 0
  L2 storm-ctrl       Disable
  access-list-log          100               6               0                 6
  copy                   30000     10485178997               0       10485178997
  receive                30000      3596301609          203297        3596504906
  L2 port-sec              500               0               0                 0
  L2 mcast-snoop         10000      1120442375               0        1120442375
  L2 vpc-low              4000               0               0                 0
  L2 l2pt                  500               0               0                 0
  L2 vpc-peer-gw          5000               0               0                 0
  L2 lisp-map-cache       5000               0               0                 0
  L2 dpss                  100               0               0                 0
  L3 glean-fast            100               0               0                 0
  L2 otv                   100               0               0                 0
  L2 netflow               500               0               0                 0

 

1 Reply 1

Hi,

what are the other ip addresses, 10.254.97.48, 10.254.201.27 etc.

 

Probably you should consider capturing packets on inbound rather mgmt to see what is coming to cpu.

 

Please post the output of  

 

sh processes cpu sort | ex 0.00%

sh processes cpu history

 

 

Thanks,

Madhu

 

 

 

,

Review Cisco Networking products for a $25 gift card