Memory Usage High

[Mark Bowyer]

We have got a few 3524XL and 2900XL switches and on a quite regular basis, the memory usage gets quite high, sometimes to the point that we cannot log onto them. Can anybody see anything strange from the sh proc mem output below and suggest any reasons why?

Total: 4026856, Used: 3701452, Free: 325404
PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process
   0   0      48732       1504     427000          0          0 *Init*
   0   0       1380     205600       1380          0          0 *Sched*
   0   0   11195648    4434952    1356788    4424948    3760648 *Dead*
   1   0        268        268       3836          0          0 Load Meter
   2   1       9628          0      17728       4536          0 Virtual Exec
   3   0          0       1888       6836          0          0 Check heaps
   4   0      20248          0      27084          0          0 Chunk Manager
   5   0     235924          0      12440     126252          0 Pool Manager
   6   0        268        268       6836          0          0 Timers
   7   0         96          0       6932          0          0 Entity MIB API
   8   0        232        136       7068          0          0 ARP Input
   9   0          0          0       3836          0          0 RAM Access (dm 0
  10   0         96          0       6932          0          0 Critical Bkgnd
  11   0    1206572    1368068      13196      20412     333600 Net Background
  12   0        364        268      12932          0          0 Logger
  13   0        268        268       6836          0          0 TTY Background
  14   0  554753076  508200536       9896       2340       5172 Per-Second Jobs
  15   0        192          0       7028          0          0 Net Input
  16   0        268        268       6836          0          0 Compute load avg
  17   0       1412     187280       6836        756     250496 Per-minute Jobs
  18   0 4282881304 3603063196       3836          0      34084 LED Control Proc
  19   0       8024    1642296       4056          0          0 Frank Aging
  20   0 1533749476 2796165632      19780     219996       2432 Port Status Proc
  21   0        244          0       7080          0          0 VM Prune Events
  22   0          0          0       6900          0          0 IP NAT Ager
  23   0  258122584  258122076       3908          0          0 GDS Frame Ager
  24   0          0          0       3836          0          0 RAM Access (gi0/
  28   0          0          0       3836          0          0 RAM Access (gi0/
  30   0  654452468 4001949108       9380       3900          0 Broadcast Storm
  31   0         96          0       3932          0          0 Port Group Chang
  33   0       1596   42300808       3932          0        304 Address Learning
  34   0          0          0       3896          0          0 RAM Access (dm 1
  35   0          0          0       3836          0          0 RAM Access (dm 2
  36   0 2751834512 3091321368      61576       2340          0 Enet Aging
  37   0      10524       8036      14176          0          0 IP Input
  38   0      43148   27252204       3932          0       3040 Address Deletion
  39   0    2373752    2370312      10276          0          0 CDP Protocol
  40   0      10260       5848       7412          0          0 Switch CGMP Prot
  41   0       8776       1940      13672          0          0 VLAN Manager
  42   0          0       1148      12836          0          0 TCP Timer
  43   0     130404          0      14440          0          0 TCP Protocols
  44   0          0          0       6836          0          0 Socket Timers
  45   0          0          0       6836          0          0 HTTP Timer
  46   0        244          0       4080          0          0 CGMP Forwarding
  47   0         96          0       3932          0          0 Cluster RARP
  48   0    1966776    1966292       7320          0          0 Cluster Base
  49   0     148852     265000       7660      79380          0 Spanning Tree
  50   0       3168        268       3988        780          0 STP Hello
  51   0     176540        268      21832      49920          0 STP Queue Handle
  52   0      97564     136436       7604      52164          0 Malibu STP Adjus
  53   0          0          0       6836          0          0 Time Range Proce
  54   0         96          0      12932          0          0 SNMP ConfCopyPro
  55   0         96          0       3932          0          0 Bridge MIB traps
  56   0          0          0       6868          0          0 SNMP Timers
  57   0  123551500  122142744    1414500       9352          0 IP SNMP
  58   0         96          0      12932          0          0 SNMP Traps
  59   0       1452        268       8020          0          0 NTP
                                  3700892 Total

[ROGER FERNANDO MAJO ZACARIAS]

Hello,

The problem associated to high use of memory is some type of attack due to high traffic coming into your switch interfaces.

We recommend to disconnect all user from the switches and left only the trunk conection.

Then reconnect one by one the devices until the problem appears so you can know who is generating the attack.

Check: show process cpu history to see if the process is above 70% or more.

Then with a packet analyzer you can reconnect the PC an analyze what type of traffic is generating.

Attentively.

[tech32]

Hmm. Let's sort this "show proc mem" by highest holding memory:

PID	TTY	Allocated	Freed	Holding	Getbufs	Retbufs	Process
57	0	123551500	122142744	1414500	9352	0	IP	SNMP
0	0	11195648	4434952	1356788	4424948	3760648	*Dead*
0	0	48732	1504	427000	0	0	*Init*
36	0	2751834512	3091321368	61576	2340	0	Enet	Aging

I'd agree. You DO have an attacker. Unfortunately, it appears to by your SNMP poller. Try stopping the polling and checking the memory after about 10 minutes. Does it slow down?