Cisco 3925 Always High Memory

johnlloyd_13 · ‎04-13-2015

hi all,

i've got a cisco 3925 which has only one site-to-site VPNs terminated.

we always encounter packet drops but checked bandwidth isn't congested.

there's no high CPU but observed memory is always high.

the router has 2GB memory installed.

i've checked another 3925 router on another site it's almost the same (around 90%).

i'd like to know if high memory is 'normal' for this kind of router?

what else i could check?

Cisco CISCO3925-CHASSIS (revision 1.0) with C3900-SPE100/K9 with 2039808K/57344K bytes of memory.

# sh mem stat his
------------------ History of Processor Mempool ------------------

PE01 06:56:59 AM Monday Apr 13 2015 UTC

      999999999999999999999999999999999999999999999999999999999999
      777777777777777777777777777777777777777777777777777777777777
100 ************************************************************
   90 ************************************************************
   80 ************************************************************
   70 ************************************************************
   60 ************************************************************
   50 ************************************************************
   40 ************************************************************
   30 ************************************************************
   20 ************************************************************
   10 ************************************************************
     0....5....1....1....2....2....3....3....4....4....5....5....6
               0    5    0    5    0    5    0    5    0    5    0
               Free memory per second (last 60 seconds)

      999999999999999999999999999999999999999999999999999999999999
      777777777777777777777777777777777777777777777777777777777777
100 ############################################################
   90 ############################################################
   80 ############################################################
   70 ############################################################
   60 ############################################################
   50 ############################################################
   40 ############################################################
   30 ############################################################
   20 ############################################################
   10 ############################################################
     0....5....1....1....2....2....3....3....4....4....5....5....6
               0    5    0    5    0    5    0    5    0    5    0
               Free memory per minute (last 60 minutes)
              * = maximum # = average

------------------ History of I/O Mempool ------------------

PE01 06:57:01 AM Monday Apr 13 2015 UTC

      666666666666666666666666666666666666666666666666666666666666
      777888777788887777777787666777788766677888777777887677677888
100
   90
   80
   70 ************************************************************
   60 ************************************************************
   50 ************************************************************
   40 ************************************************************
   30 ************************************************************
   20 ************************************************************
   10 ************************************************************
     0....5....1....1....2....2....3....3....4....4....5....5....6
               0    5    0    5    0    5    0    5    0    5    0
               Free memory per second (last 60 seconds)

      666666666666666666666666666666666666666666666666666666666666
      888888888888888888888888888888888888888888888888888888888888
100
   90
   80
   70 ############################################################
   60 ############################################################
   50 ############################################################
   40 ############################################################
   30 ############################################################
   20 ############################################################
   10 ############################################################
     0....5....1....1....2....2....3....3....4....4....5....5....6
               0    5    0    5    0    5    0    5    0    5    0
               Free memory per minute (last 60 minutes)
              * = maximum # = average

#sh proc cpu sort
CPU utilization for five seconds: 2%/1%; one minute: 2%; five minutes: 2%
PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
178     8859436   118926737         74 0.23% 0.43% 0.46%   0 IP Input
106      107496    32501709          3 0.15% 0.20% 0.21%   0 Netclock Backgro
87       28116     8169977          3 0.07% 0.08% 0.08%   0 Per-Second Jobs
346      823784    17556263         46 0.07% 0.04% 0.01%   0 IP SNMP
132        5920 1031411623          0 0.07% 0.08% 0.07%   0 Ethernet Msec Ti
122       90688    16250584          5 0.07% 0.03% 0.02%   0 BPSM stat Proces
348      875684     8777732         99 0.07% 0.03% 0.01%   0 SNMP ENGINE
   7      196192     6546813         29 0.00% 0.00% 0.00%   0 Pool Manager
   8           0           1          0 0.00% 0.00% 0.00%   0 DiscardQ Backgro
   6     4381272     1159882       3777 0.00% 0.07% 0.05%   0 Check heaps
11           0           1          0 0.00% 0.00% 0.00%   0 License Client N
   9           0           2          0 0.00% 0.00% 0.00%   0 Timers
10           0        1641          0 0.00% 0.00% 0.00%   0 WATCH_AFS
14           0           1          0 0.00% 0.00% 0.00%   0 OIR Handler
12           0           1          0 0.00% 0.00% 0.00%   0 Image License br
17        2952     1624763          1 0.00% 0.00% 0.00%   0 IPC Event Notifi
18           0      135398          0 0.00% 0.00% 0.00%   0 IPC Dynamic Cach
19           0           1          0 0.00% 0.00% 0.00%   0 IPC Session Serv
13     1625136      135398      12002 0.00% 0.01% 0.00%   0 Licensing Auto U
16     7592804     8117771        935 0.00% 0.11% 0.08%   0 Environmental mo
22         152     7934838          0 0.00% 0.00% 0.00%   0 IPC Deferred Por

<SNIP>

Leo Laohoo · ‎04-13-2015

John,

Post the output to the command "sh proc cpu sort | ex 0.00".

What is your bandwidth and what is the IOS?

It looks like a memory allocation/memory hog bug.

johnlloyd_13 · ‎04-13-2015

pre,

1 meg metro e.

was thinking this might be some cosmetic bug.

#sh proc cpu sort | ex 0.00
CPU utilization for five seconds: 1%/0%; one minute: 1%; five minutes: 2%
PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
178     8967300   120206599         74 0.55% 0.52% 0.49%   0 IP Input
106      107768    32606988          3 0.23% 0.20% 0.21%   0 Netclock Backgro
132        5924 1034753850          0 0.07% 0.07% 0.07%   0 Ethernet Msec Ti
87       28196     8196387          3 0.07% 0.07% 0.07%   0 Per-Second Jobs

#sh mem stat his
------------------ History of Processor Mempool ------------------

PE01 02:22:51 PM Monday Apr 13 2015 UTC

      999999999999999999999999999999999999999999999999999999999999
      777777777777777777777777777777777777777777777777777777777777
100 ************************************************************
   90 ************************************************************
   80 ************************************************************
   70 ************************************************************
   60 ************************************************************
   50 ************************************************************
   40 ************************************************************
   30 ************************************************************
   20 ************************************************************
   10 ************************************************************
     0....5....1....1....2....2....3....3....4....4....5....5....6
               0    5    0    5    0    5    0    5    0    5    0
               Free memory per second (last 60 seconds)

#sh clo
14:22:29.546 UTC Mon Apr 13 2015

#sh ve | i IOS
Cisco IOS Software, C3900 Software (C3900-UNIVERSALK9-M), Version 15.3(3)M3, RELEASE SOFTWARE (fc1)

Leo Laohoo · ‎04-13-2015

1 meg metro e.

Ngek, 1 Mbps and then 3925 router? That's an overkill.

It looks like a cosmetic bug alright.

johnlloyd_13 · ‎04-13-2015

i know and don't ask. it's our campus design policy :)