Buffer Misses

[tylerlucas]

One of my main routers is getting a bunch of buffer misses, resulting in packet drops.  I have pasted output below:

Genoa-C3845-Rtr#  sh buff

Buffer elements:

     1117 in free list (1119 max allowed)

     445357378 hits, 0 misses, 619 created

Public buffer pools:

Small buffers, 104 bytes (total 176, permanent 70, peak 230 @ 7w0d):

     150 in free list (20 min, 150 max allowed)

     596539015 hits, 350633 misses, 147852 trims, 147958 created

     124863 failures (0 no memory)

Middle buffers, 600 bytes (total 35, permanent 35, peak 167 @ 7w0d):

     31 in free list (10 min, 150 max allowed)

     1286575676 hits, 32303 misses, 9766 trims, 9766 created

     22400 failures (0 no memory)

Big buffers, 1536 bytes (total 50, permanent 50, peak 95 @ 7w0d):

     49 in free list (5 min, 150 max allowed)

     88445460 hits, 8034 misses, 910 trims, 910 created

     7430 failures (0 no memory)

VeryBig buffers, 4520 bytes (total 10, permanent 10, peak 16 @ 7w0d):

     9 in free list (0 min, 100 max allowed)

     1163206 hits, 6472 misses, 97 trims, 97 created

     6471 failures (0 no memory)

Large buffers, 5024 bytes (total 1, permanent 0, peak 7 @ 7w0d):

     1 in free list (0 min, 10 max allowed)

     106 hits, 6365 misses, 3210 trims, 3211 created

     6365 failures (0 no memory)

Huge buffers, 18024 bytes (total 1, permanent 0, peak 8 @ 7w0d):

     1 in free list (0 min, 4 max allowed)

     193969 hits, 6355 misses, 3081 trims, 3082 created

     6268 failures (0 no memory)

This was happening months ago, but it would be just a couple small misses a day.  I raised the small buffer pool as necessary, but it didn't seem to help much - maybe I wasn't raising it enough. 

Anyway, I checked again today, and there are tons of misses from all pool, and a lot of drops.

Also, here's a SMTP graph showing all of the misses:

http://grab.by/6Ogg

Any thoughts? Obviously I need more in the small and middle pools... Should I be raising the min, max, and permanent? I guess I'm a little reserved because I'm afraid of reserving too much memory...

Thanks

[Jason Masker]

This document may help:

http://www.cisco.com/en/US/customer/products/hw/routers/ps133/products_tech_note09186a00800a7b80.shtml

(PDF version: http://www.cisco.com/en/US/customer/products/hw/routers/ps133/products_tech_note09186a00800a7b80.shtml)

You can use the show memory commands to ensure you are not exhausting memory on the router.

[Jason Masker]

One additional thought.. if the buffer misses are a fairly new phenomenon, you may want to spend more time tracking down the cause with captures and/or other monitoring tools. It could be there is a new source of particularly bursty traffic on your network and there may be some settings on the application end which could alleviate this.

[tylerlucas]

The average throughput on this link is quite high.... I'm not sure how I could watch traffic accurately and look for something so specific... Is there a good way that I'm not thinking of?