Potential memory leakage on router connected to internet

katerina.dardoufa · ‎09-20-2012

Hello all,

I have a an issue with a router connecting to the internet.

The model is:

System image file is "flash0:c3900e-universalk9-mz.SPA.151-4.M3.bin"

Cisco CISCO3925-CHASSIS (revision 1.0) with C3900-SPE200/K9 with 1015808K bytes of memory.

Processor board ID FCZ15522065

4 Gigabit Ethernet interfaces

1 Serial(sync/async) interface

DRAM configuration is 72 bits wide with parity enabled.

256K bytes of non-volatile configuration memory.

250880K bytes of ATA System CompactFlash 0 (Read/Write)

The processor memory seems to be decreasing day by day (nearly 14% free). We are running two BGP and one EIGRP process. We are learning a total of 845220 prefixes (half by one provider, half by the other).

Show processes memory sorted, gives the following result.

“sh processes memory sorted”

Processor Pool Total:  494129192 Used:  424390396 Free:   69738796

      I/O Pool Total:  300941312 Used:   44816208 Free:  256125104

 PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

 271   0  333215596 1441780872  301284260          0          0 BGP Router

   0   0  165799456   81622364   79120052          0          0 *Init*

 213   0   69403084       8884   69409720          0          0 CEF: IPv4 proces

Could this be the root of the problem? We have IP CEF enabled. Do we have to disable it and what would be the effect on the router?


Thank you in advance,

Katerina

Giuseppe Larosa · ‎09-20-2012

Hello Katerina,

disabling CEF has a big impact on router performance in the forwarding plane so you can expect an high rise in cpu usage if you disable CEF.

Besides this many features including QoS functions like marking and CAR are based on CEF.

With more then 400000 IPv4 prefixes to be installed in the CEF table, CEF memory usage is high.

Have you noticed an increase in CEF memory usage over time, as you suspect of a memory leakage?

A low value of memory freed by CEF process may be normal as most of the BGP routes are stable over time so it is holding memory.

To be noted BGP process uses much more memory then CEF as it handles all the advertisements received on the two eBGP sessions.

Also I notice that I/O memory pool is quite big:

> I/O Pool Total: 300941312 Used: 44816208 Free: 256125104

As you can note most of I/O Pool is empty.

Hve you configured the I/O memory size manually or it has been chosen by the router at bootup?

Also the total memory of the device should be 1GB, we don't see 200 MB in the show process memory sorted output. (the sum of processor memory and I/O memory is 800 MB roughly)

To be noted the platform may have not been tested extensively as Internet router with two full internet tables internally in Cisco.

I would not disable CEF on the router.

Hope to help

Giuseppe

katerina.dardoufa · ‎09-20-2012

Giuseppe thank you for your answer!

I have been monitoring the CEF process for the past two days. It seems stable, so I have to agree with you, that memory is not freed by CEF due to BGP routes not chaning over time.

My first thought was something to do with CEF, because the Cisco Output Interpreter pointed out that the memory that it held was very high and because it was at the top of the "show process memroy" list. But your explenation reassures me that this is not an issue.

We have not configured any I/O memory size manually.

"Also the total memory of the device should be 1GB, we don't see 200 MB in the show process memory sorted output. (the sum of processor memory and I/O memory is 800 MB roughly)"

This is true! Where are the remaining 200MB?!!!

We are already in touch with our provider who proposed this model for our environment, so hopefully we will resolve the issue soon.

Thanks again!

Katerina

katerina.dardoufa · ‎09-25-2012

Issue update,

our provider stated that since we have two BGP peerings and are getting full routing tables from both peers, this is the correct behaviour for the router. With this in mind, I will monitor the memory consumption, in relation to the sum of the prefixes we are learning. So if the memory decreases, I should have learnt more prefixes, when comparing them to the previous day...

This seems logical, but I will have to monitor this for a while, just to be on the safe side!