%SYS-2-MALLOCFAIL

[Juan Ibañez]

Porfavor ayudenme con este tema

Tengo un Router Cisco 2811, este lo utilizo como gateway de voz y tiene la siguiente IOS

c2800nm-spservicesk9-mz.124-3f.bin

Los sintomas son los siguientes:

Rechaza las sesiones telnet

No acepta llamadas entrantes y salientes

Cuando se establece una llamada no se escucha ni mo Rx ni TX.

este es el log que arroja:

Log Buffer (51200 bytes):

Aug 27 19:38:53.649: %SYS-2-MALLOCFAIL: Memory allocation of 780 bytes failed from 0x40327608, alignment 32 Pool: I/O  Free: 7504  Cause: Memory fragmentation Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "Pool Manager", ipl= 0, pid= 6

-Traceback= 0x411B2D30 0x400A76C0 0x400ABBE8 0x40327610 0x400C9668  0x400ABBE8 0x40327610 0x400C9668 0x400C97CC

La verdad que he buscado documentacion en Cisco sobre este problema pero todas me recomienda realizar un upgrade de IOS.

Adjunto Show tech del equipo para dar mayor informacion.

[Sandeep Choudhary]

Hi Juan,

I am sorry , I cant understand your language.

but still throught the eeror messages i can guess the error:

Please follow this link :

https://supportforums.cisco.com/docs/DOC-12225

May be it helps.

Regards

Please rate if it helps.

[singhaam007]

Hi juan,

its hard to understand it.

but i think its a memory leaking issue.

It looks like your router is leaking memory and you need to check process.

continously monitor the output of 'show process memory sorted' output and see what process is leaking the memory.

use Sh process cpu | ex 0.00

this will give u a clean out with only process which are using memory.

thanks

Please rate if it helps.

[Arumugam Muthaiah]

Hi Juan,

I would like to add my comments along with Sandeep and Amrinder, as they said this issue looks like memory leak on the Routner. I have gone thruogh the captures which shared by you and i decoded those tracebacks.

------------------ show memory statistics ------------------

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor   44E5A980   175789696    30020440   145769256   145022680   145002188
      I/O   3F600000    10485760     4123464     6362296     6280176     6318940

Symptom:

Free and lowest memory almost close and Limited avilable memory

Cause:

1. Memory allocated or locked by a process and not freed
   
2. Another possible cause is a network/traffic event. This will often manifest itself as excessive utilization in multiple pools
   
3. A specific pool is more heavily utilized because of certain traffic that is needed in a network environment.  This traffic may, for some reason, need to be process-switched and there is no way to avoid this at the current time
   

• Need to check which pool is taking memory leak either I/O or Processor. For our case it is I/O
  
• If I/O, need to check if there is buffer over utilization
  
• Some buffer range should be constantly holding many buffers
  

Log:

Aug 27 19:38:53.649: %SYS-2-MALLOCFAIL: Memory allocation of 780 bytes failed from 0x40327608, alignment 32 Pool: I/O  Free: 7504  Cause: Memory fragmentation Alternate Pool: None  Free: 0  Cause: No Alternate pool

Please collect the below capture multiple time with different time intervels,

Note: Reserve memory for console access in any case berfore capture these output,

memory reserve console xxxx

show process memory sorted

show memory statistics

show memory io free

show memory io allocating-process totals

show memory io dead totals

show buffers

show buffers old

show memory debug leaks summary – CPU intensive!!!

show memory summary

Please share the your issue and symptoms in english and it would you much helpful to investigate the issue further.

Regards,

Aru

[Rawatpankaj745@gmail.com]

 Hello @Arumugam Muthaiah ,

I am also facing an similar issue but in my case the pool: processor is present in the switch log .i.e.  

Jun 10 05:59:56.908 AST: %SYS-2-MALLOCFAIL: Memory allocation of 45808 bytes failed from 0x2FBB814, alignment 0 Pool: Processor Free: 165568 Cause: Memory fragmentation

This only switch is creating this issue which result in whole REP Network down. After factory resetting it is again going in normal mode. Request you to please tell us the root cause and a permanent solution of this problem. Attached is the full log of that crash switch . Please do needful and also kindly share your email Id for further contact.

Thanks,

With Regards,

Pankaj 

[Joseph W. Doherty]

Unfortunately, normal operations can eventually result in memory fragmentation causing memory allocation failures. (This is because Cisco IOS doesn't have a "garbage collector".)

What can be done to minimize/mitigate this? You might reload the device daily, you might add physical RAM to the device (if possible), and/or you might review IOS features being used and determine if any tend to use much RAM or "churn it". For the latter, reconfiguration of such features might reduce the needed memory footprint. (For example, if you have a RAM syslog active, reduce its size allocation.)

[Rawatpankaj745@gmail.com]

Thanks @joseph. But I have one doubt why its affecting other Switches in the REP network result in whole network down?

[Joseph W. Doherty]

That's harder to say, however, when you start having memory allocation errors, all kinds of "strange" things can start to happen because different functions, on that device, may no longer work correctly. Perhaps that device is doing something "strange" with REP that confuses the other devices. You noted, when you reset this device, the network resumes normal operation. That being the case, if you resolve the memory allocations error, you may also resolve your overall network problem too.

[marce1000]

- De hecho, debería consultar versiones de software posteriores para su dispositivo, si están disponibles. Tome el lanzamiento recomendado o con estrellas doradas.

 M.

[Rawatpankaj745@gmail.com]

Hello @marce1000 ,

Already We are using the latest software version file in all the switches present in the network.