cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4145
Views
0
Helpful
6
Replies

6513 : Memory allocation failed

83881463a
Level 1
Level 1

Hello all,

 

I am facing this issue in one of our 6513 switches:

 

Nov  2 16:00:23: %SYS-2-MALLOCFAIL: Memory allocation of 308 bytes failed from 0x407958B4, alignment 32
Pool: I/O  Free: 37184  Cause: Memory fragmentation
Alternate Pool: None  Free: 0  Cause: No Alternate pool
 -Process= "Port manager periodic stats collection", ipl= 0, pid= 412
-Traceback= 423A66C0 423ABD30 407958BC 40795D30 4079934C 40794FD0 41000958 40F168C4 40F18DA8 40F17C58 40F1F6F0 42391138 42391124

 

The thing is that this happens when we configure authentication (with ISE) in all the ports. Deleting the configuration solves the problem, but we want the users to be authenticated.

 

The switch version is 12.2(33)SXJ10.

 

Any suggestion?

 

Thank you in advance.

 

 

6 Replies 6

Mark Malone
VIP Alumni
VIP Alumni
Usually malloc fails are mem leak bugs in software and an upgrade will fix it , but if your pushing the switch more than the memory could handle you need to check what the mem is like when its fully configured for ISE

This doc gives a few tips on malloc fails
https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/ios-151m-t/116467-technote-ios-00.html

Thank you Mark.

 

The person who manage the switch gave me the "sh tech" file and I think there is something related with the I/O Pool but I can not give a proper answer seeing this:

 

--------- show buffers --------

Buffer elements:
     500 in free list (500 max allowed)
     1981196 hits, 0 misses, 0 created

Public buffer pools:
Small buffers, 104 bytes (total 115204, permanent 1024, peak 115204 @ 00:00:01):
     0 in free list (128 min, 2048 max allowed)
     1611409 hits, 207672 misses, 38967 trims, 153147 created
     43032 failures (4423 no memory)
Medium buffers, 256 bytes (total 11989, permanent 3000, peak 11989 @ 00:00:01):
     0 in free list (64 min, 3000 max allowed)
     57670 hits, 32624 misses, 2390 trims, 11379 created
     21463 failures (4349 no memory)
Middle buffers, 600 bytes (total 6235, permanent 512, peak 6235 @ 00:00:01):
     0 in free list (64 min, 1024 max allowed)
     163916 hits, 19942 misses, 1273 trims, 6996 created
     12843 failures (5255 no memory)
Big buffers, 1536 bytes (total 4395, permanent 1000, peak 4395 @ 00:00:01):
     0 in free list (64 min, 1000 max allowed)
     56875 hits, 10876 misses, 1444 trims, 4839 created
     6112 failures (5237 no memory)
VeryBig buffers, 4520 bytes (total 57, permanent 10, peak 57 @ 00:00:01):
     0 in free list (0 min, 100 max allowed)
     1482 hits, 6033 misses, 0 trims, 47 created
     6033 failures (5217 no memory)
Large buffers, 5024 bytes (total 42, permanent 2, peak 42 @ 00:00:01):
     0 in free list (0 min, 10 max allowed)
     107 hits, 5975 misses, 7 trims, 47 created
     5975 failures (5217 no memory)
Huge buffers, 18024 bytes (total 36, permanent 2, peak 36 @ 00:00:01):
     0 in free list (0 min, 4 max allowed)
     425 hits, 5925 misses, 12 trims, 46 created
     5925 failures (5217 no memory)

 

My task is to implement ISE, and after a month I am facing this issue in this switch. And all the eyes are in ISE not in the switch.

I think, as you said, and I told them that the problem is about the memory that the switch can't handle.

 

Is there anything more specific that I must see in the switch?

 

Thank you in advance.

Leo Laohoo
Hall of Fame
Hall of Fame
Two ways to get rid of memory allocation errors:
Temporary Solution: Reboot the chassis; or
Permanent Solution: Upgrade the IOS

Thank you Leo!

 

As you said, the reboot of the chassis is a temporary solution.

 

What version of IOS do you recommend?

 

The supervisor used is Supervisor Engine 32 and the IOS version 122-33.SXJ10.

 

The only solution I can see is to upgrade to the version 12.2.33-SXI14. What do you think?

My recommendation is to read the Release Notes.  This way, one can make a solid decision as to the stability of the chosen version is valid or not.

Joseph W. Doherty
Hall of Fame
Hall of Fame
I just wanted to add, memory allocation errors are not always due to bugs, they also might be caused by your configuration and workload.

What you might want to review, is there anyway you might reduce stress on your device's memory. For example, do you have syslog to memory? If so, can you reduce the buffer allowance? Or, for instance, do you collect summary Netflow stats on the device? Do you run SLA tests, etc.?

What might also help, and already mentioned, is a device reload. However, rather than just a one time thing, this is something you might need to schedule on a reoccurring basis.

Also, if the device supports, adding RAM might eliminate the issue too.

NB: the issue with IOS, it doesn't have a garbage collector that will coalesce free RAM. So, it's possible fragmentation will preclude a MALLOC even though overall there's sufficient aggregate free RAM. By having sufficient free RAM, extreme fragmentation is less likely.
Review Cisco Networking for a $25 gift card