cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3816
Views
0
Helpful
6
Replies

3850 memory leak in iosd processor

Joseph Frost
Level 1
Level 1

We have deployed 35 Catalyst 3850 24 and 48 port switches as Layer 2 IDF closet switches, and we are monitoring with a standard tool.  We received alerts from two of the 3850-48p switches that memory went over 90%.  When I checked them with command "show proc memory" everything looked fine with 2.4GB available.  I did a little more searching and found our tool was actually checking the memory under "show mem det proc iosd":


<switch>#sh mem det process iosd
System memory : 3930840K total, 1508124K used, 2422716K free, 222004K kernel reserved
Lowest(b) : 1915232736

Process iosd, type L, PID = 8576
    338932K total, 72368K text, 34372K data, 104K stack, 288K heap


                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor   32A00000   536870912   483406000    53464912    52336864    52060720
IOS Proce   52A00008    16777216     8920180     7857036     7745736     6619672
<SNIP>

483406000 div 536870912 = .9004 or 90.04%


I checked the other switches and most had this memory allocation around or under 27%, only a couple of switch stacks were at 29%.  We decided to reboot the two switch and memory went back down under 27%.  However, just two weeks later, both of the switches are showing the memory is increasing, and one switch went back over 30% utilization:


<switch>#show mem det proce iosd               
System memory  : 3930840K total, 1420868K used, 2509972K free, 222004K kernel reserved
Lowest(b)      : 2008849460

Process iosd, type L, PID = 8569
    337904K total, 72368K text, 33348K data, 104K stack, 288K heap


                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor   32C00000   536870912   178395428   358475484   357196608   357332804
IOS Proce   52C00008    16777216     8882996     7894220     7813456     6693400
<SNIP>

178395428 div 536870912 = .3323 or 33.23%


I was thinking a complete power off and power on of the two switches in case it was some sort of transient error.  Is there anything else I should check for, or should I open a ticket with TAC?


Thank you.

6 Replies 6

Reza Sharifi
Hall of Fame
Hall of Fame

Hi,

Powering off the switches or rebooting them is not going to be a permanent solution as, the memory will leak again and the same thing will happen again. I recommend you open a ticket with TAC, as this maybe a bug in the IOS you are running.  Most likely they will recommend an IOS upgrade.

HTH

Yes, reach TAC.

3.6.5E is the code you should go with if you are running anything below.  In fact 3.6.4E fixed few memory related bugs and I am running that code for more than 12 months without major issues.

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/3e/release_notes/OL3262101.html#pgfId-918223

HTH

Rasika

Thanks guys.  Yeah, we upgrade a while back to version 03.03.05SE for a vulnerability.  Everything was fine until I found these two.

Joe,

This version (Version 03.07.02.E) is also pretty good. At the end of the day, its all about the feature set you are running but it strange to see 2 switches out of 35 with issues.

HTH

Ben Skinner
Level 1
Level 1

What code are you running?

Running version 03.03.05SE

Sorry for the delay, got side tracked.

Review Cisco Networking products for a $25 gift card