cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1600
Views
25
Helpful
9
Replies

Do "memory leaks" only exist in individual processes?

Mic_Jameson
Level 1
Level 1

Hi.

In a WS-C3850-24XS-E switch, I'm remotely troubleshooting "%PLATFORM-4-ELEMENT_WARNING: Switch 2 R0/0: smand: 2/RP/0: Used Memory value 92% exceeds warning level 90%"

 

1.  Do "memory leaks" only exist in individual processes, or can they somehow exist throughout all processes?

2. If the "show process memory sorted" reveals a process to be exhibiting abundant "Freed" memory after abundant "allocated memory", Does that rule out a memory leak for that process?

 

After the on-site tech gave me the "show tech" log on the unhealthy switch, I requested a second log on a same model second switch under similar workload, so I could have a minimum baseline. The results show basically ALL processes on the unhealthy switch to be 4x or more memory hungry.

(As a note, there exist no memory bugs associated with the IOS release.)

 

I've attached both the unhealthy switch, and healthy switch "show process memory sorted" output. (Unfortunately the different IOS' output the data with different column headers. So they cant be compared completely directly)

 

If you can suggest a good way to troubleshoot this, I'd appreciate it!

 

Thank you so much!

Mic Jameson

CCNP Enterprise

3 Accepted Solutions

Accepted Solutions

Joseph W. Doherty
Hall of Fame
Hall of Fame

1.  Do "memory leaks" only exist in individual processes, or can they somehow exist throughout all processes?

A "true" memory leak is where some software acquires memory dynamically, for some purpose, and when the purpose is fulfilled, does not release any, or some, of the acquired memory.  Although some software runs as a "process", which often "owns" acquired memory, an OS might run software as other "things" that can also "own" memory or use CPU.  I.e. one might be unable to say only memory owned by a "process" could have a memory leak, but unlikely all "processes" would have memory leaks (as memory leaks, normally are also software bugs).

2. If the "show process memory sorted" reveals a process to be exhibiting abundant "Freed" memory after abundant "allocated memory", Does that rule out a memory leak for that process?

No, but it doesn't exclude it either.

". . . same model second switch under similar workload, so I could have a minimum baseline."

Sorry, the doesn't really guarantee second switch is representative of what's happening in first switch.

Unless you're very knowledgeable about OS and device drivers type software, it can be difficult to fully appreciate the "interesting issues" that can arise when dealing with real-time events.  I.e. if process A runs even a millisecond before process B, all's good, but the converse has a bug.  Such can be a real problem to identify and debug, especially when process A, normally, occurs 99.999% of the time before process B, and further, the bug's impact is very, very tiny (like not releasing a single byte of memory it should).

Oh, and even for a situation like you describe, there might not be a "memory leak" per se caused by a bug.  IOS, I believe, does not "garbage collect" its dynamic memory.  Without it, it's possible memory gets fragmented, and the excessive memory usage is due to control block memory usage (tracking all the allocated and free memory).

View solution in original post


@Joseph W. Doherty wrote:

"show process memory sorted"


Just want to add:  This command is really helpful for platforms running on classic IOS.  For any platforms running on IOS-XE, like the 3850, this command is useless.  

A single 3850 has four CPUs and 4Gb of memory.

This command (and the command "sh process cpu") will only provide a "snapshot" of the CPU/memory health of the  entire stack like an "average" value.  This command does not provide a snapshot of the memory level of each switch members of the stack.  

View solution in original post


@MicJameson wrote:

Why is it bad to post screenshots?


1.  Because people who post screenshots usually edit out the "good bits". 

2.  Have you seen some of the screenshots posted?  There are some which are barely readable. 

View solution in original post

9 Replies 9

Leo Laohoo
Hall of Fame
Hall of Fame

Post the complete output to the following commands: 

  1. sh version
  2. sh platform resources
  3. sh platform software status con brief

NOTE:  Please do not post screenshots.

Thank you for your reply Leo.

I have no access to the switch until Monday, and only through a remote tech, and my boss will want an update.

Is there anything I should examine with the "show-tech" logs i have?

 


@MicJameson wrote:

Is there anything I should examine with the "show-tech" logs i have?


Too early to examine "sh tech".


@MicJameson wrote:

and my boss will want an update.


I work with what I've got.  The three commands I have requested is just the "start".  The output from those three commands will point me to the direction of the next set of commands.  If the output have to wait `til Monday, then the next update will be sometime on Monday.  

Alternatively, raise a TAC Case.

IMPORTANT:  When raising the TAC Case, make sure to queue the ticket to America/Latin America (RTP) or Europe/Middle East desk.  

Thank you Leo. I appreciate your responses.

Hi Leo.

 

It is true that I am not familiar with the rules and culture of these message-boards.

Why is it bad to post screenshots?


@MicJameson wrote:

Why is it bad to post screenshots?


1.  Because people who post screenshots usually edit out the "good bits". 

2.  Have you seen some of the screenshots posted?  There are some which are barely readable. 

Joseph W. Doherty
Hall of Fame
Hall of Fame

1.  Do "memory leaks" only exist in individual processes, or can they somehow exist throughout all processes?

A "true" memory leak is where some software acquires memory dynamically, for some purpose, and when the purpose is fulfilled, does not release any, or some, of the acquired memory.  Although some software runs as a "process", which often "owns" acquired memory, an OS might run software as other "things" that can also "own" memory or use CPU.  I.e. one might be unable to say only memory owned by a "process" could have a memory leak, but unlikely all "processes" would have memory leaks (as memory leaks, normally are also software bugs).

2. If the "show process memory sorted" reveals a process to be exhibiting abundant "Freed" memory after abundant "allocated memory", Does that rule out a memory leak for that process?

No, but it doesn't exclude it either.

". . . same model second switch under similar workload, so I could have a minimum baseline."

Sorry, the doesn't really guarantee second switch is representative of what's happening in first switch.

Unless you're very knowledgeable about OS and device drivers type software, it can be difficult to fully appreciate the "interesting issues" that can arise when dealing with real-time events.  I.e. if process A runs even a millisecond before process B, all's good, but the converse has a bug.  Such can be a real problem to identify and debug, especially when process A, normally, occurs 99.999% of the time before process B, and further, the bug's impact is very, very tiny (like not releasing a single byte of memory it should).

Oh, and even for a situation like you describe, there might not be a "memory leak" per se caused by a bug.  IOS, I believe, does not "garbage collect" its dynamic memory.  Without it, it's possible memory gets fragmented, and the excessive memory usage is due to control block memory usage (tracking all the allocated and free memory).


@Joseph W. Doherty wrote:

"show process memory sorted"


Just want to add:  This command is really helpful for platforms running on classic IOS.  For any platforms running on IOS-XE, like the 3850, this command is useless.  

A single 3850 has four CPUs and 4Gb of memory.

This command (and the command "sh process cpu") will only provide a "snapshot" of the CPU/memory health of the  entire stack like an "average" value.  This command does not provide a snapshot of the memory level of each switch members of the stack.  

Thank you sir for your deep response. I definitely appreciate it. You have helped someone today indeed.

Review Cisco Networking for a $25 gift card