cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3636
Views
10
Helpful
10
Replies

CSCuz11169 - High memory utilization observed on catalyst 3650%2F3850

miller-p
Level 1
Level 1

Can anyone confirm the fixed release in 16.3.x?

The DDTS Bug notes state 16.3.1, but also say 16.3.1 is affected.

The release notes state 16.3.1 and 16.3.2 resolve this caveat.  However, I think I'm seeing this caveat in 16.3.3.

 

Thanks in advance.

10 Replies 10

Leo Laohoo
Hall of Fame
Hall of Fame
Unless there is an urgent need to, I'd avoid 16.X.

Agreed. However, I need to run 16.x to meet design requirements.


@miller-p wrote:
However, I need to run 16.x to meet design requirements.

Ok, then look for an IOS-XE version that doesn't end with .0, .1, or .2.  Look for something with a higher "suffix" number.  This means that the train is being "fixed" or updated.  

andrey_babkin
Level 1
Level 1

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 52 WS-C3650-48TD 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
2 28 WS-C3650-24TD 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
3 52 WS-C3650-48PD 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
4 52 WS-C3650-48PD 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL

 sho switch stack-bandwidth
                   Stack       Current
Switch#   Role     Bandwidth   State
------------------------------------------------------------
*1       Active    160G        Ready
 2       Standby   160G        Ready
 3       Member    160G        Ready
 4       Member    160G        Ready

 

show platform software status control-processor
1-RP0: online, statistics updated 8 seconds ago
Load Average: healthy
1-Min: 0.98, status: healthy, under 5.00
5-Min: 0.94, status: healthy, under 5.00
15-Min: 0.84, status: healthy, under 5.00
Memory (kb): healthy
Total: 3958932
Used: 3299124 (83%), status: healthy
Free: 659808 (17%)
Committed: 3894864 (98%), over 95%
Per-core Statistics
CPU0: CPU Utilization (percentage of time spent)
User: 8.39, System: 2.89, Nice: 0.00, Idle: 88.51
IRQ: 0.00, SIRQ: 0.19, IOwait: 0.00
CPU1: CPU Utilization (percentage of time spent)
User: 6.80, System: 2.70, Nice: 0.00, Idle: 90.50
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU2: CPU Utilization (percentage of time spent)
User: 7.60, System: 1.90, Nice: 0.00, Idle: 90.50
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU3: CPU Utilization (percentage of time spent)
User: 7.79, System: 2.09, Nice: 0.00, Idle: 90.10
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00

2-RP0: unknown, statistics updated 4 seconds ago
Load Average: healthy
1-Min: 0.33, status: healthy, under 5.00
5-Min: 0.43, status: healthy, under 5.00
15-Min: 0.46, status: healthy, under 5.00
Memory (kb): healthy
Total: 3958932
Used: 3186676 (80%), status: healthy
Free: 772256 (20%)
Committed: 3853292 (97%), over 95%
Per-core Statistics
CPU0: CPU Utilization (percentage of time spent)
User: 4.60, System: 2.30, Nice: 0.00, Idle: 92.99
IRQ: 0.00, SIRQ: 0.10, IOwait: 0.00
CPU1: CPU Utilization (percentage of time spent)
User: 6.10, System: 1.80, Nice: 0.00, Idle: 92.09
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU2: CPU Utilization (percentage of time spent)
User: 4.30, System: 1.70, Nice: 0.00, Idle: 94.00
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU3: CPU Utilization (percentage of time spent)
User: 5.29, System: 1.79, Nice: 0.00, Idle: 92.80
IRQ: 0.00, SIRQ: 0.09, IOwait: 0.00

3-RP0: unknown, statistics updated 6 seconds ago
Load Average: healthy
1-Min: 0.22, status: healthy, under 5.00
5-Min: 0.26, status: healthy, under 5.00
15-Min: 0.30, status: healthy, under 5.00
Memory (kb): healthy
Total: 3958932
Used: 1364940 (34%), status: healthy
Free: 2593992 (66%)
Committed: 1562800 (39%), under 95%
Per-core Statistics
CPU0: CPU Utilization (percentage of time spent)
User: 4.69, System: 1.39, Nice: 0.00, Idle: 93.80
IRQ: 0.00, SIRQ: 0.09, IOwait: 0.00
CPU1: CPU Utilization (percentage of time spent)
User: 3.69, System: 1.09, Nice: 0.00, Idle: 95.20
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU2: CPU Utilization (percentage of time spent)
User: 4.19, System: 1.19, Nice: 0.00, Idle: 94.60
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU3: CPU Utilization (percentage of time spent)
User: 4.40, System: 1.10, Nice: 0.00, Idle: 94.40
IRQ: 0.00, SIRQ: 0.10, IOwait: 0.00

4-RP0: unknown, statistics updated 1 seconds ago
Load Average: healthy
1-Min: 0.43, status: healthy, under 5.00
5-Min: 0.35, status: healthy, under 5.00
15-Min: 0.33, status: healthy, under 5.00
Memory (kb): healthy
Total: 3958932
Used: 1375756 (35%), status: healthy
Free: 2583176 (65%)
Committed: 1577320 (40%), under 95%
Per-core Statistics
CPU0: CPU Utilization (percentage of time spent)
User: 6.90, System: 0.90, Nice: 0.00, Idle: 92.20
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU1: CPU Utilization (percentage of time spent)
User: 3.30, System: 1.70, Nice: 0.00, Idle: 95.00
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU2: CPU Utilization (percentage of time spent)
User: 4.10, System: 2.50, Nice: 0.00, Idle: 93.40
IRQ: 0.00, SIRQ: 0.00, IOwait: 0.00
CPU3: CPU Utilization (percentage of time spent)
User: 2.99, System: 1.29, Nice: 0.00, Idle: 95.60
IRQ: 0.00, SIRQ: 0.09, IOwait: 0.00

Geez. That's still high.
However, give it a few days for the stack to "calm down".

robertomuro1
Level 1
Level 1

Looks like something triggers this,  I have 5 additional stacks running on this same version and memory looks a little bit high but reports as healthy.

 

 

 

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 56 WS-C3850-48T 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
2 56 WS-C3850-48T 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL
3 56 WS-C3850-48T 16.3.3 CAT3K_CAA-UNIVERSALK9 INSTALL

 

Mar 13 18:04:10 CDT: %PLATFORM-3-ELEMENT_CRITICAL:Switch 1 R0/0: smand: 3/RP/0: Used Memory value 98% exceeds critical level 95%
Mar 13 18:14:20 CDT: %PLATFORM-3-ELEMENT_CRITICAL:Switch 1 R0/0: smand: 3/RP/0: Used Memory value 98% exceeds critical level 95%
Mar 13 18:24:30 CDT: %PLATFORM-3-ELEMENT_CRITICAL:Switch 1 R0/0: smand: 3/RP/0: Used Memory value 98% exceeds critical level 95%

 

3-RP0: unknown, statistics updated 5 seconds ago
Load Average: healthy
1-Min: 0.42, status: healthy, under 5.00
5-Min: 0.42, status: healthy, under 5.00
15-Min: 0.43, status: healthy, under 5.00
Memory (kb): critical
Total: 3958928
Used: 3870652 (98%), status: critical
Free: 88276 (2%)
Committed: 4701140 (119%), over 99%
Per-core Statistics

scottlikecisco
Level 1
Level 1

We experienced this issue as well while running 16.3.5b code, we upgraded to 16.3.6 which is the current starred release code and so far the memory looks good. We are running stacks of 7-8 switches per stack.

We are experiencing this issue while running 16.3.6........

Anyone else having this issue?

%PLATFORM-3-ELEMENT_CRITICAL:Switch 1 R0/0: smand:  1/RP/0: Used Memory value 96% exceeds critical level 95%

After upgrading to version 16.3.7, this issue has been resolved! 

We're having the same High Memory on Control Processor issue with 16.5.3b. Have you experienced the problem on 16.3.6?