cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
17704
Views
55
Helpful
28
Replies

WLC 9800 CPU Utilize Issue

bakaholic39
Level 1
Level 1

Hi,

I have a WLC 9800 on Firmware OS : 17.9.3
19 APs and 100 Clients 

Everything looks normal, but there are reports that the CPU utilization rate is too high. The problem is only 7th core. Has anyone encountered the same problem as me. What should I do to solve it?

Thank you!

cisco_wlc3.jpgcisco_wlc2.jpgcisco_wlc1.jpg

28 Replies 28

Leo Laohoo
Hall of Fame
Hall of Fame

Please share the complete output to the following commands: 

sh platform resources
sh platform software status con brief

Here are the results.

IMG_5471.jpeg

IMG_5472.jpeg

IMG_5473.jpeg

 Thank you for your response.

NOTE:  The two commands are a "real-time snapshot".   It is not historical. 

The first one looks at the data plane.  The value are normal. 

The second one looks at the control plane.  And the memory utilization of 30% is not normal.   

One of the CPU, #7, is grinding.  

Please post the complete output to the next command:  sh processes cpu platform sorted location 0

NOTE:  This output will produce a very long "page".  I just want to see the "first" page.

 

With this command, all process seems very normal except #7 CPU
messageImage_1689143856861.jpg

After that I tried this command. There seems to be something wrong with process " ucode_pkt_PPE0 "
My question is why doesn't the system share load with other cores.

messageImage_1689084075700.jpg

Thank you for your response.

Raise a TAC Case with EUMA or NAM/LATAM.  I am out of answers to what this process does.  

High ucode_pkt_PPE0 is "normal" - I seem to recall there's a recent change to how it's displayed (I think CSCve73211 mentioned below) but you can see mention of it here https://blogs.cisco.com/networking/wireless-catalyst-9800-wlc-kpis-part-1

As to the high CPU on that core - look at your wncd's: show process cpu platform sorted | incl wncd

If you confirm wncd causing it then look at why. 
19 APs and 100 clients is very small so that's surprising but check through best practise guide on AP tagging for distribution across wncd's.
Also, if you're using web-auth - turn OFF https redirects - they are very CPU hungry and will cause high CPU and client issues.

jagan.chowdam
Spotlight
Spotlight

Check CPU utilization: Identify if there is any process using lot of CPU using the following command
sh processes cpu platform sorted | ex 0% 0% 0%

Check CPU utilization per wncd process
show processes cpu platform sorted | i wncd

Check Memory utilization: Identify if device is running out of memory and if there is some process holding lot of memory.
show platform resources (basic)
show process memory platform sorted
show processes memory platform accounting (advanced)


CJ

/**Please rate all useful responses**/

 

 

Here are the results

IMG_5477.jpeg

IMG_5476.jpeg

IMG_5479.jpeg


Is there any problem with process ucode_pkt_PPE0?

Thank you for your response.

jagan.chowdam
Spotlight
Spotlight

For CPU monitoring we need to run following command multiple times. See if there are any processes showing CPU above 80% consistently and not just a spike.

sh processes cpu platform sorted | ex 0% 0% 0%

Catalyst 9800-CL and 9800-L platforms use CPU cores for data forwarding. Therefore, it is expected to see high CPU in ucode_pkt_PPE0. For those platforms to evaluate data plane performance use command: 

show platform hardware chassis active qfp datapath utilization | i Load

 

Refer the following Cisco 9800 KPI blog for detailed explanation as which commands to run.

https://blogs.cisco.com/networking/wireless-catalyst-9800-wlc-kpis-part-1

 

CJ

 

calverthall
Level 1
Level 1

Hello,  Did you get any clarity on this issue?  We just upgraded to 17.9.3 and noticed the exact same thing.  I just opened a TAC, but figured I would ask.

Thanks, Austin

Still no solution. I already opened TAC but he replied me that it's normal state without giving any further information.

Thanks.  I received the "it is expected to see high CPU in  one of the cores, basically this is a normal behavior and it could be on any core, you always will see one of the cores on high utilization" response as well.

I am not 100% convinced I would be getting alerts like this during the summer (we are a school that is out of session) and that this was normal prior to the upgrade.  My other thought would be why have 10 cores if only one runs at 99.7%...

Austin