07-11-2023 02:57 AM
Hi,
I have a WLC 9800 on Firmware OS : 17.9.3
19 APs and 100 Clients
Everything looks normal, but there are reports that the CPU utilization rate is too high. The problem is only 7th core. Has anyone encountered the same problem as me. What should I do to solve it?
Thank you!
07-11-2023 03:29 AM
Please share the complete output to the following commands:
sh platform resources
sh platform software status con brief
07-11-2023 06:51 AM
Here are the results.
Thank you for your response.
07-11-2023 05:53 PM
NOTE: The two commands are a "real-time snapshot". It is not historical.
The first one looks at the data plane. The value are normal.
The second one looks at the control plane. And the memory utilization of 30% is not normal.
One of the CPU, #7, is grinding.
Please post the complete output to the next command: sh processes cpu platform sorted location 0
NOTE: This output will produce a very long "page". I just want to see the "first" page.
07-11-2023 11:55 PM
With this command, all process seems very normal except #7 CPU
After that I tried this command. There seems to be something wrong with process " ucode_pkt_PPE0 "
My question is why doesn't the system share load with other cores.
Thank you for your response.
07-12-2023 12:43 AM
Raise a TAC Case with EUMA or NAM/LATAM. I am out of answers to what this process does.
07-29-2023 10:06 AM
High ucode_pkt_PPE0 is "normal" - I seem to recall there's a recent change to how it's displayed (I think CSCve73211 mentioned below) but you can see mention of it here https://blogs.cisco.com/networking/wireless-catalyst-9800-wlc-kpis-part-1
As to the high CPU on that core - look at your wncd's: show process cpu platform sorted | incl wncd
If you confirm wncd causing it then look at why.
19 APs and 100 clients is very small so that's surprising but check through best practise guide on AP tagging for distribution across wncd's.
Also, if you're using web-auth - turn OFF https redirects - they are very CPU hungry and will cause high CPU and client issues.
07-11-2023 06:40 AM
Check CPU utilization: Identify if there is any process using lot of CPU using the following command
sh processes cpu platform sorted | ex 0% 0% 0%
Check CPU utilization per wncd process
show processes cpu platform sorted | i wncd
Check Memory utilization: Identify if device is running out of memory and if there is some process holding lot of memory.
show platform resources (basic)
show process memory platform sorted
show processes memory platform accounting (advanced)
CJ
/**Please rate all useful responses**/
07-11-2023 07:05 AM - edited 07-11-2023 07:09 AM
Here are the results
Is there any problem with process ucode_pkt_PPE0?
Thank you for your response.
07-11-2023 06:54 AM
For CPU monitoring we need to run following command multiple times. See if there are any processes showing CPU above 80% consistently and not just a spike.
sh processes cpu platform sorted | ex 0% 0% 0%
Catalyst 9800-CL and 9800-L platforms use CPU cores for data forwarding. Therefore, it is expected to see high CPU in ucode_pkt_PPE0. For those platforms to evaluate data plane performance use command:
show platform hardware chassis active qfp datapath utilization | i Load
Refer the following Cisco 9800 KPI blog for detailed explanation as which commands to run.
https://blogs.cisco.com/networking/wireless-catalyst-9800-wlc-kpis-part-1
CJ
07-17-2023 08:37 AM
Hello, Did you get any clarity on this issue? We just upgraded to 17.9.3 and noticed the exact same thing. I just opened a TAC, but figured I would ask.
Thanks, Austin
07-17-2023 09:00 AM
Still no solution. I already opened TAC but he replied me that it's normal state without giving any further information.
07-17-2023 10:09 AM
Thanks. I received the "it is expected to see high CPU in one of the cores, basically this is a normal behavior and it could be on any core, you always will see one of the cores on high utilization" response as well.
I am not 100% convinced I would be getting alerts like this during the summer (we are a school that is out of session) and that this was normal prior to the upgrade. My other thought would be why have 10 cores if only one runs at 99.7%...
Austin