WLC 9800 CPU Utilize Issue

[bakaholic39]

Hi,

I have a WLC 9800 on Firmware OS : 17.9.3
19 APs and 100 Clients 

Everything looks normal, but there are reports that the CPU utilization rate is too high. The problem is only 7th core. Has anyone encountered the same problem as me. What should I do to solve it?

Thank you!

[bakaholic39]

It's not only in the UI, if you get resources utilization from the CLI you will see the CPU usage is above threshold.
If the engineer said this is a normal, then just please remove this RED Alert from the UI. The dashboard should not display any RED Alerts if the system is functioning properly. Because it's a matter of a monthly report.

[Leo Laohoo]

@bakaholic39, @bakaholic39, thanks for the Bug ID. 

At 115 cases (since 06 June 2017) picture below sums it all up.  

[edmonroy]

Catalyst 9800-CL and 9800-L platforms use CPU cores for data forwarding. Therefore, it is expected to see high CPU in ucode_pkt_PPE0. For those platforms to evaluate data plane performance use command: “show platform hardware chassis active qfp datapath utilization | i Load”

https://blogs.cisco.com/networking/wireless-catalyst-9800-wlc-kpis-part-1

[Leo Laohoo]

---

@edmonroy wrote:
For those platforms to evaluate data plane performance use command: “show platform hardware chassis active qfp datapath utilization | i Load”

---

Thanks for this info, @edmonroy.

Question:  What is the difference between the command you recommended against "sh platform resources"? 

I have compared the values between the two on four 9800-80 (two standalone and two in VSS) and I do not see any difference.  

[Rich R]

@edmonroy let's ignore ucode_pkt_PPE0 which we all now agree is "normal".

Let's concentrate on the CPU core which is at 100%.  Are you suggesting that is also normal? 
When we had that it WAS definitely customer affecting (caused silent packet drops) - we had customer complaints and reproduced the problems ourselves. Problems resolved once CPU was restored to normal ~30% by turning off https redirects.

[SebastianHL]

We just hit that issue, too. We do not have HTTPS redirect enabled (or I am too blind to find the correct config flag 🙂 ).

I also want to point out that in the blog the CPU cores show no more than 70 % usage in the five minute interval. At least in our case, the CPU (also 7 for us) shows 100% at any time (and any interval).

[Rich R]

Regarding https redirect:
parameter-map type webauth global
no intercept-https-enable
secure-webauth-disable
webauth-http-enable

If that doesn't help open a TAC case for them to identify the reason for the high CPU. 
Have you checked your config with the config analyser (below) and gone through the Best Practice guide (also below)?

[Lukas Svoboda]

We run two C9800-L-F in HA pair, IOS XE 17.9.4. I confirm the exact same issue (core 7 permanently 100% on both controllers), red High CPU Utilization alert on GUI dashboard.

[charliemcrae]

I have a C9800-L running 17.9.4a and have observed this issue too.

[Leo Laohoo]

It is "normal" until TAC comes up with a Bug ID, like CSCwh75934.

[Rich R]

As Leo says this is normal for the packet forwarding (data plane) CPU core on 9800-L and 9800-CL - also documented in:
https://blogs.cisco.com/tag/wireless-catalyst-9800-wlc-kpis-series
If it's something else then you need to troubleshoot to reduce the CPU and open a TAC case if you need help.

[Leo Laohoo]

Another Bug ID has appeared:  CSCwh10989

[Scott Fella]

Ignore it... that is what I have done on my home lab 9800-L.  It has gone away after I upgraded to 17.12.1 

[Booker]

If you are using 9800-CL or 9800-L then it is ok because it lacks hardware chip that the 7th core is emulating the process all the time. Could be ignored due to this document.

Understand High CPU Usage Reported for the Dataplane on Catalyst 9800 - Cisco

[Leo Laohoo]

I wish Cisco technical staff can be more "consistent":  If Cisco says that it is normal for the ucode_pkt_PPE0 process to have high CPU then what does Cisco say when there is a memory leak attributed to ucode_pkt_PPE0 process (CSCwj61168:  IOS-XE Router experiences memory leak in ucode_pkt_PPE0 process), a feature?