cisco prime infrastructure 1.3 makes every 15 minutes cpu high on cisco 800 routers

[filipvanacker]

I have an issue with cisco prime infrastructure 1.3 and cisco 880 routers, every 15 minutes the cpu goes to 100% on those routers, but I cannot find the job on the prime which is causing this, if I look in the job dashboard I have some jobs like sam_poller_22_15_getvpnpoller_27977292 that runs every 15 minutes (other jobs that I see CDBPoller_dmvpnPoller and CDBPoller_getvpnpoller)   , probably is it on of these jobs, but how can I stop it (job type : poller job, owner =system) and what are they doing?

thanks for any assistance

[Vinod Arya]

It is usually SNMP polling which should be responsible for taking the CPU high. We usually dont need to stop/disable anything on NMS tools, but infact we can have devices troubleshooted to find which is causing high CPU.

Please collect following outputs when you see high CPU :

> show proc cpu sort | exclude 0.00

> show stacks <PID>

     ***PID should be the Process ID (PID) of process taking highest CPU.

Suspected is mostly SNMP, in case so, please share show run | incl snmp output, as well as show log.

-Thanks

Vinod

**Rating Encourages contributors, and its really free. **

[filipvanacker]

see here the asked output

838-R01#show proc cpu sort | exclude 0.00
CPU utilization for five seconds: 99%/0%; one minute: 16%; five minutes: 9%
PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
324    34590588      252832     136812 96.42% 10.68%  3.60%   0 SNMP ENGINE     
  85    10018308     2526859       3964  0.97%  2.18%  2.29%   0 COLLECT STAT COU
   5      791164       87818       9009  0.73%  0.17%  0.14%   0 Check heaps     
217         660         315       2095  0.48%  0.45%  0.15%   8 Virtual Exec    
109     1462056     4362400        335  0.24%  0.28%  0.29%   0 CDP Protocol    
246       96432      379298        254  0.08%  0.05%  0.07%   0 Crypto IKMP     
337       10504    15236838          0  0.08%  0.06%  0.07%   0 PPP Events      
135       35448      833582         42  0.08%  0.03%  0.02%   0 Spanning Tree   
  74      374096     4281239         87  0.08%  0.07%  0.07%   0 EEM ED ND       
121      322956     1131499        285  0.08%  0.13%  0.09%   0 IP Input        

838-R01#sh stacks 324
Process 324:  SNMP ENGINE
  Stack segment 0x88305E88 - 0x88308D68
  FP: 0x88307DF8, RA: 0x82CBCC00
  FP: 0x883082D8, RA: 0x82FA1164
  FP: 0x88308710, RA: 0x82FA15E8
  FP: 0x88308738, RA: 0x82FA17FC
  FP: 0x88308758, RA: 0x8005F7E0
  FP: 0x88308770, RA: 0x8005383C
  FP: 0x88308790, RA: 0x80053958
  FP: 0x88308800, RA: 0x80053A58
  FP: 0x88308AB0, RA: 0x8003CDA0
  FP: 0x88308AC8, RA: 0x8003805C
  FP: 0x88308B00, RA: 0x8006FDCC
  FP: 0x88308B28, RA: 0x8006B5E0
  FP: 0x88308B80, RA: 0x8006E57C
  FP: 0x88308BF8, RA: 0x82A03CBC
  FP: 0x88308C70, RA: 0x82A0A3B8
  FP: 0x88308CD8, RA: 0x829F6050
  FP: 0x88308D58, RA: 0x82A2A7C4
  FP: 0x88308D60, RA: 0x805F2404
  FP: 0x0, RA: 0x805D96BC

sh run | in snmp

snmp-server group MONITORING v3 priv access ACLSNMP_RO

snmp-server community ArgSnmp RO ACLSNMP_RO

snmp-server ifindex persist

snmp-server trap-source Loopback0 snmp-server group MONITORING v3 priv access ACLSNMP_RO
snmp-server community ArgSnmp RO ACLSNMP_RO
snmp-server ifindex persist
snmp-server trap-source Loopback0

the log is saying nothing usefull

the cpu is only 1 or 2 minutes for 100%

thanks for the assistance

[Vinod Arya]

One more detail that is essential is the output of show version from the device.

It would be required to decode the tracebacks, show stacks have produced.

-Thanks

[filipvanacker]

Cisco IOS Software, C880 Software (C880DATA-UNIVERSALK9-M), Version 15.3(2)T, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Thu 28-Mar-13 13:08 by prod_rel_team

ROM: System Bootstrap, Version 12.4(22r)YB5, RELEASE SOFTWARE (fc1)

Arg-gebouw-838-R01 uptime is 6 days, 16 hours, 54 minutes
System returned to ROM by power-on
System image file is "flash:c880data-universalk9-mz.153-2.T.bin"
Last reload type: Normal Reload
Last reload reason: power-on

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Cisco 881G (MPC8300) processor (revision 1.0) with 196608K/65536K bytes of memory.
Processor board ID FCZ1442C0GJ

5 FastEthernet interfaces
2 terminal lines
1 Virtual Private Network (VPN) Module
1 Cellular interface
256K bytes of non-volatile configuration memory.
126000K bytes of ATA CompactFlash (Read/Write)

License Info:

License UDI:

-------------------------------------------------
Device#   PID                   SN
-------------------------------------------------
*0        CISCO881G-K9          FCZ1442C0GJ    

License Information for 'c880-data'
    License Level: advipservices   Type: Permanent
    Next reboot license Level: advipservices

Configuration register is 0x2102

greetings

[Vinod Arya · Accepted solution]

This is what i expected. I know many platform like 29xx/39xx or ISR's has known issues with Cisco Prime Infrastructure when it polls CISCO-LICENSE-MGMT-MIB, devices go high on CPU.

You have to block clmgmtLicenseInfoEntry by creating a view on your 800 router by configuring snmp view.

Following is the command/example:

Step 1: Configure SNMP View:

snmp-server view iso included

snmp-server view   clmgmtLicenseInfoEntry excluded

Link community string to SNMP View:

snmp-server community view ro|rw

Example per your configuration:

snmp-server view PrimeInfra iso included

snmp-server view PrimeInfra clmgmtLicenseInfoEntry excluded

snmp-server community ArgSnmp view PrimeInfra RO ACLSNMP_RO

OR you can try following on your Cisco PI:

Manually update file below,

Open the /opt/CSCOlumos/conf/sam/template.xml, modify the template id=20  (licensePoller)

Update tag for value 600 to 86400 for all attrributes () of

template id =20

Old: 600

New: 86400

Some known bugs on such issues are :

CSCug97914 and CSCuc72024

Please check and let me know the updates.

-Thanks

Vinod

**Rating Encourages contributors, and its really free. **

[filipvanacker]

thanks, it is working

[Vinod Arya]

Glad to know I was able to help!