cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4085
Views
0
Helpful
4
Replies

Crypto IKMP High CPU Load

shkerimov
Level 1
Level 1

Hello!

On C3825 with IOS 15.1(4)M5, I faced with issue:

I see high cpu load by Crypto IKMP proccess:

CPU utilization for five seconds: 92%/11%; one minute: 87%; five minutes: 86%

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

295    96696680      625524     154590 76.42% 70.87% 71.46%   0 Crypto IKMP

  65     7719140      608642      12682  1.08%  1.32%  1.33%   0 Per-Second Jobs

   6      838308       93487       8967  1.00%  0.16%  0.12%   0 Check heaps

139     1534084     6029064        254  0.75%  0.69%  0.62%   0 IP Input

But we haven`t a lot of traffic ~10 Mbit/s.

We have 1 GRE tunnel to branch, and crypto map for remote clients and some ASA`s

I didn`t find any information how I can tshoot this situation.

How I can deal with this? Why CPU load so high?

Router1#show crypto engine accelerator statistic

Device:   Onboard VPN

Location: Onboard: 0

        :Statistics for encryption device since the last clear

         of counters 1325 seconds ago

                 370618 packets in                      370618 packets out

               78673429 bytes in                      77224979 bytes out

                    279 paks/sec in                        279 paks/sec out

                    474 Kbits/sec in                       466 Kbits/sec out

                 171515 packets decrypted               199103 packets encrypted

               24579216 bytes before decrypt          52645763 bytes encrypted

               15116445 bytes decrypted               63556984 bytes after encrypt

                      0 packets decompressed                 0 packets compressed

                      0 bytes before decomp                  0 bytes before comp

                      0 bytes after decomp                   0 bytes after comp

                      0 packets bypass decompr               0 packets bypass compres

                      0 bytes bypass decompres               0 bytes bypass compressi

                      0 packets not decompress               0 packets not compressed

                      0 bytes not decompressed               0 bytes not compressed

                  1.0:1 compression ratio                1.0:1 overall

                Last 5 minutes:

                  69175 packets in                       69175 packets out

                    230 paks/sec in                        230 paks/sec out

                 380964 bits/sec in                     376846 bits/sec out

                2652816 bytes decrypted                9459770 bytes encrypted

                  71697 Kbits/sec decrypted             255669 Kbits/sec encrypted

                  1.0:1 compression ratio                1.0:1 overall

Errors:

                      0 pkts dropped                         0 ppq full

                      0 tx parts overflow                    0 rx parts overflow

                      0 replenishment failure                0 zero len

                      0 flow inputs bad                      0 cmd invalid

                      0 IPV4 len                             0 IPV6 len

                      0 algor invalid

                      0 bad shadow particle                  0 algor disabled

                      0 pre tx fail                          0 dma error

                      0 dbit miss                            0 pipeline abort

                      0 failsafe timeout                     0 reserv

                      0 bad sz count                         0 bad shdw

                      0 bad flow tx                          0 spi mismatch

                      0 bad flow rx                          0 auth fail

                      0 udm fs fail                          0 pad fail

                      0 addr limit fixup fail                0 seq fail

                      0 quad fix sp                          0 quad fix mp

                      0 quad fix cont

Thanks!

4 Replies 4

Marcin Latosiewicz
Cisco Employee
Cisco Employee

Typically due to incoming or outgoing negotation requests.

I suggest opening a TAC case to move faster.

show crypto isa sa

show crypto isa stats (hidden)

That's the minimum you should provide, I'd also suggest clearing the isa stats and taking them a few seconds apart while CPU is high.

#show crypto isa sa
IPv4 Crypto ISAKMP SA
dst             src             state          conn-id status
109.x.x.x   79.105.x.x  QM_IDLE           1196 ACTIVE L2L
109.x.x.x   77.41.x.x    QM_IDLE           1157 ACTIVE VPNClient
109.x.x.x   85.88.x.x    QM_IDLE           1162 ACTIVE L2L
109.x.x.x   2.61.x.x      QM_IDLE           1179 ACTIVE L2L
109.x.x.x   178.212.x.x QM_IDLE           1178 ACTIVE L2L
109.x.x.x   178.210.x.x  QM_IDLE           1191 ACTIVE VPNClient
109.x.x.x   178.210.x.x  QM_IDLE           1186 ACTIVE VPNClient
109.x.x.x   178.210.x.x  QM_IDLE           1182 ACTIVE VPNClient
109.x.x.x   178.210.x.x  QM_IDLE           1180 ACTIVE VPNClient
109.x.x.x   178.210.x.x  QM_IDLE           1177 ACTIVE VPNClient
109.x.x.x   194.186.x.x   QM_IDLE           1187 ACTIVE L2L
109.x.x.x   62.32.x.x    QM_IDLE           1146 ACTIVE L2L

IPv6 Crypto ISAKMP SA

Router1#show crypto isa stats
ISAKMP Process Packet Stats
---------------------------
IKE Received Packets.......276181
IKE Transmit Packets.......278526
IKE Int Q Depth [0]........0
IKE Int Q Peak [0].........0
IKE Int Q Depth [1]........0
IKE Int Q Peak [1].........0
IKE Int Q Depth [2]........0
IKE Int Q Peak [2].........1
IKE Int Q Depth [3]........0
IKE Int Q Peak [3].........12
IKE Int Q Depth [4]........0
IKE Int Q Peak [4].........0
IKE IPC Q Depth............0
IKE IPC Q Peak.............1
IKE P1 Retransmitted.......13
IKE P2 Retransmitted.......143
IKE P1 Rcvd Retransmit.....12
IKE P2 Rcvd Retransmit.....11
IKE Dup Retransmit.........0
Pak too long in queue......0
Packets too long in queues
dropped by IKE Dispatcher..0
NAT Keepalives Received....16
IKE call reenqueue         0
IPSec Node Dead Reasons:
   (errored reason mark with *)
   No reason                               ...0
  *By Error                                ...0
   By User Command                         ...0
   By Expired Lifetime                     ...0
   No Error                                ...197
   Informational (in) state 1              ...273268
   Informational (in) state 2              ...0
   Done with xauth request/reply exchange  ...117
   Transaction mode done                   ...81
   Saved QM no longer needed               ...2
   IKMP_NO_ERR_NO_TRANS                    ...0
   P2 Re-tx timer expired (CONF_ADDR)      ...0
   Config mode cleanup                     ...0
   QM done                                 ...0
   QM done (commit)                        ...0
   QM done (await)                         ...944
   IKE deleted                             ...22
  *Delete Larval                           ...1
  *Phase 2 err count exceeded              ...11
  *Decrypt_payload failed                  ...0
  *Invalid payload                         ...0
  *No IV for Transaction                   ...0
  *DELAYED_QM_TIMER expired                ...0
  *QM no hash                              ...0
  *QM bad hash                             ...0
  *QM not authenticated                    ...0
  *QM rejected                             ...16
  *QM not accepted                         ...0
  *No ke payload                           ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *Invalid reason code                     ...0

IKE SA Dead Reason:
   No reason                                    ...111
  *By error                                     ...9
   BY user command                              ...132
   BY expired lifetime                          ...0
   No error                                     ...0
   Delete no delete                             ...0
   P1 delete notify (in)                        ...0
   VRF removed from profile                     ...0
   Death by tree-walk                           ...0
   End of ipsec tunnel                          ...0
   IKE SA Lifetime Exceeded                     ...0
  *Receive initial contact                      ...0
  *P1 errcounter exceeded (PEERS_ALIVE_TIMER)   ...18
  *Needed xauth                                 ...18
  *XAUTH fail                                   ...0
  *Client cancel xauth prompt                   ...0
  *XAUTH not complete 1                         ...8
  *XAUTH not complete 2                         ...0
  *Fail to allocate ip address                  ...0
  *Failed to allocate a connection id           ...0
  *Phase1 SA policy proposal not accepted       ...52
  *Recevied fatal informational                 ...0
  *SA err counter exceeded (info)               ...0
  *Death by retransmission P1                   ...0
  *Death by retransmission P2                   ...0
  *Death by retransmission throw                ...0
  *Encrypt failure                              ...0
  *Delete_me flag/throw                         ...0
  *IKMP_ERR_NO_RETRANS                          ...0
  *gen_ipsec_isakmp_delete but doi isakmp       ...0
  *QM_TIMER expired                             ...0
  *IKE Fragmentation Failure                    ...0
  *                                             ...0
  *Invalid reason code                          ...0

After clear stats

Router1#show crypto isa stats
ISAKMP Process Packet Stats
---------------------------
IKE Received Packets.......61
IKE Transmit Packets.......60
IKE Int Q Depth [0]........0
IKE Int Q Peak [0].........0
IKE Int Q Depth [1]........0
IKE Int Q Peak [1].........0
IKE Int Q Depth [2]........0
IKE Int Q Peak [2].........0
IKE Int Q Depth [3]........0
IKE Int Q Peak [3].........1
IKE Int Q Depth [4]........0
IKE Int Q Peak [4].........0
IKE IPC Q Depth............0
IKE IPC Q Peak.............0
IKE P1 Retransmitted.......0
IKE P2 Retransmitted.......0
IKE P1 Rcvd Retransmit.....0
IKE P2 Rcvd Retransmit.....0
IKE Dup Retransmit.........0
Pak too long in queue......0
Packets too long in queues
dropped by IKE Dispatcher..0
NAT Keepalives Received....0
IKE call reenqueue         0
IPSec Node Dead Reasons:
   (errored reason mark with *)
   No reason                               ...0
  *By Error                                ...0
   By User Command                         ...0
   By Expired Lifetime                     ...0
   No Error                                ...0
   Informational (in) state 1              ...61
   Informational (in) state 2              ...0
   Done with xauth request/reply exchange  ...0
   Transaction mode done                   ...0
   Saved QM no longer needed               ...0
   IKMP_NO_ERR_NO_TRANS                    ...0
   P2 Re-tx timer expired (CONF_ADDR)      ...0
   Config mode cleanup                     ...0
   QM done                                 ...0
   QM done (commit)                        ...0
   QM done (await)                         ...0
   IKE deleted                             ...0
  *Delete Larval                           ...0
  *Phase 2 err count exceeded              ...0
  *Decrypt_payload failed                  ...0
  *Invalid payload                         ...0
  *No IV for Transaction                   ...0
  *DELAYED_QM_TIMER expired                ...0
  *QM no hash                              ...0
  *QM bad hash                             ...0
  *QM not authenticated                    ...0
  *QM rejected                             ...0
  *QM not accepted                         ...0
  *No ke payload                           ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *                                        ...0
  *Invalid reason code                     ...0

IKE SA Dead Reason:
   No reason                                    ...0
  *By error                                     ...0
   BY user command                              ...0
   BY expired lifetime                          ...0
   No error                                     ...0
   Delete no delete                             ...0
   P1 delete notify (in)                        ...0
   VRF removed from profile                     ...0
   Death by tree-walk                           ...0
   End of ipsec tunnel                          ...0
   IKE SA Lifetime Exceeded                     ...0
  *Receive initial contact                      ...0
  *P1 errcounter exceeded (PEERS_ALIVE_TIMER)   ...0
  *Needed xauth                                 ...0
  *XAUTH fail                                   ...0
  *Client cancel xauth prompt                   ...0
  *XAUTH not complete 1                         ...0
  *XAUTH not complete 2                         ...0
  *Fail to allocate ip address                  ...0
  *Failed to allocate a connection id           ...0
  *Phase1 SA policy proposal not accepted       ...0
  *Recevied fatal informational                 ...0
  *SA err counter exceeded (info)               ...0
  *Death by retransmission P1                   ...0
  *Death by retransmission P2                   ...0
  *Death by retransmission throw                ...0
  *Encrypt failure                              ...0
  *Delete_me flag/throw                         ...0
  *IKMP_ERR_NO_RETRANS                          ...0
  *gen_ipsec_isakmp_delete but doi isakmp       ...0
  *QM_TIMER expired                             ...0
  *IKE Fragmentation Failure                    ...0
  *                                             ...0
  *Invalid reason code                          ...0

First of all you have debugs on ... probably not the best idea ever :-)