Buy or Renew
Buy or Renew
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
1006
Views
0
Helpful
5
Replies

High CPU utilization and freezing on CSR1000v

Go to solution
Menon
Level 1
Level 1

‎02-05-2022 08:46 AM

Hi there,

 

There was an issue with our network with CSR1000v. When checking the CPU, it was showing high utilization, the HSRP state kept changing, and for a few minutes, the BGP was not working. Clearly, something is wrong - it automatically resolves itself. Unfortunately, we are out of reason - So what happened there we don't know - We'd appreciate any information you can offer as to what happened so we can avoid the same issue in the future.

 

 

logg : 

*Feb 5 15:24:35.831: %HSRP-5-STATECHANGE: GigabitEthernet9.520 Grp 20 state Speak -> Standby
*Feb 5 15:24:35.831: %HSRP-5-STATECHANGE: GigabitEthernet9.521 Grp 21 state Speak -> Standby
*Feb 5 15:24:35.832: %HSRP-5-STATECHANGE: GigabitEthernet9.524 Grp 24 state Speak -> Standby
*Feb 5 15:24:35.832: %HSRP-5-STATECHANGE: GigabitEthernet9.522 Grp 22 state Speak -> Standby
*Feb 5 15:24:35.832: %HSRP-5-STATECHANGE: GigabitEthernet9.530 Grp 30 state Speak -> Standby
*Feb 5 15:24:35.840: %HSRP-5-STATECHANGE: GigabitEthernet3 Grp 55 state Speak -> Standby
*Feb 5 15:24:35.841: %HSRP-5-STATECHANGE: GigabitEthernet7 Grp 18 state Speak -> Standby
*Feb 5 15:24:35.841: %HSRP-5-STATECHANGE: GigabitEthernet9.529 Grp 29 state Speak -> Standby
*Feb 5 15:24:35.843: %HSRP-5-STATECHANGE: GigabitEthernet9.528 Grp 28 state Speak -> Standby
*Feb 5 15:24:35.843: %HSRP-5-STATECHANGE: GigabitEthernet9.523 Grp 23 state Speak -> Standby
*Feb 5 15:24:36.001: %HSRP-5-STATECHANGE: GigabitEthernet6 Grp 17 state Speak -> Standby
*Feb 5 15:24:36.002: %HSRP-5-STATECHANGE: GigabitEthernet4 Grp 51 state Standby -> Active
*Feb 5 15:24:36.291: %HSRP-5-STATECHANGE: GigabitEthernet8 Grp 19 state Speak -> Standby
*Feb 5 15:24:36.291: %HSRP-5-STATECHANGE: GigabitEthernet9.527 Grp 27 state Speak -> Standby
*Feb 5 15:24:36.467: %HSRP-5-STATECHANGE: GigabitEthernet4 Grp 51 state Active -> Speak
*Feb 5 15:24:35.785: %IOSXE-5-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:003 TS:0000395799868 %PUNT_INJECT-5-DROP_PUNT_CAUSE: punt policer drops packets, cause: arp (0x7) from GigabitEthernet9.524 src mac: XXXX.XXXX.XXXX
*Feb 5 15:24:37.182: %HSRP-5-STATECHANGE: GigabitEthernet4 Grp 51 state Speak -> Standby
*Feb 5 15:26:45.811: %IOSXE-5-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:001 TS:00003957705012679527 %PUNT_INJECT-5-DROP_PUNT_CAUSE: punt policer drops packets, cause: arp (0x7) from GigabitEthernet9.528 src mac: XXXX.XXXX.XXXX
*Feb 5 15:27:30.608: %FMANFP-6-IPACCESSLOGP: R0/0: fman_fp_image: list WAN_IN denied tcp 75.34.62.67(52278) -> 75.34.62.67(443), 1 packet
*Feb 5 15:29:40.811: %FMANFP-6-IPACCESSLOGP: R0/0: fman_fp_image: list WAN_IN denied tcp 75.34.62.67(42062) -> 75.34.62.67(443), 1 packet
*Feb 5 15:31:02.407: %HSRP-5-STATECHANGE: GigabitEthernet4 Grp 51 state Standby -> Active
*Feb 5 15:31:02.415: %HSRP-5-STATECHANGE: GigabitEthernet9.523 Grp 23 state Standby -> Active
*Feb 5 15:31:02.422: %HSRP-5-STATECHANGE: GigabitEthernet5 Grp 12 state Standby -> Active
*Feb 5 15:31:02.433: %HSRP-5-STATECHANGE: GigabitEthernet9.530 Grp 30 state Standby -> Active
*Feb 5 15:31:02.440: %HSRP-5-STATECHANGE: GigabitEthernet7 Grp 18 state Standby -> Active
*Feb 5 15:31:02.450: %HSRP-5-STATECHANGE: GigabitEthernet9.520 Grp 20 state Standby -> Active
*Feb 5 15:31:02.459: %HSRP-5-STATECHANGE: GigabitEthernet9.529 Grp 29 state Standby -> Active
*Feb 5 15:31:02.467: %HSRP-5-STATECHANGE: GigabitEthernet3 Grp 55 state Standby -> Active
*Feb 5 15:31:02.475: %HSRP-5-STATECHANGE: GigabitEthernet9.522 Grp 22 state Standby -> Active
*Feb 5 15:31:02.500: %HSRP-5-STATECHANGE: GigabitEthernet9.525 Grp 25 state Standby -> Active
*Feb 5 15:31:02.509: %HSRP-5-STATECHANGE: GigabitEthernet9.527 Grp 27 state Standby -> Active
*Feb 5 15:31:02.516: %HSRP-5-STATECHANGE: GigabitEthernet9.524 Grp 24 state Standby -> Active
*Feb 5 15:31:02.524: %HSRP-5-STATECHANGE: GigabitEthernet8 Grp 19 state Standby -> Active
*Feb 5 15:31:02.529: %HSRP-5-STATECHANGE: GigabitEthernet9.528 Grp 28 state Standby -> Active
*Feb 5 15:27:30.608: %FMANFP-6-IPACCESSLOGP: R0/0: fman_fp_image: list WAN_IN denied tcp 75.34.62.67(52278) -> 75.34.62.67(443), 1 packet
*Feb 5 15:29:40.811: %FMANFP-6-IPACCESSLOGP: R0/0: fman_fp_image: list WAN_IN denied tcp 75.34.62.67(42062) -> 75.34.62.67(443), 1 packet
*Feb 5 15:31:02.407: %HSRP-5-STATECHANGE: GigabitEthernet4 Grp 51 state Standby -> Active
*Feb 5 15:31:02.415: %HSRP-5-STATECHANGE: GigabitEthernet9.523 Grp 23 state Standby -> Active
*Feb 5 15:31:02.422: %HSRP-5-STATECHANGE: GigabitEthernet5 Grp 12 state Standby -> Active
*Feb 5 15:31:02.433: %HSRP-5-STATECHANGE: GigabitEthernet9.530 Grp 30 state Standby -> Active
*Feb 5 15:31:02.440: %HSRP-5-STATECHANGE: GigabitEthernet7 Grp 18 state Standby -> Active
*Feb 5 15:31:02.450: %HSRP-5-STATECHANGE: GigabitEthernet9.520 Grp 20 state Standby -> Active
*Feb 5 15:31:02.459: %HSRP-5-STATECHANGE: GigabitEthernet9.529 Grp 29 state Standby -> Active
*Feb 5 15:31:02.467: %HSRP-5-STATECHANGE: GigabitEthernet3 Grp 55 state Standby -> Active
*Feb 5 15:31:02.475: %HSRP-5-STATECHANGE: GigabitEthernet9.522 Grp 22 state Standby -> Active
*Feb 5 15:31:02.500: %HSRP-5-STATECHANGE: GigabitEthernet9.525 Grp 25 state Standby -> Active
*Feb 5 15:31:02.509: %HSRP-5-STATECHANGE: GigabitEthernet9.527 Grp 27 state Standby -> Active
*Feb 5 15:31:02.516: %HSRP-5-STATECHANGE: GigabitEthernet9.524 Grp 24 state Standby -> Active
*Feb 5 15:31:02.524: %HSRP-5-STATECHANGE: GigabitEthernet8 Grp 19 state Standby -> Active
*Feb 5 15:31:02.529: %HSRP-5-STATECHANGE: GigabitEthernet9.528 Grp 28 state Standby -> Active

 

 

Thanks.

Labels:
0 Helpful
1 Accepted Solution

Accepted Solutions

Go to solution
Leo Laohoo
Hall of Fame
Hall of Fame

‎02-05-2022 04:59 PM

Contact Cisco TAC and if they can verify if this is CSCvj62552.

View solution in original post

0 Helpful
5 Replies 5

Go to solution
Georg Pauwen
VIP
VIP

‎02-05-2022 12:15 PM

Hello,

 

I am not sure if the CRS1000v supports the below command, but if it does, post the output of:

 

show platform hardware qfp active infrastructure punt statistics type per-cause

0 Helpful

Go to solution
Menon
Level 1
Level 1
In response to Georg Pauwen

‎02-07-2022 06:35 AM

Hi Georg,

 

Thank you for your valuable replay, please find the O/P of the given command:

 

R2#show platform hardware qfp active infrastructure punt statistics type per-cause
Global Per Cause Statistics

  Number of punt causes =   148 

  Per Punt Cause Statistics
                                                        Packets               Packets
  Counter ID  Punt Cause Name                           Received              Transmitted           
  ------------------------------------------------------------------------------------------------
  003         Layer2 control and legacy                 39419283              39419283              
  007         ARP request or response                   2019270661            2016677579            
  008         Reverse ARP request or repsonse           1109                  1109                  
  010         Incomplete adjacency                      1                     1                     
  011         For-us data                               14598213              14598214              
  021         RP<->QFP keepalive                        2057403               2057403               
  024         Glean adjacency                           804640                804640                
  026         QFP ICMP generated packet                 5143497               5143497               
  029         RP handled ICMP                           28                    28                    
  030         RP injected For-us data                   10                    10                    
  045         BFD control                               52                    52                    
  055         For-us control                            321871257             321871264             
  060         IP subnet or broadcast packet             689858                689858                
  064         Service Engine packet                     516                   516                   
  144         PMIPv6 control packets                    2                     2                     


  Number of inject causes = 52

  Per Inject Cause Statistics
                                                        Packets               Packets
  Counter ID  Inject Cause Name                         Received              Transmitted           
  ------------------------------------------------------------------------------------------------
  002         QFP destination lookup                    12812250              12789353              
  005         QFP <->RP keepalive                       2057403               0                     
  007         QFP adjacency-id lookup                   321733871             321733873             
  009         QFP ICMP generated packet                 5143497               5143313               
  012         ARP request or response                   1373247               1373245

Thanks.

0 Helpful

Go to solution
Leo Laohoo
Hall of Fame
Hall of Fame

‎02-05-2022 04:59 PM

Contact Cisco TAC and if they can verify if this is CSCvj62552.

0 Helpful

Go to solution
Menon
Level 1
Level 1
In response to Leo Laohoo

‎02-07-2022 06:42 AM

Hi Leo,

 

Sure - We will look into that.

 

Thanks.

0 Helpful

Go to solution
MHM Cisco World
VIP
VIP

‎02-07-2022 03:08 PM

CSR1000v, 
if the this router get full internet BGP prefix then that time the router can not handle all prefix.
check this point and make input prefix filter or ask ISP to filter some prefix.
also there are many HSRP group exchange the message, you can config one HSRP group and make other follow it if it connect through same trunk.

0 Helpful
Customers Also Viewed These Support Documents