cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2016
Views
10
Helpful
11
Replies

C9404R Standby Sudden Reboot

Noah L
Level 1
Level 1

May I ask why the Standby on the C9404R rebooted suddenly. Please see logs below.

May 30 17:18:51 PST: %IOSXE_OIR-6-REMCARD: Card (cc) removed from chassis 2 slot 1
May 30 17:18:51 PST: %IOSXE_OIR-6-REMCARD: Card (cc) removed from chassis 2 slot 2
May 30 17:18:51 PST: %IOSXE_OIR-6-REMCARD: Card (cc) removed from chassis 2 slot 3
May 30 17:18:51 PST: %IOSXE_OIR-6-REMCARD: Card (cc) removed from chassis 2 slot 4
May 30 17:18:51 PST: %IOSXE_PEM-6-REMPEM_FM: PEM/FM chassis 2 slot P1 removed
May 30 17:18:51 PST: %IOSXE_PEM-6-REMPEM_FM: PEM/FM chassis 2 slot P2 removed
May 30 17:18:51 PST: %IOSXE_PEM-6-REMPEM_FM: PEM/FM chassis 2 slot P3 removed
May 30 17:18:51 PST: %IOSXE_PEM-6-REMPEM_FM: PEM/FM chassis 2 slot P4 removed
May 30 17:18:51 PST: %IOSXE_PEM-6-REMPEM_FM: PEM/FM chassis 2 slot P9 removed
May 30 17:18:51 PST: %IOSXE_OIR-6-REMSPA: SPA removed from chassis 2 subslot 1/0, interfaces disabled
May 30 17:18:51 PST: %IOSXE_OIR-6-REMSPA: SPA removed from chassis 2 subslot 2/0, interfaces disabled
May 30 17:18:51 PST: %IOSXE_OIR-6-REMSPA: SPA removed from chassis 2 subslot 4/0, interfaces disabled
May 30 17:18:51 PST: %REDUNDANCY-3-STANDBY_LOST: Standby processor fault (PEER_NOT_PRESENT)
May 30 17:18:51 PST: %REDUNDANCY-3-STANDBY_LOST: Standby processor fault (PEER_DOWN)
May 30 17:18:51 PST: %REDUNDANCY-3-STANDBY_LOST: Standby processor fault (PEER_REDUNDANCY_STATE_CHANGE)
May 30 17:18:51 PST: %SPA_OIR-6-OFFLINECARD: SPA (C9400-LC-48S) offline in chassis 2 subslot 1/0
May 30 17:18:51 PST: %SPA_OIR-6-OFFLINECARD: SPA (C9400-SUP-1XL) offline in chassis 2 subslot 2/0
May 30 17:18:51 PST: %STACKMGR-6-CHASSIS_REMOVED: Chassis 1 R0/0: stack_mgr: Chassis 2 has been removed from the stack.
May 30 17:18:52 PST: %SPA_OIR-6-OFFLINECARD: SPA (C9400-LC-48S) offline in chassis 2 subslot 4/0
May 30 17:18:52 PST: %RF-5-RF_RELOAD: Peer reload. Reason: EHSA standby down
May 30 17:18:52 PST: %IOSXE_REDUNDANCY-6-PEER_LOST: Active detected switch 2 is no longer standby

 

show version

Cisco IOS XE Software, Version 16.09.03
Cisco IOS Software [Fuji], Catalyst L3 Switch Software (CAT9K_IOSXE), Version 16.9.3, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2019 by Cisco Systems, Inc.

11 Replies 11

Hi

 Are this logs from the Primary switch? Can you show the logs from the Standby?

Hi Flavio,

Switches are in VSS. Yes this is from the Primary switch.

Alright, they are in VSS.  But still you have two switches, right?

"Active detected switch 2 is no longer standby"

 

What I meant is, did you plugged a console cable on the second box to see if any log could give a clue about what happened?

 

 

Let me check on this.

Wont the both switches output the same logs since they are in VSS?

If we had a split brain scenario, the switch must store the logs somewhere.

marce1000
VIP
VIP

 

 - FYI : https://bst.cloudapps.cisco.com/bugsearch?pf=prdNm&kw=IOSXE_OIR-6-REMCARD&bt=custV&sb=anfr

 Apparently the full bug report is not accessible for the general public , but if you hoover over the link you can see a summary description of it on the right. Some further suggestion   .  consider using advisory release :  https://software.cisco.com/download/home/286320244/type/282046477/release/Amsterdam-17.3.5 , if not yet done.

                         These might be related too :

  https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvs30569

  https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvy18431

 M.

  

   

 M.



-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
    When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

I dont think this is the bug we are hitting

 

Cat9400 standalone chassis with dual supervisor inserted , after issue the "redundancy force-switchover" command, the previous active supervisor doesn't come e up.

Supervisor is up

"show module" from the new active supervisor doesn't see the new standby supervisor, and the console output of the new standby supervisor stuck at loading the IOS image.

On show module, Standby supervisor status is OK.

"show log" on new active supervisor shows below l..

I cant see past this line.

 

            - I would still advise to go for the mentioned advisory release , if not yet installed

 M.



-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
    When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

Noted on this.

I would like to determine the root cause to give as a reason for the upgrade.

 

          - That's the point , if the bug reports are not determining the cause (fully) , it is always good to go for the advisory release and check if that can resolve the issue.

 M.



-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
    When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

Leo Laohoo
Hall of Fame
Hall of Fame

@Noah L wrote:

May 30 17:18:52 PST: %RF-5-RF_RELOAD: Peer reload. Reason: EHSA standby down


EHSA Standby Down is a "generic" term.  If the cause of the crash is not "coded" or not found in the coded list of reasons, it will fall into this generic term.  

Please post the complete output to the following commands: 

  1. dir crashinfo-2:
  2. dir bootflash-2:core
  3. sh platform resources
  4. sh platform software status con brief
Review Cisco Networking for a $25 gift card