CSCut03205 - SPA modules on ASR1K show "missing" under show platform output

[siportalsrl]

Something worse might happen. You don't see "missing" but you could not see anything.

I'm working on an ASR1002-X with this image loaded: asr1002x-universalk9.03.12.00.S.154-2.S-std.SPA.bin

  I added a SPA-1X10GE-L-V module. There are already two of these modules in the chassis.

The OIR feature should have recognized the module ...

Instead this happen:

ASR1002-a#show hw-module all fpd
==== ====================== ====== =============================================
                             H/W   Field Programmable   Current   Min. Required
Slot Card Type               Ver.  Device: "ID-Name"    Version      Version
==== ====================== ====== ================== =========== ==============
0/0 6XGE-BUILT-IN           1.0   1-2KP HSPA BULLSEY     2.34        2.34
---- ---------------------- ------ ------------------ ----------- --------------
0/1 SPA-1X10GE-L-V2         1.5   1-10GE I/O FPGA        1.9         1.9
---- ---------------------- ------ ------------------ ----------- --------------
0/2 SPA-1X10GE-L-V2         1.5   1-10GE I/O FPGA        1.9         1.9
==== ====================== ====== =============================================

ASR1002-a#request  platform software console attach 0/3
% Error: A SPA card is not available at 0/3        <----

ASR1002-b#hw-module subslot 0/3 start
%Command cannot be executed. Card not present subslot 0/3
ASR1002-b#show hw-module subslot all oir
Module        Model                Operational Status
------------- -------------------- ------------------------
subslot 0/0   6XGE-BUILT-IN        ok
subslot 0/1   SPA-1X10GE-L-V2      ok
subslot 0/2   SPA-1X10GE-L-V2      ok

There is no way to have the module on-line. So I had the reboot the router.
It is interesting that during shutdown the system sees the card:

*Nov 30 08:53:04.981: %IOSXE_OIR-6-REMSPA: SPA removed from subslot 0/0, interfaces disabled
*Nov 30 08:53:04.981: %IOSXE_OIR-6-REMSPA: SPA removed from subslot 0/1, interfaces disabled
*Nov 30 08:53:04.981: %IOSXE_OIR-6-REMSPA: SPA removed from subslot 0/2, interfaces disabled
*Nov 30 08:53:04.982: %IOSXE_OIR-6-REMSPA: SPA removed from subslot 0/3, interfaces disabled
*Nov 30 08:53:04.984: %SPA_OIR-6-OFFLINECARD: SPA (6XGE-BUILT-IN) offline in subslot 0/0
*Nov 30 08:53:04.985: %SPA_OIR-6-OFFLINECARD: SPA (SPA-1X10GE-L-V2) offline in subslot 0/1
*Nov 30 08:53:04.985: %SPA_OIR-6-OFFLINECARD: SPA (SPA-1X10GE-L-V2) offline in subslot 0/2
*Nov 30 08:53:04.985: %SPA_OIR-6-OFFLINECARD: SPA (SPA-1X10GE-L-V2) offline in subslot 0/3

During the reboot the card is finally recognized.

*Nov 30 08:53:12.148: %SPA_OIR-6-ONLINECARD: SPA (SPA-1X10GE-L-V2) online in subslot 0/3

Gianrico Fichera

[Palani Mohan · Accepted solution]

Gianrico

OIR is kinda misunderstood and misleading. To me, OIR is when I first remove an existing module and then insert a "like" module. So, instead of OIR, it should be ORI!.

Inserting new hw in an empty slot is not (technically) OIR.

While the term OIR indicates the hw components may be removed/inserted when the device is online, it does not mean that you can remove/insert the component without giving notice to the router. By notice to the router, I mean issuing cmds like hw-module stop|shutdown OR no power enable and such.

Before we remove/replace hw, we need to go through the process of removing cables, anyways. Why not take the extra step of powering down that slot/subslot alone before removing the hw?

Take a look at CSCuo77698 Its status is "Closed" meaning that it is recommended to turn power off to the slot/subslot before removing/inserting hw.

Kind regards .... Palani

[siportalsrl]

thankyou for your answer, I'll try to do it

[siportalsrl]

I understand that ASR1k doesn't support something like this:

"The Modular Cisco Catalyst switches, such as the 6500, 6000, 5500, 5000, 4500, and 4000 Series, support Online Insertion and Removal (OIR) or Hot Swap of all modules (power supplies, fan trays, Supervisor Modules and other Line and Service Modules). You can add, replace, or remove modules without interrupting the system power or causing other software or interfaces to shut down."

I'm right?

thankyou

[Palani Mohan · Accepted solution]

Hi Gianrico

The modules for ASR/1ks are expected to behave in a manner consistent with what you stated "add, replace, or remove modules without interrupting the system power or causing other software or interfaces to shut down."

Problem with this statement is that it does not mean that everytime it is done, there won't be any service interruption. Anytime we touch hw, there is a potential to create a service interruption.

CSCuo77698 was filed to investigate this problem. This has been closed after unsuccessful attempts to reproduce the same in our labs. The Close comments state that "possible fpga image corruption during auto upgrade triggered due to improper spa insertion"

As a user, kindly consider safe practice anytime there is a need to remove/insert hw:

• turn power off, for the slot
• remove cables
• Remove hw (optional)
• Insert replacement/new

Following the above steps significantly reduces the unexpected outages.

Kind regards .... Palani

[mohanantassp]

*Aug 11 04:16:31.224: %SMART_LIC-6-EXPORT_CONTROLLED: Usage of export controlled features is not allowed
*Aug 11 04:16:34.553: %ISR_THROUGHPUT-6-CRYPTO: Crypto level has been set to 50000 kbpsesg switch eventq crimson DB not initialized
*Aug 11 04:16:36.671: %SMART_LIC-6-AGENT_READY: Smart Agent for Licensing is initialized
*Aug 11 04:16:36.672: %SMART_LIC-6-AGENT_ENABLED: Smart Agent for Licensing is enabled
*Aug 11 04:16:36.672: %SMART_LIC-6-EXPORT_CONTROLLED: Usage of export controlled features is not allowed
*Aug 11 04:16:39.648: %SMART_LIC-6-HA_ROLE_CHANGED: Smart Agent HA role changed to Active.
*Aug 11 04:16:41.734: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
*Aug 11 04:16:42.524: %LINK-3-UPDOWN: Interface Lsmpi0, changed state to up
*Aug 11 04:16:42.543: %LINK-3-UPDOWN: Interface EOBC0, changed state to up
*Aug 11 04:16:42.544: %LINK-3-UPDOWN: Interface LIIN0, changed state to up
*Aug 11 04:16:36.824: %CMLIB-6-THROUGHPUT_VALUE: R0/0: cmand: Throughput license found, throughput set to 50000 kbps
*Aug 11 04:16:44.091: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
*Aug 11 04:16:44.094: %LINEPROTO-5-UPDOWN: Line protocol on Interface Lsmpi0, changed state to up
*Aug 11 04:16:44.094: %LINEPROTO-5-UPDOWN: Line protocol on Interface EOBC0, changed state to up
*Aug 11 04:16:44.095: %LINEPROTO-5-UPDOWN: Line protocol on Interface LIIN0, changed state to up
*Aug 11 04:16:45.122: %ONEP_BASE-6-SS_ENABLED: ONEP: Service set Base was enabled by Default
*Aug 11 04:16:47.302: %SYS-7-NVRAM_INIT_WAIT_TIME: Waited 0 seconds for NVRAM to be available
*Aug 11 04:16:48.008: %CRYPTO_ENGINE-5-KEY_ADDITION: A key named TP-self-signed-453923930 has been generated or imported by crypto config
*Aug 11 04:16:48.020: %CRYPTO_ENGINE-5-KEY_ADDITION: A key named 1224E-EB-001.digi.com.my has been generated or imported by crypto config
*Aug 11 04:16:48.034: %SYS-2-PRIVCFG_DECRYPT: Successfully apply the private config file
*Aug 11 12:16:48.164 MYT: %SYS-6-CLOCKUPDATE: System clock has been updated from 04:16:48 UTC Mon Aug 11 2025 to 12:16:48 MYT Mon Aug 11 2025, configured from console by vty0.
*Aug 11 12:16:48.276 MYT: %PKI-6-TRUSTPOINT_CREATE: Trustpoint: TP-self-signed-453923930 created succesfully
*Aug 11 12:16:48.285 MYT: %PKI-6-TRUSTPOINT_CREATE: Trustpoint: SLA-TrustPoint created succesfully
*Aug 11 12:16:48.371 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
*Aug 11 12:16:48.387 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback10, changed state to up
*Aug 11 12:16:49.442 MYT: %AAAA-4-CLI_DEPRECATED: WARNING: Command has been added to the configuration using a type 0 password. However, type 0 passwords will soon be deprecated. Migrate to a supported password type
*Aug 11 12:16:49.442 MYT: %AAAA-4-CLI_DEPRECATED: WARNING: Command has been added to the configuration using a type 0 password. However, type 0 passwords will soon be deprecated. Migrate to a supported password type
*Aug 11 12:16:49.443 MYT: %AAAA-4-CLI_DEPRECATED: WARNING: Command has been added to the configuration using a type 0 password. However, type 0 passwords will soon be deprecated. Migrate to a supported password type
*Aug 11 12:16:49.443 MYT: %AAAA-4-CLI_DEPRECATED: WARNING: Command has been added to the configuration using a type 0 password. However, type 0 passwords will soon be deprecated. Migrate to a supported password type
*Aug 11 12:16:49.469 MYT: %SYS-5-CONFIG_I: Configured from memory by console
*Aug 11 12:16:49.485 MYT: %IOSXE_OIR-6-REMSPA: SPA removed from subslot 0/0, interfaces disabled
*Aug 11 12:16:49.486 MYT: %IOSXE_OIR-6-REMSPA: SPA removed from subslot 0/1, interfaces disabled
*Aug 11 12:16:49.501 MYT: %SPA_OIR-6-OFFLINECARD: SPA (C1111-2x1GE) offline in subslot 0/0
*Aug 11 12:16:49.508 MYT: %SPA_OIR-6-OFFLINECARD: SPA (C1111-ES-4) offline in subslot 0/1
*Aug 11 12:16:49.573 MYT: %IOSXE_OIR-6-INSCARD: Card (fp) inserted in slot F0
*Aug 11 12:16:49.574 MYT: %IOSXE_OIR-6-ONLINECARD: Card (fp) online in slot F0
*Aug 11 12:16:49.599 MYT: %IOSXE_OIR-6-INSCARD: Card (cc) inserted in slot 0
*Aug 11 12:16:49.599 MYT: %IOSXE_OIR-6-ONLINECARD: Card (cc) online in slot 0
*Aug 11 12:16:49.664 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan111, changed state to down
*Aug 11 12:16:49.665 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan422, changed state to down
*Aug 11 12:16:49.717 MYT: %IOSXE_OIR-6-INSSPA: SPA inserted in subslot 0/0
*Aug 11 12:16:49.724 MYT: %IOSXE_OIR-6-INSSPA: SPA inserted in subslot 0/1
*Aug 11 12:16:49.888 MYT: %SYS-5-RESTART: System restarted --
Cisco IOS Software [Gibraltar], ISR Software (ARMV8EL_LINUX_IOSD-UNIVERSALK9-M), Version 16.12.4, RELEASE SOFTWARE (fc5)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2020 by Cisco Systems, Inc.
Compiled Thu 09-Jul-20 20:16 by mcpre
*Aug 11 12:16:49.927 MYT: %SSH-5-ENABLED: SSH 1.99 has been enabled
*Aug 11 12:16:50.528 MYT: %LINK-5-CHANGED: Interface Vlan1, changed state to administratively down
*Aug 11 12:16:50.630 MYT: %CRYPTO_ENGINE-5-KEY_ADDITION: A key named TP-self-signed-453923930.server has been generated or imported by crypto-engine
*Aug 11 12:16:51.176 MYT: %SNMP-5-COLDSTART: SNMP agent on host 1224E-EB-001 is undergoing a cold start
*Aug 11 12:16:55.591 MYT: %PNP-6-PNP_DISCOVERY_STOPPED: PnP Discovery stopped (Startup Config Present)
*Aug 11 12:16:57.900 MYT: %TRANSCEIVER-6-INSERTED: R0/0: iomd: transceiver module inserted in Gigabitethernet0/0/0
*Aug 11 12:16:59.071 MYT: %SPA_OIR-6-ONLINECARD: SPA (C1111-2x1GE) online in subslot 0/0
*Aug 11 12:16:59.584 MYT: %SPA_OIR-6-ONLINECARD: SPA (C1111-ES-4) online in subslot 0/1
*Aug 11 12:16:59.877 MYT: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/0, changed state to down
*Aug 11 12:17:01.523 MYT: %LINK-3-UPDOWN: Interface GigabitEthernet0/1/0, changed state to down
*Aug 11 12:17:01.536 MYT: %LINK-3-UPDOWN: Interface GigabitEthernet0/1/1, changed state to down
*Aug 11 12:17:02.511 MYT: %PKI-6-TRUSTPOINT_CREATE: Trustpoint: CISCO_IDEVID_SUDI created succesfully
*Aug 11 12:17:02.531 MYT: %CRYPTO_ENGINE-5-KEY_ADDITION: A key named CISCO_IDEVID_SUDI has been generated or imported by pki-sudi
*Aug 11 12:17:02.558 MYT: %PKI-6-TRUSTPOINT_CREATE: Trustpoint: CISCO_IDEVID_SUDI0 created succesfully
*Aug 11 12:17:02.565 MYT: %PKI-2-NON_AUTHORITATIVE_CLOCK: PKI functions can not be initialized until an authoritative time source, like NTP, can be obtained.
*Aug 11 12:17:05.036 MYT: %LINK-3-UPDOWN: Interface GigabitEthernet0/1/1, changed state to up
*Aug 11 12:17:06.037 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1/1, changed state to up
*Aug 11 12:17:06.044 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan111, changed state to up
*Aug 11 12:17:07.028 MYT: %LINK-3-UPDOWN: Interface GigabitEthernet0/1/0, changed state to up
*Aug 11 12:17:08.028 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1/0, changed state to up
*Aug 11 12:17:08.036 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan422, changed state to up
*Aug 11 12:17:20.889 MYT: %SMART_LIC-6-EXPORT_CONTROLLED: Usage of export controlled features is not allowed
*Aug 11 12:17:20.890 MYT: %CALL_HOME-6-CALL_HOME_ENABLED: Call-home is enabled by Smart Agent for Licensing.
*Aug 11 12:17:33.882 MYT: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: admin] [Source: LOCAL] [localport: 0] at 12:17:33 MYT Mon Aug 11 2025
*Aug 11 12:17:47.644 MYT: %TRANSCEIVER-6-REMOVED: R0/0: iomd: Transceiver module removed from Gigabitethernet0/0/0
*Aug 11 12:17:51.653 MYT: %TRANSCEIVER-6-INSERTED: R0/0: iomd: transceiver module inserted in Gigabitethernet0/0/0
*Aug 11 12:17:56.623 MYT: %LINK-3-UPDOWN: Interface GigabitEthernet0/0/0, changed state to up
*Aug 11 12:17:57.624 MYT: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0/0, changed state to up
*Aug 11 12:18:16.574 MYT: %SYS-6-LOGOUT: User admin has exited tty session 0()

kind of a similar problem, when i restart the router, the SFP module become down. once the router is fully up i need to unplug the SFP and plug it back and then the link will be up.. anyone face this issues in C1111-4p, ,i have 3 to 4 sites having the same issues.. please help