Cisco 4K PVDM4 stuck in booting

[johnlloyd_13]

hi,

we have a 4K SIP router and got some issue with outbound calls.

saw these logs and PVDM4 stucked in 'booting'

the PVDM4 still shows in the 'show inventory'

my questions are:

1.can this be resolved by a reboot?

2.or do i need to reseat the PVDM4 module?

3.is the module for TAC RMA?

NAME: "PVDM subslot 0/4", DESCR: "PVDM4-128 Voice DSP Module"
PID: PVDM4-128 , VID: V02 , SN: FOCxx

Jan 27 23:36:23.739 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4
Jan 27 23:43:33.766 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 27 23:43:33.769 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4
Jan 27 23:50:43.795 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 27 23:50:43.798 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4
Jan 27 23:57:53.820 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 27 23:57:53.823 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4

Jan 28 00:40:53.997 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 28 00:40:54.000 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4
Jan 28 00:48:04.022 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 28 00:48:04.025 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4
Jan 28 00:55:14.052 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 28 00:55:14.055 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4
Jan 28 01:02:24.084 UTC: %FARM_DSPRM-6-CARDREMOVEINFO: dspfarm card slot 4 is removed.
Jan 28 01:02:24.087 UTC: %SPA_OIR-6-OFFLINECARD: SPA (PVDM4-128) offline in subslot 0/4

#show hw-module subslo 0/4 oir
Module Model Operational Status
----------------------- -------------------- ------------------------
subslot 0/4 PVDM4-128 booting

#sh platform
Chassis type: ISR4xx/K9

Slot Type State Insert time (ago)
--------- ------------------- --------------------- -----------------
0 ISR4431/K9 ok 43w2d
0/0 ISR4431-X-4x1GE ok 43w2d
0/4 PVDM4-128 booting 00:00:42
R0 ISR4431/K9 ok, active 43w2d
F0 ISR4431/K9 ok, active 43w2d
P0 PWR-4430-AC ok 43w2d
P1 PWR-4430-AC ok 43w2d
P2 ACS-4430-FANASSY ok 43w2d

Slot CPLD Version Firmware Version
--------- ------------------- ---------------------------------------
0 16092742 16.12(2r)
R0 16092742 16.12(2r)
F0 16092742 16.12(2r)

[balaji.bandi]

what version code running ?

when you insert the card, can you post complete logs (do you see any other error other than posted one above ?)

Also other option - if you get maintenance window - Turn Off whole Router and re-insert the module and check.

https://www.cisco.com/c/en/us/td/docs/routers/access/interfaces/NIM/hardware/installation/guide/InstPVDM4-book.html

[johnlloyd_13]

hi balaji,

it's running on 16.12.x

the PDVM4 is not a NIM card but it's mounted on the motherboard, so i'll need a downtime for this.

[balaji.bandi]

Sure you need to have download and reseat and test it.

[liviu.gheorghe]

Maybe it's a good ideea if you have another spare PVDM4 and when you open the router and re-seat the PVDM4 see if this solves the problem. If not change it with the spare one.

If you don't have a spare PVDM4 and still have support on the router, you should open a TAC case.

[ES Berlin]

Has the ‘stuck in booting’ problem been solved somehow?

I have a similar problem with an LTE module (NIM-LTEA-EA)

000139: *Jan 24 12:22:42.167 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000140: *Jan 24 12:22:42.167 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000141: *Jan 24 12:22:42.170 MET: %SPA_OIR-6-OFFLINECARD: SPA (NIM-LTEA-EA) offline in subslot 0/2
000142: *Jan 24 12:29:52.213 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000143: *Jan 24 12:29:52.214 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000144: *Jan 24 12:29:52.216 MET: %SPA_OIR-6-OFFLINECARD: SPA (NIM-LTEA-EA) offline in subslot 0/2
000145: *Jan 24 12:37:02.254 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000146: *Jan 24 12:37:02.254 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000147: *Jan 24 12:37:02.257 MET: %SPA_OIR-6-OFFLINECARD: SPA (NIM-LTEA-EA) offline in subslot 0/2
000148: *Jan 24 12:44:12.290 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000149: *Jan 24 12:44:12.291 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000150: *Jan 24 12:44:12.293 MET: %SPA_OIR-6-OFFLINECARD: SPA (NIM-LTEA-EA) offline in subslot 0/2
000151: *Jan 24 12:51:22.332 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000152: *Jan 24 12:51:22.333 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000153: *Jan 24 12:51:22.335 MET: %SPA_OIR-6-OFFLINECARD: SPA (NIM-LTEA-EA) offline in subslot 0/2
000154: *Jan 24 12:58:32.378 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN
000155: *Jan 24 12:58:32.378 MET: %CELLWAN-2-MODEM_DOWN: Modem in slot 0/2 is DOWN

#sh platform
Chassis type: ISR4321/K9

Slot Type State Insert time (ago)
--------- ------------------- --------------------- -----------------
0 ISR4321/K9 ok 01:13:17
0/0 ISR4321-2x1GE ok 01:11:49
0/1 NIM-ES2-4 ok 01:11:48
0/2 NIM-LTEA-EA booting 00:04:51
R0 ISR4321/K9 ok, active 01:13:17
F0 ISR4321/K9 ok, active 01:13:17
P0 PWR-4320-AC ok 01:12:26
P2 ACS-4320-FANASSY ok 01:12:26

Slot CPLD Version Firmware Version
--------- ------------------- ---------------------------------------
0 17100927 16.12(2r)
R0 17100927 16.12(2r)
F0 17100927 16.12(2r)

#show inventory

NAME: "NIM subslot 0/2", DESCR: "NIM-LTEA-EA Module"

PID: NIM-LTEA-EA       , VID: V01  , SN: FOC***

#show hw-module subslot 0/2 oir
Module Model Operational Status
----------------------- -------------------- ------------------------
subslot 0/2 NIM-LTEA-EA booting

The problem appeared under the firmware:

isr4300-universalk9.17.09.03a.SPA.bin

But i also tried:

isr4300-universalk9.16.09.07.SPA.bin
isr4300-universalk9.17.09.05e.SPA.bin
isr4300-universalk9.17.12.04a.SPA.bin

And I also swapped the NIM between subslot 0/1 and 0/2 and tried another NIM-LTEA-EA-modul

But the error remained or moved to the other subslot.

Do any of you have any other ideas? Unfortunately I don't have a backup router

[liviu.gheorghe]

@ES Berlin it looks like a problem of module seating or a problem with the module itself.

First of all, can you look closely on the connector of the module? Examine the connector for bent pins or something stuck in the connector.

Second, try to examine the connector on the router - the slots in which the module was inserted - again look for the integrity of the pins.

Has this module ever worked or is it a new module?

[ES Berlin]

Just a RMA of the device helped me..