SPA50xG IP Phone Recovery Utility (SoS mode recovery)

[Patrick Born]

In the event that a SPA50xG IP phone experiences an interruption of power during a firmware upgrade, the phone's firmware images could be rendered unusable. If this occurs, the phone will display a message "SOS phone in recovery mode".

The phone will automatically attempt to acquire a valid image from the "last upgrade URL" that it knows about.

You can manually recover a SPA50xG phone from this state using the attached recovery utility. You can use the utility regardless of the firmware version of the phone.

Using the recovery utility

When the phone is trying to access the “last upgrade URL”, it will not be able to handle a recovery request from the PC. Once the phone gives up, you will be able to successfully use the recovery utility:

1. Make sure the default TFTP port (69) is not occupied on the computer. For example if you have a TFTP server such as Pumpkin or SolarWinds running, shut the TFTP service down.
2. You might have to disable the firewall or other security measures on the PC to allow TFTP to function properly.
3. Run the recovery utility and enter the phone’s SN when prompted. You can use “FFFFFFFFFFFF” (12 ‘F’s) instead of the phone's serial number.
4. The recovery utility should find the phone in recovery mode.
5. Follow the instructions on screen to start the recovery process.
6. Once the phone has recovered and rebooted, you can upgrade to the desired version of firmware.

<end>

[Alfa-Krasnoyarsk]

Здравствуйте, Патрик. 

Скажите, если на телефоне Cisco spa504g нет на дисплее SOS Phone In Recovery Mode. Т.е. подсветка дисплея горит, а на дисплее ничего не отображается. Что в этом случае можно сделать? Есть три телефона и все с подобной проблемой? 
Заранее спасибо за ответ. 

Hello, Patrick.

Say, if the phone is not on Cisco spa504g display SOS Phone In Recovery Mode. Those. the display light is on and nothing appears on the display. What then can be done? There are three phones all with a similar problem?
Thanks in advance for your reply.

[Dan Lukes]

If nothing appear on display, then phone's hardware is faulty.

If the phone is in warranty, lets authorized service repair it.

If out of warranty and you have spare display, you can replace it. Note I'm not claiming the display is faulty in your particular case, but it's common fault so you can try.

If it will not help, the phone is just dead.

[Alfa-Krasnoyarsk]

Hi, Dan Lukes.

When monitoring of traffic exchange on a port to connect the phone still shows signs of life. Apparently, the phone is still working, because lloader loaded lighting bulbs after the most likely problem is that there is no firmware. The display itself after the empty, lit only by light. There are three such apparatuses with the same fault. hard reset was made probably after that and left flashing. Interested in question, it is now possible to restore the phone to a working state. Phones are not under warranty.

При мониторинге обмена трафиком на порту подключения телефон все же показывает признаки жизни. Судя по всему, телефон все же рабочий, т.к.  лloader грузится зажигая лампочки после включения, скорее всего проблема в отсутствии прошивки. Сам дисплей после включения пустой, горит только подсветка. Есть три таких аппарата с одной и той же неисправностью. Скорее всего был сделан жесткий ресет, после чего и удалилась прошивка. Интересует вопрос, как теперь можно восстановить телефоны в рабочее состояние. Телефоны не на гарантии. 

[Dan Lukes]

Important note: sometime it's not allowed to upgrade from firmware version A to version B. For example, it's not possible to upgrade SPA50xG phone from firmware older than 7.5.2b to firmware newer than 7.5.2b directly.

Restriction like it may apply to recovery firmware as well.

If your's SPA504G in recovery mode claim recovery firmware version 7.4.4rec you can recover it to 7.4.9 (for example) but attempt to flash 7.5.5 may fail.

See this message for some additional information.

Also, you should not recovery your phone to older version than the recovery firmware is. Some newest hardware models are not supported by old firmware. E.g. if you have phone claiming recovery firmware version 7.5.4rec you should never try to recover it to 7.5.2 firmware.

Read always Release Notes of the firmware you are trying to recover to. It may contain further information.

[dagbakke2]

For anyone finding this document and is stuck nonetheless, I found there to be a timing component involved.

When the phone boots to the S.O.S. recovery screen, it will ask for a DHCP address. It is important that the recovery utility in the firmware distribution gets to run immediately after the phone has gotten an address.

So:

1. ensure your DHCPserver is running and serves addresses

2. fire up the recovery utility, ensure the machine where it is running is on the same subnet as the phone.

3. Choose recovery, enter 12 Fs. And hold it

4. Powercycle phone.

5. As soon as phone says S.O.S. or your DHCPserver indicates an address has been assigned to the phone, hit enter in the recovery utility.

At this point, the recovery utility should be able to connect to the phone, reveal what recovery version is running on the phone and offer the possibility to recover.

Also:

I had success with the 7.6.2 recovery utility, even though the phone claimed to be running the 7.5.1rec image. No success with any other version from the 7.5-series. May be related to the fact that the phone got interrupted when trying to flash 7.6.2.

[Dan Lukes]

I had success with the 7.6.2 recovery utility, even though the phone claimed to be running the 7.5.1rec image. No success with any other version from the 7.5-series.

So surprising information to be missed. Thus I bricked firmware in my test phone (don't try it at home) to verify it. I have no phone with 7.5.1rec inside, but the 7.4.4rec I have can be used as well.

I successfully recovered it with 7.6.2, 7.6.1, 7.6.0 as well as 7.5.5 recovery utility. it seems there's no 7.5.2b barrier here.

I assume it's matter of signing. Just guessing as I have no access to Cisco internals.

The running phone's image rejects upgrade unless new firmware is signed by key issued by certificate authority recognized as trusted (by running firmware).

I assume the original firmwares up to 7.5.2b are signed by key issued by CAold authority and recognize CAold to be trusted only, while firmwares newer than 7.5.2b are signed by a CAnew issued certificate and recognize both CAold and CAnew trusted.

7.5.2b itself is "bridge firmware". While signed by a CAold certificate (thus it is accepted by old firmware) it recognize both CAold and CAnew as trusted (so it can take new firmwares).

Originally, the same has been valid for ROM recovery firmware burned into phone.

But sometime in the past Cisco has considered to add CAnew certificate even to old firmwares burned into phone during manufacturing phones. As result, phones manufactured after such date recognize CAnew trusted thus it can take new firmware binary during recovery flashing. 7.5.2b barrier doesn't exist here.

Casual old firmware has not been recompiled&rereleased with CAnew embedded, thus the "short way" apply to recovery process only. If you have phone with running pre-7.5.2b firmware, the 7.5.2b is still the mandatory intermediate step to upgrade it to recent firmware.

[Dan Lukes]

It is important that the recovery utility in the firmware distribution gets to run immediately after the phone has gotten an address.

Although your advise is generally correct and should be honored, the time seems not to be cause. Or. at least, the only cause.

I did two test with phone in SOS recovery state - even after 8 hours (first test) or 14 hours (second test) past power on I has been able to recover firmware with no issue.

I also verified the phone is recoverable even after prematurely interrupted recovery attempt - I interrupted at least ten recovery attempts prematurely (with no reboot of the phone in question). Even after it the phone has been able to take (uninterrupted) recovery procedure with no issue.

I has used SPA504G (hw. ver 1.0.4) with 7.4.4rec recovery firmware.

[dagbakke2]

Huh.

That is interesting. I must have had 15-20 attempts at recovery without being able to detect the device, before it suddenly clicked when my movements became 'automated' enough. I then had a a few successful detections, but unsuccessful recovery attempts with the 7.5.7s recovery utility, before succeeding completely with 7.6.2.

Timing most definitely appeared to be an issue here. But as you noted, I have a different recovery image than you. And a 514G.

Anyways, I recovered my phone. And spent more time than I should have doing so. But it became an interesting challenge.  :-)

[Dan Lukes]

Hm, the further analysis require packets captured on the wire. But you have broken phone no longer and my equipment is not affected by issue. So time to forget  ;-)

I have a different recovery image than you. And a 514G.

Moreover I'm using own recovery utility (so I can recover even remote phone with no Windows-running computer available).

But different recovery code (in phone, not the different utility) is my preferred explanation of the behavior difference we observed. Unless it has been caused by your LAN network equipment (multicasts used during recovery process may be subject of filtering on some switches).

[uHu]

Resurrection of old thread - Hoping to resurrect a few SPA504G as well.

I have 4 dead SPA504G, three of hw version 01 and one v02. I also have 36 working SPA504G. This setup is a part of the emergency comms in the school, using paging via an Asterisk server, all on battery backup, and I need another 2-3 rooms covered. The classroom installed ones are wall mounted and without a handset.

All the four phones behave the same way: When they get power (PoE), the three bottom lights (buttons) light up for about 8 sec, then the screen lights up together with the 4 line buttons. The 4 line buttons go out after 12 sec, but the screen stays lit forever, but not showing anything. No text, logo, or anything on the screen; only background light.

On the network side, the switch port lights up, and the network app (Unifi Network) shows the port as in use, but there is no IP or traffic, or any detection of the connected device. I have not tried any sniffer to see if any traffic is sent at all.

I have tried the recovery tool in the opening post of this thread, both via the lan, with Windows10 PC and phone connected to the same switch; and directly connected to a stand-alone PC with a PoE injector betweeen. I get no communication at all.

Reading different forum posts, it seems that the phones might be unrecoverable. Does anyone know of a way of waking these phones to life again? Is it possible to solder in a new ROM e.g.

Alternatively, looking to find some more 2nd hand SPA504G phones, but haven't found any for the last few months. Which other phone could I use, which is easy to configure, easy to lock down to avoid misuse, and with a good, loud speaker?

Thankful for any advice, whichever flavour.