7.4.8 Firmware for SPA508G Problems

[bill.roland]

Is anyone else seeing major problems when updating the SPA508G to the latest 7.4.8 firmware?  I sure am.  I used CCA to drag and drop the new firmware for the 525G and the 508G.  The 525G's all updated without any problem, but the 5 508G's I updated were all messed up by this firmware.  The firmware would download and it said it upgraded successfully, but after coming back up, they were all broken.  Best of all, they all exhibited slightly different behavior.  One would randomly reboot while just sitting idle, not doing anything.  Another one would reboot if you hit the speakerphone button or picked up the handset.  Another would make or receive a call, but got stuck and showed the call still going after hanging up the handset.  Strangely enough, hitting the Menu key works ok and the menu functions all seem to work fine, its just anything involving the phone portion (making or receiving calls) that is broken. I went into the UC560 (running software pack 8.1.0) CLI and changed the phone load back to 7.4.7 for the SPA508G's, rebooted the phones, they downgraded the firmware and came back up fine.

I downloaded a fresh copy of the firmware and tried again, with the same results.  I could accept one bad phone upgrade, but all of them, and just that one model?  Is anyone else seeing this?

[stpaulalex]

I'm planning to upgrade about a dozen phones this weekend to 7.4.8.

I've already installed it on a test 508G, and it seems to have fixed

some problems we were experiencing (though I only tested it for about 30

minutes). I'll try to remember to post how it goes.

-Alex

[bill.roland]

I'll be eager to hear your results.  I did manage to get one SPA508G to take the new firmware without any strange effects, so I am sure the BIN file is ok, but something about this upgrade is not sitting well with the majority of my 508s.

[Steven Howes]

During the UC300 product trial, they gave us 7.4.8v ('v' seems to be a special UC300 build). This made the 508 very unstable (to the point of being unusable) but the 504 worked. This was reported, and a test build of 7.4.8v2 was given to us (which resolved the issue). It would be interesting if this fix wasn't moved from the 'v2' built into the 'normal' build before release...

[Steven DiStefano]

Bill,

Thank you for reporting this and I hope I can tell you its not a phone FW problem, but I dont know that for certain yet and am investigating internally with the phone engineering team.  I will come back here and post what I learn ASAP.   If anyone is also seeing issues with difficulty keeping SPA508 Registered, or rebooting after calls sent or received, please post your experience here and open a case of you have the time.

Truly sorry for the inconvenience here to Bill and others.

Steve

[bill.roland]

Thanks everybody.  I just posted it as a heads up to anyone getting ready to try this on a large deployment; they might want to do some extended testing first.  As I was able to roll back to 7.4.7 successfully, it didn't really cause any real trouble.

[Steven DiStefano]

Excellent advise Bill !!!

We have several of our best Engineers looking at what could potentially cause this since its not easily reproducible, but it seems to have been reported a few times as a sporadic incident.  As such we are taking it very seriously.  I dont want to post anything prematurely, but I would think that 'staging' your deployment makes ALOT of sense right here.  It is really a critical step to success.  We dont want to see anyone show up and throw up, so set a few phones up and confirm their operation with your configuration before you deploy for the next 24 hours, at which time we should have feedback....

very sorry but it is this community that will help many thanks to good people like you.

Steve

[Steven DiStefano · Accepted solution]

FEEDBACK:

The bug is triggered by memory corruption in a stack overflow scenario. However, this bug only affects customers running the following config – SPA 50x or SPA 30x model phone + UC500 in SPCP model. It DOES NOT AFFECT the SPA 50x/SPA 30x phones deployed in SIP environments. Additionally, the SPA525G2 is unaffected by this bug regardless of SIP or SPCP deployment. The bug, though re-creatable, actually occurs in a pseudo-random manner i.e you need a specific configuration on the phones to trigger the bug.  Once you hit that config, the bug can be re-created almost at will. With a different config, you would not know of the existence of the bug.

The problem was brought to our notice at 6am PST yesterday.  We filed a bug CSCto11646 to address it at 10.45am. By 8pm last night, an image with the fix incorporated had been developed. That image is undergoing regression tests as I type this.
 
Immediate next steps for our field and customers:
1.      For customers with SIP deployments, continue to use 7.4.8.  It is a good release.

2.      For customers with UC500 SPCP deployments, use software pack 8.1.0 (which has 7.4.6 firmware bundled) OR separately download 7.4.7.

We plan to post a regression tested 7.4.8a to CCO on 4/4/2011 (if all testing goes well) at which time, customers using UC500 can upgrade to the 7.4.8a version of SPA phone firmware. This firmware will also be rolled into the next software pack for the UC500 customers.

[bill.roland]

Thanks for the info Steven!  Our SPA phones are indeed running in SPCP mode.  Will look forward to the fix.  Thanks again.

Bill

[stpaulalex]

Thanks for getting this straightened out Steven-- much appreciated.  Since I'm running my phones in SIP on Asterisk, I'm going to run the update this weekend.  By the way, is there any chance the firmware's source code is publicly available?

[William Gucciard]

I'm having the same issue with a spa508G and UC320; luckly it's a demo kit and not being deployed at a customer site.

Firmware on this particular phone is 7.4.7 v2.

Phone boots to the point of Initializing network then sits for about 10 seconds then reboots.  So far I haven't been able to roll back the firmware because of how unstable it is.  Occasionally it will boot completely but then only stays up for about 30 seconds.  Durring that time any changes to the config via the phone cause a reboot, access to the HTTP config causes the same and none of the CCA or Firmware recovery tools seem to be able to see or connect to it on the network.  Is there another way to flash that phone with a different firmware?

[hoorock89]

William,

I'd recommend removing the 508g from the UC320 environment and either power brick the phone and run a factory reset off the pohone once it comes up, or connect it to a PoE switch that's not connected to the UC320 and factory resetting it from there.  After the phone factory resets while not attached to any call control, unplug it and put it back on the UC320.  I've had a phone or two have the issue you described and wiping the config with a factory reset while not attached to any call control has seemed to do the trick.

Hope this helps...

Brent