Cisco Sidecar Registration Issues

[Bob Boklewski]

Guys, 

I have about 20 SPA504G's and maybe half that have 32 attendant consoles.  This is a managed service from our ISP so no PBX on site.  Since we got this system about a year and a half ago we have always seemed to have random issues with the attendant consoles.  All lights would go blank except maybe a red light in the upper left or sometimes a few green ones, but all the rest have dropped off.  The rest of the buttons come back after a little while, but lately it seems like it is getting worse.  My ISP has changed from UDP to TCP for signaling, which did not fix the issue.  It is a random thing and it has happened to all side cars up until this point, so no one attendant console is okay.  Sometimes mine can be out while my neighbors is okay and vice versa.  My ISP is now saying it is my firewall (Cisco ISA570) causing the problem because of NAT.  I beg to differ as from what I gather, the attendant console hits the same proxy server for registration as does the phone itself.  We never have an issue with loss of service, dropped calls, one way audio, etc.  There are zero issues in this regard, the only issue is the sidecars.  I have enabled keep alive and turned on Nat mapping on the phone itself as it was suggested on some sites, but really I am reaching at this point.  Just wondering what you all think here, trying to get this resolved.

Thanks, 

BB

[Bob Boklewski]

Anybody out there with insight?

[Bob Boklewski]

Any cisco voip engineers here?

[Dan Lukes]

All Cisco experts evaporated about two years ago. Expect no response from Cisco guys here. Buy support contract and call SMB TAC. But according my experience, don't expect help even from them.

If you wish a reply from volunteers unrelated to Cisco, you need to disclose details. So turn on syslog&debug messages and catch them as well as SIP communication between your device and upstream provider. It will help you (or someone else) to analyze the issue.

[mdobiac]

Hello Bob Boklewski,

As I do not have any debugs to work off of, there are a few things you can look at to see if it may be a provider issue or how they may have their configuration or association of your devices working with their services.

However I do have a few questions.  When you power cycle the phone does the side car come back up and work as expected?  Additionally after the sidecar is operational about how long does it stay operational?

Now on to what could be transpiring.  When the SPA500S is not working if you can log into the web interface of the phone via its IP address through any browser.  Once logged in click on Admin login and advanced.  (If you get prompted to log in using a Username and password you will need to contact the provider as they offer their services via provisioning your devices and we will not have access to manage them.)

If you were not prompted for a log in then go under the Attendant Console Tab.  Once you click on that tab you should be under the Unit 1 tab.  The fields you will be interested in are:

Unit Enabled:  should be Yes

Unit Online:  should be Yes

Subscribe expires:  by default it is 1800 seconds.

My first thought is that the unit is no longer Online.  This can be cause from a few things.  The aux cable (though yours sounds like they are still good), or they are not resyncing upon the subscription renewal.

As I am not the provider they could be having a syncing issue when the side car is trying to renew at a different time from when the phone is scheduled to renew and that could potentially cause confusion with the sidecar syncing up to the phone/provider.  Basically the sidecar may not resync due to the timer issues or that the phone no longer sees that it has a side car associated to it.

One thing you could possibly check is with your provider to see if the phones and the side car can resync at the same times.  So set the Registration expires timer on the SPA504G to match 1800 seconds so that the phone and side car resync at the same time.  This may stabilize the side car/s.

Other than that it will really depend on the debugs to see what is truly going on.

Hope this helps.

Regards,

Michael D.

[Bob Boklewski]

Thanks for the posts guys.  So, my ISP wants to rule out an issue because we are behind a NAT device.  I will plug the phone straight into a second interface my ISP enabled for me on their router, will see what happens.  I did also try under the web setting (EXT1 tab) to enable NAT Keep Alive and NAT Mapping, but that didn't do anything.

Also, when I reboot the phone when the sidecar is out, the sidecar always comes back online again.  Sometimes my sidecar can go out 4 times a day, other times one.  All of our phones that have sidecars it does it to.  Some can be working while others are not, but every time I see one out, there is always other ones out as well.  They seem to be out about 30 minutes, so it definitely seems like when it tries to register it can't.  I will definitely check to see if it says unit enabled/unit online when I see it go out again, and after checking, the subscribe expires: 1800. 

Here is something to note too.  When the sidecar goes out I generally get some random extensions back on the sidecar, and they are not in the order that they should be in on the sidecar.  The led's always start in the upper left of the sidecar after it goes out, one pops on, maybe another, maybe another, might show like 4 lit up and that is it, strange.  Then around 30 minutes later they all pop back on in the correct order.  Sometimes it is longer than 30 minutes, wondering if sometimes it does have issues registering after a second attempt, not sure.  I will try to note the times as well. 

Also, the strange thing is, we never lose a phone connection and my users are on calls that last almost 6 hours a day.  It is only the sidecar issue.  

[mdobiac]

Hello Bob Boklewski,

Another thing that could be tried is looking at the timers.  Since the phone resync ever 3600 seconds (1hr) and the sidecar resyncs ever 1800 seconds (30 minutes) it could be that when the phone is resyncing it is taking longer than the sidecar and the sidecar does not resync.

What you could also ask the provider if it will be okay to change the Registration expire time to 28800 (8 hours) to see if that stabilizes the side cars as well.  I recommend you ask the provider because it could potentially disrupt services depending on the providers configuration on their side.

Regards,

Michael D.

[Bob Boklewski]

We are working on this with our ISP.  We did bypass our work firewall and plugged a phone in a secondary interface on the ISP router and we still got outages on the sidecar.  So, it isn't our network causing it.  I am going to try and get some packet captures to see possibly what is going on.  I will post when I can get some data gathered.

[mdobiac]

Hello Bob Boklewski,

Thank you for the follow up and I look forward to seeing the captures to help give some insight into the issue.

Regards,

Michael D.