This issue happens at least once a day. Out of nowhere and suddenly, many IP phones randomly at the same time Unregistered from Callmanager and register back immediately. Below is a sample of the logs in event viewer:
Error: DeviceUnregistered - Device unregistered.
Device name.: SEP0017943A98BA
Device IP address.: 192.168.205.72
Device type. [Optional]: 8
Device description [Optional].: Steve Le 5422 |6123
Reason Code [Optional].: 8
App ID: Cisco CallManager
Cluster ID: StandAloneCluster
Node ID: 192.168.205.251
Explanation: A device that has previously registered with Cisco CallManager has unregistered. This event may be issued as part of normal unregistration event or due to some other reason such as loss of keepalives.
Recommended Action: No action is required if unregistration of this device was expected.
I've also realized that just exactly before this happens, I receive email notification about "LowCallManagerHeartbeatRate" :
Call Manager heartbeat rate below 24 beats per minute.
Current heartbeat rate is 19 beats per minute.
I receive this notification at least 5-6 times a day but not all of them are followed by above phone Unregistration issue. This issue seems was started just recently while without really making any changes in CallManager configuration . I'm also convinced that it could not be caused by IP Phones because it's quiet randomly and everytime some other phones get unregistered (Not to mention, CUE CTI ports happen to get unregistered as well). So my guess is the problem lays within callmanager itself but have no idea why? are devices missing keepalive from callmanager? If so could this be because of that? I do appreciate if anyone who has come across similar problem or know how to fix it please leave me a comment.-Thanks
Well, there are many possibilities here...can you attach a topology of what your environment looks like? Generic is fine. Just placement of servers and connection types. One thing that could be of issue is a physical or hardware issue that is manifesting itself such as a bad NIC (or NIC going bad) on a server or even a faulty cable which is causing loss of communication between nodes. Let's see what you're working with and go from there.
Please rate helpful posts!
I had a similar issue a few years back while running 4.1.3…random numbers of phones (sometimes as few as 2 or 3 to as much as 40) would just un-register completely or I would notice them jump to the backup. Eventually, I found the processor was spiking on the servers for long periods of time (20-40 seconds) because of a virus scan. I suggest you verify your processors are maxing out. It may sound a bit basic…but it’s very easy to rule out.
I hope this helps and Good luck!
Please check the reason code of the phones getting unregistered. Also you can collect a trace from back of the phone port and from CUCM. Check keep alives. Beside other reasons one reason is network issue.
1) you need to check reason code and read the detail from documentation.
2) by checked wireshark traces from the back of the phone and CUCM traces you can figure out if keep alives are getting lost in the network which will make phones to go down or reload.
Above all you mentioned all the phones face same problem check switch and see may be one module in the switch is having problem and only phones from that module reload.
Hope above help.