I had a customer with a UC540 call approx. three or four days ago, and he couldn't get from his home PC into his work PC via RDP. The work PC is behind a UC540. He restarted the UC540 (power cycle) and it seemed to start behaving.
Today he was having voice quality issues and I noticed a decent amount of jitter on the line (I use PingPlotterPro to monitor his Internet connection). So I had him power-cycle his AT&T U-Verse modem. After that, he was unable to get any Internet connections so he had to power-cycle his UC540.
Things seemed to improve a bit for a while, but then call quality suffered and I started to dig into the UC540 a bit, looking at its built-in diagnostics page. The output on the diagnostics page had this:
UC_540
Home Exec Configure
--------------------------------------------------------------------------------
Syslog logging: enabled (0 messages dropped, 10 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
No Active Message Discriminator.
No Inactive Message Discriminator.
Console logging: level debugging, 287 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 294 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
Persistent logging: disabled
No active filter modules.
Trap logging: level informational, 340 message lines logged
Logging Source-Interface: VRF Name:
Log Buffer (4096 bytes):
DNSOVERLOAD: Low available memory: dropping <id# 37096> from <cli 192.168.10.3>.
000270: Aug 15 22:28:32.467: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 20075> from <cli 192.168.10.3>.
000271: Aug 15 22:28:32.603: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 44542> from <cli 192.168.10.3>.
000272: Aug 15 22:28:33.407: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 39508> from <cli 192.168.10.2>.
000273: Aug 15 22:28:33.467: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 20075> from <cli 192.168.10.3>.
000274: Aug 15 22:28:35.467: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 20075> from <cli 192.168.10.3>.
000275: Aug 15 22:28:36.247: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 11572> from <cli 192.168.10.3>.
000276: Aug 15 22:28:50.224: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for CCSIP CCB Pool . No memory available -Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FA951Cz 803043E0z 802E9ECCz
000277: Aug 15 22:29:03.652: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 46135> from <cli 192.168.10.3>.
000278: Aug 15 22:29:04.660: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 46135> from <cli 192.168.10.3>.
000279: Aug 15 22:29:05.648: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 46135> from <cli 192.168.10.3>.
000280: Aug 15 22:29:05.848: %SYS-2-MALLOCFAIL: Memory allocation of 736768 bytes failed from 0x82FE819C, alignment 8
Pool: Processor Free: 2282616 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FAB350z 82FCF168z 82FE81A0z 82FA924Cz 82FA9478z 803043E0z 802E9ECCz
000281: Aug 15 22:29:05.848: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for CCSIP CCB
[snip]
I did a couple of restarts of the UC540 via a telnet "reload" and kept getting that same result. Then I rebooted the PC I was using (which was 192.168.10.3) and made sure CCA wasn't started, then did another reload and now I'm getting this:
UC_540
Home Exec Configure
--------------------------------------------------------------------------------
Syslog logging: enabled (0 messages dropped, 10 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
No Active Message Discriminator.
No Inactive Message Discriminator.
Console logging: level debugging, 256 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 263 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
Persistent logging: disabled
No active filter modules.
Trap logging: level informational, 304 message lines logged
Logging Source-Interface: VRF Name:
Log Buffer (4096 bytes):
Processor Free: 5774524 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FAB350z 82FCF168z 82FE81A0z 82FA924Cz 82FA9478z 803043E0z 802E9ECCz
000245: Aug 15 22:48:51.765: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for CCSIP CCB Pool . No memory available -Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FA951Cz 803043E0z 802E9ECCz
000246: Aug 15 22:48:56.593: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 19106> from <cli 192.168.10.2>.
000247: Aug 15 22:49:09.914: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for CCSIP CCB Pool . No memory available -Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FA951Cz 803043E0z 802E9ECCz
000248: Aug 15 22:49:27.947: %SYS-2-MALLOCFAIL: Memory allocation of 736768 bytes failed from 0x82FE819C, alignment 8
Pool: Processor Free: 5231612 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FAB350z 82FCF168z 82FE81A0z 82FA924Cz 82FA9478z 803043E0z 802E9ECCz
000249: Aug 15 22:49:27.947: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for CCSIP CCB Pool . No memory available -Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FA951Cz 803043E0z 802E9ECCz
000250: Aug 15 22:49:31.595: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 8676> from <cli 192.168.10.2>.
000251: Aug 15 22:49:36.595: %DNSSERVER-3-UDPDNSOVERLOAD: Low available memory: dropping <id# 8676> from <cli 192.168.10.2>.
000252: Aug 15 22:49:45.859: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for CCSIP CCB Pool . No memory available -Process= "Chunk Manager", ipl= 6, pid= 1
-Traceback= 8177F9F8z 82FA951Cz 803043E0z 802E9ECCz
[snip]
I'm truncating those logs a bit, I'm not sure how many lines helpers here may need telling us we're low on memory.
So I'm not sure how to proceed. After a reload, the SIP trunk takes maybe a call or two, and then I can't call in any more. I'm not sure about calling OUT, I'm remote to the site.
Any ideas on where to start?
