1941 Router frozen from time to time

[ciskonoypi]

Hi!

Please help me solve this problem. Ciso 1941 router frozen once a day, sometimes after 2 to 7 days. When the router frozen, no internet connection and cannot login/ping ethernet ports. I can login to console port and copy the error messages below. Reload the router and it will return back to normal operation. Re-installed IOS but still the same.

IOS Version 15.1(2)T2,

Cisco CISCO1941/K9 (revision 1.0) with 487424K/36864K bytes of memory.

DRAM configuration is 64 bits wide with parity disabled.

255K bytes of non-volatile configuration memory.

250880K bytes of ATA System CompactFlash 0 (Read/Write)

Thanks in advance.

000115: *Jun 30 15:08:40.065 CST: %SYS-2-MALLOCFAIL: Memory allocation of 21840 bytes failed from 0x24888070, alignment 0

Pool: Processor  Free: 849136  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "Licensing Auto Update Process", ipl= 0, pid= 12 -Traceback= 0x23414708z 0x24A13350z 0x24885A78z 0x248BD2FCz 0x2487FEB4z 0x2487DD1Cz 0x2487E648z 0x2487D614z 0x2102AC40z 0x233E7980z 0x233E7964z

000116: *Jun 30 15:08:40.069 CST: %LICENSE-2-VLS_ERROR: 'VLSnotifyBirthAndExpiryEvents' failed with an error - rc = 25 - 'Error[25]: Could not understand the message received from license server on host "no-net".

Client-server version mismatch?

' -Traceback= 0x233E7980z 0x233E7964z

000117: *Jun 30 15:09:11.597 CST: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x23431D80, alignment 32

Pool: Processor  Free: 859552  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "IP ARP Adjacency", ipl= 0, pid= 115 -Traceback= 0x23415F6Cz 0x2342F788z 0x224DD56Cz 0x224DDB18z 0x225338F4z 0x22868CDCz 0x22869040z 0x22869198z 0x233E7980z 0x233E7964z

000118: *Jun 30 15:09:11.597 CST: %ADJ-3-ALLOCATEFAIL: Failed to allocate an adjacency -Traceback= 0x224DDB18z 0x225338F4z 0x22868CDCz 0x22869040z 0x22869198z 0x233E7980z 0x233E7964z

000119: *Jun 30 15:09:40.077 CST: %LICENSE-2-VLS_ERROR: 'VLSnotifyBirthAndExpiryEvents' failed with an error - rc = 25 - 'Error[25]: Could not understand the message received from license server on host "no-net".

Client-server version mismatch?

' -Traceback= 0x233E7980z 0x233E7964z

000120: *Jun 30 15:09:41.601 CST: %SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from 0x23432024, alignment 0

Pool: Processor  Free: 859552  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "IP Input", ipl= 0, pid= 117 -Traceback= 0x234117BCz 0x2342FA2Cz 0x21ED3BA4z 0x21ED4420z 0x21ED13CCz 0x22865440z 0x228654F0z 0x228751F4z 0x223DFFF4z 0x22856554z 0x22856964z 0x2283F8DCz 0x22857E90z 0x2283FF94z 0x228401D0z

0x228402C4z

000121: *Jun 30 15:10:11.601 CST: %SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from 0x23432024, alignment 0

Pool: Processor  Free: 859552  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "IP Input", ipl= 0, pid= 117 -Traceback= 0x234117BCz 0x2342FA2Cz 0x21ED253Cz 0x21ECEA74z 0x22863CA0z 0x22863D1Cz 0x22864EECz 0x22864FB0z 0x22865488z 0x228654F0z 0x228751F4z 0x223DFFF4z 0x22856554z 0x22856964z 0x2283F8DCz

0x22857E90z

000122: *Jun 30 15:10:40.081 CST: %LICENSE-2-VLS_ERROR: 'VLSnotifyBirthAndExpiryEvents' failed with an error - rc = 25 - 'Error[25]: Could not understand the message received from license server on host "no-net".

Client-server version mismatch?

' -Traceback= 0x233E7980z 0x233E7964z

000123: *Jun 30 15:10:41.613 CST: %SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from 0x23432024, alignment 0

Pool: Processor  Free: 859408  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "ARP Input", ipl= 0, pid= 29 -Traceback= 0x234117BCz 0x2342FA2Cz 0x21ED3BA4z 0x21ED4420z 0x21ED1464z 0x228667F4z 0x2286711Cz 0x21ECAEE4z 0x233E7980z 0x233E7964z

000124: *Jun 30 15:11:11.617 CST: %SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from 0x23432024, alignment 0

Pool: Processor  Free: 859408  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "ARP Input", ipl= 0, pid= 29 -Traceback= 0x234117BCz 0x2342FA2Cz 0x21ED3BA4z 0x21ED4420z 0x21ED1464z 0x228667F4z 0x2286711Cz 0x21ECAEE4z 0x233E7980z 0x233E7964z

000125: *Jun 30 15:11:40.085 CST: %LICENSE-2-VLS_ERROR: 'VLSnotifyBirthAndExpiryEvents' failed with an error - rc = 25 - 'Error[25]: Could not understand the message received from license server on host "no-net".

Client-server version mismatch?

' -Traceback= 0x233E7980z 0x233E7964z

000126: *Jun 30 15:11:41.629 CST: %SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from 0x23432024, alignment 0

Pool: Processor  Free: 859408  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "IP Input", ipl= 0, pid= 117 -Traceback= 0x234117BCz 0x2342FA2Cz 0x21ED3BA4z 0x21ED4420z 0x21ED13CCz 0x22865440z 0x228654F0z 0x228751F4z 0x223DFFF4z 0x22856554z 0x22856964z 0x2283F8DCz 0x22857E90z 0x2283FF94z 0x228401D0z

[vmiller · Accepted solution]

Try an update to the IOS. If you have a service contract with Cisco I'd open a case.

(one of their first steps is to have you update the IOS).

[paolo bevilacqua · Accepted solution]

Agree with vmiller above, but also if you have crap like inspection, firewall, cbac, etc, remove it by all means.

[rahul.kukreja · Accepted solution]

Hi Jaime,

Here memory fragmentation is oberved.

it seems that although the total amount of memory is free, but the biggest block is not much which can be allocated to the requested process.

also you can have a look at sh mem stat command output, to  check further what is the size of largest free block being available  for Processor Memory Pool.

you can open TAC request to investigate further what is cuasing the leak providing output of show tech, show mem all totals.

Regards

[ciskonoypi]

Hi vmiller,

I will upgrade to the latest IOS version. Thanks!

Hi Paolo,

I've already done that but the problem still there. Thanks

Hi Rahul,

I will do that. Thanks

[bashira41]

Hi Jamie

I'm currently experiencing the same issue when using vpn on a cisco 1941 with IOS' 15.1(4)M3, 15.2(2)T & 15.2(2)T1

Can you advise which IOS you upgraded your Cisco 1941 to resolve the issue?

Kindest regards

Aleem