topic Re: Cisco ASA 5512-X ""Lua runtime: not enough memory" in Network Security

Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Tue, 14 Jan 2025 15:01:54 GMT

Good day everyone.

Yesterday, I was working on a Cisco ASA 5512-X Firewall that was being used as a VPN Firewall when suddenly I was disconnected from the ASDM. My SSH session from Putty was closed, and I am unable to get back into any of them.

I took a console cable directly connected to the Firewall. I was greeted with the usual log message but with the message "Lua runtime: not enough memory" which kept interrupting my inputs while I was troubleshooting.

when I used the "show memory" command this is what caught my attention

Free memory: 117768378 bytes (7%)
Used memory: 1628998880 bytes (93%)
------------- ------------------
Total memory: 1746767258 bytes (100%)

i assume the high used memory was why my remote sessions were cut off so abruptly and the reason why i can't get back in using ASDM or SSH.

Strangely enough, i was still able to connect and use the VPN without issues, others were still able to connect and use the VPN as well.

I have a feeling i am dealing with the Bug labeled CSCto76775 and i am currently in the process of backing up the running-config and submitting a downtime request form so i can Power-Cycle the device

I have uploaded two text files for reference, one is the "show version" of the device in question and another one is an abbreviated output of the "show memory detail"

My question is, is there another way i can free up the memory so i can re-gain access to the ASDM and re-establish SSH connections or is Power Cycling the only way?

Thank you for your time

Any help would be appreciated.

-Aaron

PS: if this is in the wrong area, please let me know so i can re-ask the question in the correct location.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

MHM Cisco World — Tue, 14 Jan 2025 15:36:28 GMT

I will check it

Thanks for waiting

MHM

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

MHM Cisco World — Tue, 14 Jan 2025 15:46:29 GMT

Show proc cpu-hog

Show proc mem

Show mem detail

Show perfmon

Share above

Thanks

MHM

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

Mark Elsen — Tue, 14 Jan 2025 17:29:30 GMT

- FYI : https://bst.cloudapps.cisco.com/bugsearch/bug/CSCta09265
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCto76775

M.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Tue, 14 Jan 2025 17:48:01 GMT

MHM here are the outputs of the commands you wanted me to run

-Aaron

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Tue, 14 Jan 2025 17:52:46 GMT

Thank you for the links to those bug reports M.

I do see that alot of my device's symptoms seem to corallite with the bug reports.

it looks like the only solution, as far as i could see, is to power-cycle the firewall in question.

What i don't understand is with such a high memory load everything is running properly the Firewall and the VPN. i would assume that i would start to see slowdowns or reports of slow and or failing connections to the VPN.

is there something im missing or not understanding?

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Tue, 14 Jan 2025 18:35:34 GMT

Quick update

I am unable to backup the running-config to USB or TFTP i keep getting the error "No Memory"

i assume it has to do with the high memory usage.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

Mark Elsen — Tue, 14 Jan 2025 19:11:38 GMT

- Can you still show it with show running-config ?
In that case you could for instance connect with PuTTY and enable logging of the session.
The above command can then be used to have the output of the running configuration , being preserved
in the PuTTY logfile too (e.g.)

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Tue, 14 Jan 2025 20:25:54 GMT

I did not think to do it that way, thank you for the suggestion.

My only issue is the repeating "Lua runtime: not enough memory" that will be happening but that can be easily edited out.

For extra peace of mind, are there any commands you can think of that I should run to make sure that once i power-cycle the device it will come back up.

i don't plan on removing any power supplies, once i get the approval for the window of time i will just connect via console cable and just do the reload command.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

Mark Elsen — Wed, 15 Jan 2025 09:46:13 GMT

@aaron-rousch wrote : >....For extra peace of mind, are there any commands you can think of that I should run to make sure that once i power-cycle the device it will come back up.
Not that I know off , usually when there is a problem concerning resources it comes down to 'just power-cycle or reboot' ; the earlier method I mentioned was kind of an emergency saver for the running configuration if needed.

- Often useful on ASA platforms is the https://software.cisco.com/download/home/286311499/type/286312309/os
CLI Analyzer (needs to be downloaded first). It can run diagnostics on the ASA , see buttons at the top.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Wed, 15 Jan 2025 14:49:36 GMT

Thank you for the CLI Analyzer, i have downloaded it and will start looking at the documentation for it.

I have one more question, for you Sir. do you know of a way to suppress the "Lua runtime: not enough memory" messages so that i can get an uninterrupted output of the show running-config?

Thank you again

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

Mark Elsen — Wed, 15 Jan 2025 15:10:29 GMT

- No , not if the device is in that current state ; your (only) options are to reboot and check if that can help and also look
at the most recent (advisory) software version for your particular model. Upgrading becomes advised especially
if you are currently using an older release ,

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Wed, 15 Jan 2025 15:18:40 GMT

Understood.

Thank you for all the help, i am currently in the process of hopefully getting a Cisco support contract with the company i work for.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

MHM Cisco World — Wed, 15 Jan 2025 15:59:45 GMT

You share output yesterday' give me some time to check it.

MHM

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Wed, 15 Jan 2025 16:02:22 GMT

Okay, thank you MHM i will remove the "Accept as Solution" and keep the thread open untill i hear back from you.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

JesseSmith8517 — Tue, 04 Feb 2025 18:44:37 GMT

I just saw it in code version 9.20.3.7 - Cisco ASAv10

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

DG-120 — Tue, 11 Feb 2025 12:16:17 GMT

Also seen this issue on 9.20.3.7 this week the same as the previous poster (and also running ASAv10's) - We upgraded from 9.18.4.24 a couple of weeks ago in the hope it would resolve this issue, but I guess it hasn't.

Our primary device (lets call it ASA1) after the firmware update was at 25% free memory. This has gradually dropped pretty continuously since restart and finally dropped below 10% free today. It may reclaim 1% occasionally, but it's basically a continuous downward trend from the moment the device is made active, regardless of current workload.

Below 10% free memory on the active device seems to be the point at which the 'Lua runtime' error begins to occur and new AnyConnect/Putty sessions begin failing.

The secondary device (ASA2) has been sat happily at around 23-24% free memory for the past couple of weeks. I've made ASA2 the active device today, it's currently still sat at 23% free. I restarted ASA1 and that is now back at 25% free and in standby.

I imagine over the next couple of weeks I will see ASA2 now gradually lose free memory, never regaining any regardless of it's workload, and eventually when it creeps towards 10% I will be forced to failover to ASA1 and start the process over again.

It's a simple 'fix' but it's also extremely frustrating.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

aaron-rousch — Tue, 11 Feb 2025 14:29:00 GMT

So, is it a continuous problem regardless of updates? That's disheartening. I've started looking at ways to reduce the firewall's memory usage, remove unused network objects, and simplify ACLs and NAT rules, hoping that would help.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

JesseSmith8517 — Tue, 11 Feb 2025 14:36:13 GMT

Good morning,

I want to add on some new information being tried out at my company. My coworker did some digging, using a tool on the 'show tech' output. He noticed evidence of a 'password spray attack' - I'm hoping that I entering the correct term used. After consulting with TAC, we're doing the following:
upgrading our ASAvs to the recommended code per train
implementing three additional threat-detection commands.

Here are the command being added:

threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
threat-detection service invalid-vpn-access
threat-detection service remote-access-authentication hold-down 10 threshold 10
threat-detection service remote-access-client-initiations hold-down 10 threshold 20

So far, we have about 81 ASAvs on the 9.20.3.x code that have the above command implemented. I'm monitoring them daily to see if any of them experience any of the various memory issues.

I'll try to to keep posting if I learn anything new.

Re: Cisco ASA 5512-X ""Lua runtime: not enough memory"

DG-120 — Tue, 11 Feb 2025 14:40:12 GMT

As far as I can tell it seems continuous regardless of load on the device or config. Our config hasn't changed much, just the firmware versions. The last 2 we have used both seem to have this issue. A few months ago the throughput on the devices was mostly the same, the firmware version is the only real change.

Just something I want to rule out, for other people seeing this same issue... do you happen to have any IKEv1/v2 VPN's to other devices, and if so what are they connecting to? Our ASA has a number of IKEv2 connections to Draytek Vigor devices at our other locations. The only other thing I can think of that would have changed is the firmware version running on those devices. It should have zero impact, but it's just a thought I would like to rule out.

Anyone else with this issue have any connections to Draytek Vigor's that have had firmware updates in the past couple of months?