topic Re: DDOS and "Error processing payload: Payload ID: 1" in Network Security

DDOS and "Error processing payload: Payload ID: 1"

rstockum — Wed, 10 Jul 2024 11:26:01 GMT

Hello
A few days ago we had a DDOS attack against our ASA 5545 firewall.
Within a time window of about 15 minutes, almost 1,000,000 connections were established from slightly more than 10,000 Bot-Net IP addresses, which led to a high CPU load on the firewall and overflow of the session tables immediately after the attack began.
The effect was sporadic connection and call drops over a period of about an hour.
The FW log file contained 965,603 entries with the following messages:

"%ASA-3-713048: IP = <different unknown IP addresses>, Error processing payload: Payload ID: 1"

Is this a known vulnerability?
Has anyone had any experience with this?

Thanks in advance!

Re: DDOS and "Error processing payload: Payload ID: 1"

Marius Gunnerud — Wed, 10 Jul 2024 12:21:28 GMT

What messages come before and after the "Error processing payload..." message?

You say the connections were established, were they established to a server / IP behind the firewall? Usually if this was to an IP behind the firewall you would only see allowed or drop logs. So, I would almost think that this is traffic destined for the firewall and possibly AnyConnect / remote access VPN.

Without more context to the connection it will be difficult to formulate a hypothesis on if it is a vulnerability or something else.

if this is time-sensitive then I suggest opening a TAC case and request immediate assistance.

Re: DDOS and "Error processing payload: Payload ID: 1"

Mark Elsen — Wed, 10 Jul 2024 12:25:47 GMT

>...Is this a known vulnerability?
- It depend what you mean by vulnerability as a DDOS attack is always external
It is advised to upgrade the ASA to the latest advisory software version , if applicable
for solid firewall performance and handling of atttacks ,

Re: DDOS and "Error processing payload: Payload ID: 1"

rstockum — Wed, 10 Jul 2024 13:40:30 GMT

Hi Marius,

thx for your reply!
Unfortunately, I do not see any noticeable event in the log before and after these messages. Everything looks completely normal - just the usual acl matches and VPN events.

These messages are the only usable information.
I agree with you that the target was the firewall itself and not a target behind the firewall. Therefore, only the source IP addresses are visible. And yes, the ASA-3-713048 event is related to VPN connections.

I suspect that the ASA is trying to assign the incoming payload packets to an active VPN connection, but cannot find a match.
Nevertheless, the FW has to process these excessive payload packets, which leads to the DDOS effect.

The Begin of this messages is 10:29:35 and the End at 10:42:20 - In this short time window, 965603 error processing payload messages with 11498 different IP addresses from 186 different networks were received.
Most of these IP addresses were addresses from CN.

I therefore assume that this was a dedicated targeted attack and not the result of a faulty VPN configuration.

Recently, there have also been an increasing number of brute force attacks on the authentication service of the firewall, which are already known in the community.

However, the type of attack was new to me and I could not find anything about it on the Internet.

Re: DDOS and "Error processing payload: Payload ID: 1"

Marius Gunnerud — Wed, 10 Jul 2024 14:01:05 GMT

If the source IPs are from different IPs as you are mentioning, then yes this is not a misconfiguration.

The unfortunate thing about these types of attacks is that they are very difficult if not impossible to prevent. This being that VPN connections will be coming in from random public IPs and if you block that...well then no one can connect anyway.

What you can do is tighten up which countries can connect to the VPN by blocking country IP address space in a control plane ACL. So for example, if you are not expecting users to connect via VPN from Russia, China, Korea, etc. you can create a control plane ACL containing these IP address spaces and block them. This will not prevent a DDoS but i can limit the attack surface.

Re: DDOS and "Error processing payload: Payload ID: 1"

MHM Cisco World — Wed, 10 Jul 2024 21:08:05 GMT

try capture traffic and see it source destination protocol ID
if you can share capture here

MHM

Re: DDOS and "Error processing payload: Payload ID: 1"

rstockum — Thu, 11 Jul 2024 05:29:45 GMT

Hi Marius,
Yes, that's exactly what I do! I have defined a CPLANE ACL for the bad geolocations and also shunned all these IP addresses.
The strange thing is that the attack stopped by itself after about 15 minutes, without my intervention.
Thanks you!

Re: DDOS and "Error processing payload: Payload ID: 1"

rstockum — Thu, 11 Jul 2024 05:34:42 GMT

Hi - So far there have been no further attacks of this kind.
But if they happen again and there is enough time, I will capture the traffic to get detailed information.