Cisco Cisco Secure Email (Cloud) Gateway vulnerable to SMTP smuggling

[filiadata]

Full read: https://sec-consult.com/blog/detail/smtp-smuggling-spoofing-e-mails-worldwide/

I short, the end-of-data command in SMTP is specified as a dot surrounded by carriage return and line feed: <CR><LF>.<CR><LF>. However the Cisco Secure Email Gateways apparently also treat <CR>.<CR> or <LF>.<LF> as end-of-data and automatically try to "repair" those line breaks by converting them into <CR><LF>. This allows a sender to craft a special email message which suddenly is split into multiple messages as soon as it traverses a Cisco Secure Email Gateway. The contents of all parts of the message are controlled by the sender and are successfully authenticated by the sending MTA. To sum it up, this allows attackers to create email messages from all kind of foreign domains which the Cisco Secure Email Gateway and all subsequent email servers will see as successfully DMARC authenticated, although they are actually completely fake.

Unfortunately it seems Cisco is the only vendor that treats this behaviour as a feature and not a bug, so all customers of both on-premise Ironport and Cisco Cloud Email Security need to take action by themselves and manually disable the fixup of line breaks on their systems under "Network > Listeners > Select the inbound listener > CR and LF Handling". Make sure any option EXCEPT the default "Clean messages" is selected.

[blopezde]

Hello.

"Cisco has assessed Cisco Secure Email Gateway and Cisco Cloud Secure Email and concluded that even if, depending on the configuration, the products can accept emails containing the sanctioned characters, no security checks will be bypassed for either the first email or the subsequently "smuggled" one, so the exploitation of the issue does not have any detrimental consequence on email security."

The above text is from document: https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwh10142

[mkoch]

is there any offcial recommendation/guidance? Cisco is really dropping the ball on this one, there is not even a security advisory...

Today i got recommendations from BSI and CERT-EU to change to listener settings to "ALLOW" which is deprecated according to the ESA GUI??? now what?

[lkgs]

There still is an active advisory from EU-Cert CERT-EU - SMTP Smuggling Vulnerability in CISCO Secure Email Gateway (europa.eu) mentioning this flaw (no working SPF, DMARC, DKIM check for the smuggled mail) if having the "clean" option under "Network > Listeners > Select the inbound listener > CR and LF Handling".

We made sure ourself to set it to Reject, since Allow is deprecated and "Clean" seems to be the default value for new installations. From my understanding the issue is with the "cleaning" process in ESA. 

It is unsettling to have these opposing claims. Cisco saying: Yes we create two messages ("depending on the configuration") but there is no security risk. And on the other side the security researchers and CERTs who claim that the "clean" options allow attackers to successfully spoof mails, they won't claim it if they hadn't tested it. So maybe Cisco tested something slightly different? As I read there are different approaches with combining the dot character and CR/LF and most products are only affected by one combination. 

Can't you say your customers exactly what happens with these smuggling attempts in the ESA message pipeline? 

And what does "Customer can choose "Reject messages with bare CR or LF characters" for preventing smtp smuggling scenarios, this setting may limit interoperability with some third party SMTP implementations, so customers are advised to evaluate the best setting based on the specifics of their environment." mean?

That we should set it to reject but live with the consequences? Is the system only "secure" with this workaround or it is secure even with the "Clean messages" option? 

[tminchin]

we did some testing and found that the anti-spam filter seemed to bin attempts to smuggle (not great but not the worse outcome)

[filiadata]

The only setting which is vulnerable to the SMTP smuggling attack is "Clean messages of bare CR and LF characters", which is the default setting. With this setting, the Cisco Cisco Secure Email (Cloud) Gateway will "repair" broken line endings with CRLF, and thereby turn the smuggled email message into a new, second email message.

The setting "Allow messages with bare CR or LF characters (DEPRECATED)" will simply pass on messages with broken line endings to the next email server to deal with them, whereas "Reject messages with bare CR or LF characters" will reject all such messages with the SMTP error code "554 message body contains illegal bare CR/LF characters." Every rejection will also be logged to mail_logs as "Receiving Failed: Illegally formed message body (bare CR or LF characters)".

Unfortunately there is not setting "Reject messages with bare CR or LF characters at end of message", which would be enough to prevent the SMTP smuggling attack. Instead the setting "Reject messages with bare CR or LF characters" will cause the Cisco Cisco Secure Email (Cloud) Gateway to reject all messages which contain bare CR or LF anywhere in the email. This will cause false positives because there are email servers out there which produce such broken messages.

Ciscos bug report mentioned above reads "no security checks will be bypassed for either the first email or the subsequently smuggled one". That's not true, the attack does bypass security checks, namely the DMARC check.
Example: An email from a Hotmail account which contains a smuggled message from support@microsoft.com is received by the Cisco Secure Email (Cloud) Gateway. After "cleaning" the broken line endings, we will end up with a second email message from support@microsoft.com, sent by an unauthorized sender, which was accepted by the Cisco Secure Email (Cloud) Gateway. With a RFC conforming behaviour this would not have been possible, as the email would have been rejected due to the DMARC policy of microsoft.com when being sent by a random server (and Microsofts own servers do not allow customers to send emails with a From address support@microsoft.com)

[mkoch]

I changed the setting to "reject" und now we got dropped/rejected legitimate eMails.

ALLOW is deprecated and passes responsiblity to the next server. CLEAN allows DMARC bypass and REJECT rejects legitmate emails. And no response/guidance/recommendation/response frm Cisco...

[cdion]

Yes, reject doesnt seems a valid solution today.

The only solution that might work (not fully tested yet) is to apply a content-filter with a  body-match condition based on dictionary with 3 simple regexs (matching at least one CR or LF on any side of the dot in the middle) :

(\n|\r)\.\r\n

\r\n\.(\n|\r)

(\n|\r)\.(\n|\r)

An action might be to send messages to Quarantine for further investigation.

As alternative, I tried a solution with regex matching the basic headers "From: + To: + Subject:" in message-body,  but i have a big number of false positives due to "reply" behavior on all MUAs out there (seems that always embedding the headers of replied message in the new message body)

interesting topic.

[SR_O]

I've opened tickets with one of our partners and they checked this topic with Cisco and gave the below feedback. Unfortunately not an official statement from Cisco

[...]

1. Cisco’s default configuration is clean. this means “setting may limit interoperability with some third party SMTP implementations, so customers are advised to evaluate the best setting based on the specifics of their environment”. That you need to discuss internally with your colleagues whether to reject or allow, or to keep the default unchanged. Because each company’s environment is different and their needs are different, so this is still the case. You decide for yourself.
2. Setting reject there should be relatively few emails with questions like this, unless there is a malicious attack using such emails.

3. Setting to Allow means the ESA will be processing and delivering a message that is not conforming to the RFC when such characters are present, which can lead to bounces and potentially bounce scatter spam if the next hop does not accept the message.

Clean is the recommended setting by default currently the bug is "terminated
Terminated means that the bug is invalid, and everything mentioned as a workaround is - as a consequence - invalid too.

This bug is not affecting the ESA and is terminated as such

[...]

[Michael.Terlien]

@SR_O wrote

3. Setting to Allow means the ESA will be processing and delivering a message that is not conforming to the RFC when such characters are present, which can lead to bounces and potentially bounce scatter spam if the next hop does not accept the message.

---

That is really just a problem created by the original sender though. It is good that Cisco has a setting in place to clean these non RFC conform e-mails. But if that settings means it also facilitates SMTP Smuggling, I would rather just deliver it as is to the mail server.

Either way I do feel like Cisco could do more here. Instead of playing the 'it's not a bug, it's a feature' card, and leaving it up to the customer on what to do. They could update this setting to better detect SMTP Smuggling attempts and block them.
We choose to change out environments to "Allow" as this could better give destination servers a chance to block the smuggling attempts. But I secretly hope Cisco is working on something in the background to combat SMTP Smuggling. Only time will tell...