Core issue
When the ESMTP application inspection feature is enabled, the PIX Firewall allows mail servers to receive the fifteen commands, while it rejects all other commands and never sends them to the mail server.
Extended Simple Mail Transfer Protocol (ESMTP) application inspection adds support for eight extended SMTP commands, which include AUTH, EHLO, ETRN, HELP, SAML, SEND, SOML and VRFY. Along with support for seven RFC 821 commands (DATA, HELO, MAIL, NOOP, QUIT, RCPT, RSET), the security appliance supports a total of fifteen SMTP commands.
Resolution
ESMTP application inspection restricts the types of SMTP commands that can pass through the security appliance and adds monitoring capabilities to provide better protection against attacks.
ESMTP is an enhancement to the SMTP protocol and is similar in most respects to SMTP. The application inspection process for ESMTP is similar to that of SMTP application inspection, and includes support for SMTP sessions. Most commands used in an ESMTP session are the same as those used in an SMTP session. However, an ESMTP session is considerably faster and offers more options related to reliability and security (delivery status notification, for example).
The inspect esmtp command includes the functionality previously provided by the fixup protocol smtp command. It also provides additional support for some ESMTP commands.
When this feature is enabled, it only allows mail servers to receive the seven SMTP minimum-required commands and provides support for the eight ESMTP commands mentioned. These commands are described in Section 4.5.1 of RFC 821. All other commands are rejected by the PIX and never sent to the mail server.
Other ESMTP commands, such as ATRN, STARTLS, ONEX, VERB, CHUNKING, and private extensions are not supported. Unsupported commands are translated into Xs, which are rejected by the internal server. This results in an error message, such as 500 Command unknown: 'XXX'. The incomplete commands are discarded.
In order to allow the flow of mail traffic when such server implementations are used, issue the no form of the inspect esmtp command in class configuration mode to disable the feature.
For more details, refer to the Managing SMTP and Extended SMTP Inspection section of Applying Application Layer Protocol Inspection.
