Delivery expired (message too old)

[elias.winburne]

Hello,

I am getting a few errors each day related to the following type of message.  I have done some web searches, and a few pointed to Windows 2008 DNS as a possible problem.  However, our Ironports point to our external DNS which is BIND.   Here is the relevant data from message tracking.

SMTP delivery connection (DCID 163673330) opened from IronPort interface 10.193.13.181 to IP address 10.192.3.45 on port 25.

19 Jul 2011 17:20:01 (GMT -04:00)

(DCID 163673329) Message 492404228 to mbreeding@vcom.vt.edu bounced by destination server. Reason: 5.4.7 - Delivery expired (message too old) ('000', ['[Errno 61] Connection refused']) [(\'X-SBRS\', \'None\'), (\'X-HAT\', \'Group: RELAYLIST, Policy: $RELAYED\'), (\'X-VITAFiles\', \'Attached: image001.jpg, image002.jpg\'), (\'subject\', \'RE: orientation\'), (\'from\', \'"Evans, PATRICIA EVANS (DBHDS)" <Patricia.Evans@dbhds.virginia.gov>\'), (\'to\', \'"Breeding, Myra" <mbreeding@vcom.vt.edu>\')]

Anyone else see this?  Any ideas on how to fix it?

Thanks ahead of time!

Elias

[Christopher Smith]

Greetings Elias,

With the data you provided its a bit difficult to say if this is related to a DNS issue or not. A couple of things you may want to do here to further diagnose this.

If delivery attempts fail, this means the destination's mail server  did not accept a given message after 72 hours or 100 iterations of  consecutive tries (example).   This issue can be seen either incoming  (delivery to your internal groupware server) or outgoing (towards  another domain via the Internet).   While this does not necessarily  signal a problem with your local IronPort ESA, you can work around this  by extending the retransmission period or number of attempts, and  therefore increasing the chances for successful delivery.   These values  are controlled by the IronPort's Bounce Profiles.

Note: If you  wish to make the retransmission timeout more lax, increasing the value  may increase your chance for successful delivery.   The overall duration  (i.e., "maximum number of seconds") defines times in terms of seconds.  The default value is 259,200 seconds, or three days. We'd recommend  increasing this threshold to 345,600 seconds (four days) or 432,000  (five days) if necessary.

To edit the "Default" bounce profile via the GUI:

1) Go to Network > Bounce Profiles > select your Profile Name
2) set "Maximum Number of Retries" to change the number of times we attempt delivery
3) set "Maximum Time in Queue" to specify the total length of time to store the message for retransmission
4) set "Maximum Time to Wait per Message" to give the maximum interval between retries

Alternatively, we can use the CLI 'bounceconfig' command to set the same values.

You may also want to enable the domain debug logs to this host. This way you can see the full smtp conversation between your appliance and the remote host. This may provide some more detail on the previous failures before it gets to this state.

Additionally you may want to also consult the mail logs for the previous failures.

Below is some more information on setting up the domain debug logs.

The domain debug log is a system log  designed to record all SMTP traffic between a specific domain and an   Email Security Appliance (ESA)  for a finite number of sessions. This  log type can assist in troubleshooting issues that relate to a specific  recipient domain or host. Each session is recorded until the number of  session defined has been reached, at which time the log will stop  collecting data. You can stop domain debug before all sessions have been  recorded by deleting or editing the log subscription.

Configuration

Logs can be configured and created through the IronPort CLI   using the logconfig command or via the GUI.

To configure logs via the GUI, see the Advanced User Guide: Log Subscriptions .

Below is an example of creating a Domain Debug Log subscription using the CLI:.

example.run> logconfig

Currently configured logs:
1. "antispam" Type: "Anti-Spam Logs" Retrieval: FTP Poll
2. "antivirus" Type: "Anti-Virus Logs" Retrieval: FTP Poll
3. "asarchive" Type: "Anti-Spam Archive" Retrieval: FTP Poll
4. "avarchive" Type: "Anti-Virus Archive" Retrieval: FTP Poll
5. "bounces" Type: "Bounce Logs" Retrieval: FTP Poll
6. "cli_logs" Type: "CLI Audit Logs" Retrieval: FTP Poll
7. "error_logs" Type: "IronPort Text Mail Logs" Retrieval: FTP Poll
8. "euq_logs" Type: "IronPort Spam Quarantine Logs" Retrieval: FTP Poll
9. "euqgui_logs" Type: "IronPort Spam Quarantine GUI Logs" Retrieval: FTP Poll
10. "ftpd_logs" Type: "FTP Server Logs" Retrieval: FTP Poll
11. "gui_logs" Type: "HTTP Logs" Retrieval: FTP Poll
12. "mail_logs" Type: "IronPort Text Mail Logs" Retrieval: FTP Poll
13. "reportd_logs" Type: "Reporting Logs" Retrieval: FTP Poll
14. "reportqueryd_logs" Type: "Reporting Query Logs" Retrieval: FTP Poll
15. "scanning" Type: "Scanning Logs" Retrieval: FTP Poll
16. "sntpd_logs" Type: "NTP logs" Retrieval: FTP Poll
17. "status" Type: "Status Logs" Retrieval: FTP Poll
18. "system_logs" Type: "System Logs" Retrieval: FTP Poll
19. "updater_logs" Type: "Updater Logs" Retrieval: FTP Poll

Choose the operation you want to perform:
- NEW - Create a new log.
- EDIT - Modify a log subscription.
- DELETE - Remove a log subscription.
- SETUP - General settings.
- LOGHEADERS - Configure headers to log.
- HOSTKEYCONFIG - Configure SSH host keys.
[]> new

Choose the log file type for this subscription:
1. IronPort Text Mail Logs
2. qmail Format Mail Logs
3. Delivery Logs
4. Bounce Logs
5. Status Logs
6. Domain Debug Logs
7. Injection Debug Logs
8. System Logs
9. CLI Audit Logs
10. FTP Server Logs
11. HTTP Logs
12. NTP logs
13. LDAP Debug Logs
14. Anti-Virus Logs
15. Anti-Virus Archive
16. Scanning Logs
17. IronPort Spam Quarantine Logs
18. IronPort Spam Quarantine GUI Logs
19. Reporting Logs
20. Reporting Query Logs
21. Updater Logs
[1]> 6

Please enter the name for the log:
[]> debug_example

Enter the name of the domain for which you want to record debug information.
[]> example.com

Please enter the number of SMTP sessions you want to record for this domain.
[1]> 8

Choose the method to retrieve the logs.
1. FTP Poll
2. FTP Push
3. SCP Push
4. Syslog Push
[1]>

Filename to use for log files:
[example.com.text]>

Please enter the maximum file size:
[10485760]>

Please enter the maximum number of files:
[10]>

Currently configured logs:
1. "antispam" Type: "Anti-Spam Logs" Retrieval: FTP Poll
2. "antivirus" Type: "Anti-Virus Logs" Retrieval: FTP Poll
3. "asarchive" Type: "Anti-Spam Archive" Retrieval: FTP Poll
4. "avarchive" Type: "Anti-Virus Archive" Retrieval: FTP Poll
5. "bounces" Type: "Bounce Logs" Retrieval: FTP Poll
6. "cli_logs" Type: "CLI Audit Logs" Retrieval: FTP Poll
7. "debug_example" Type: "Domain Debug Logs" Retrieval: FTP Poll
8. "error_logs" Type: "IronPort Text Mail Logs" Retrieval: FTP Poll
9. "euq_logs" Type: "IronPort Spam Quarantine Logs" Retrieval: FTP Poll
10. "euqgui_logs" Type: "IronPort Spam Quarantine GUI Logs" Retrieval: FTP Poll
11. "ftpd_logs" Type: "FTP Server Logs" Retrieval: FTP Poll
12. "gui_logs" Type: "HTTP Logs" Retrieval: FTP Poll
13. "mail_logs" Type: "IronPort Text Mail Logs" Retrieval: FTP Poll
14. "reportd_logs" Type: "Reporting Logs" Retrieval: FTP Poll
15. "reportqueryd_logs" Type: "Reporting Query Logs" Retrieval: FTP Poll
16. "scanning" Type: "Scanning Logs" Retrieval: FTP Poll
17. "sntpd_logs" Type: "NTP logs" Retrieval: FTP Poll
18. "status" Type: "Status Logs" Retrieval: FTP Poll
19. "system_logs" Type: "System Logs" Retrieval: FTP Poll
20. "updater_logs" Type: "Updater Logs" Retrieval: FTP Poll

Choose the operation you want to perform:
- NEW - Create a new log.
- EDIT - Modify a log subscription.
- DELETE - Remove a log subscription.
- SETUP - General settings.
- LOGHEADERS - Configure headers to log.
- HOSTKEYCONFIG - Configure SSH host keys.
[]>

example.run> commit

Domain Debug Log

Below  is an example of a Domain Debug Log looks like when the IronPort  appliance delivers a message to the recipient domain: "example.com". 

Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '220 ESmtp mail.example.com ESMTP service ready'
Tue Mar 22 16:52:07 2005 Info: 411 Sent: 'EHLO ironport.com'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250-mail.example.com'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250-8BITMIME'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250-SIZE 31981568'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250 PIPELINING'
Tue Mar 22 16:52:07 2005 Info: 411 Sent: 'MAIL FROM:<user@ironport.com>'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250 sender <user@ironport.com> ok'
Tue Mar 22 16:52:07 2005 Info: 411 Sent: 'RCPT TO:<test@example.com>'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250 recipient <test@example.com> ok'
Tue Mar 22 16:52:07 2005 Info: 411 Sent: 'DATA'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '354 go ahead'
Tue  Mar 22 16:52:07 2005 Info: 411 Sent: 'Received: from unknown  (HELO)(10.250.7.164)rn by ironport.com with SMTP; 22 Mar 2005 16:52:08  -0800rn'
Tue Mar 22 16:52:07 2005 Info: 411 Sent: 'Message-ID:  <000d01c52f43$48dacba0$a407fa0a@ironport.com>rnFrom: "User"  <user@ironport.com>rnTo:<test@example.com>rn  Subject:TestrnDate:Tue,22Mar200516:57:28-0800rnMIME-Version:1.0rn
Content-Type:multipart/alternative;rntboundary="----=_NextPart_000_000A_01C52F00.3AA3B580"rnX-Priority:  3rnX-MSMail-Priority: Normalrn X-Mailer: Microsoft Outlook Express  6.00.2900.2180rnX-MimeOLE: Produced ByMicrosoft  MimeOLEV6.00.2900.2180rnrnThis is a multi-part  messageinMIMEformat.rnrn------=_NextPart_000_000A_01C52F00.3AA3B580rnContent-Type:text/plain;rntcharset=  "iso-8859-1"rnContent-Transfer-Encoding: quoted-printablernrnThis  isthebodyofthemail.rnThisisadisclaimer.rnrn------=_NextPart_000_000A_01C52F00.3AA3B580rnContent-Type:text/html;rntcharset=   "iso-8859-1"rnContent-Transfer-Encoding:quoted-printablernrnrnrnrnrnrnrnrn

This is the  body of thernmail.

 This is a  disclaimer.rn  

rnrn------=_NextPart_000_000A_01C52F00.3AA3B580--rn'
Tue Mar 22 16:52:07 2005 Info: 411 Sent: '.rn'
Tue Mar 22 16:52:07 2005 Info: 411 Rcvd: '250 ok dirdel'
Tue Mar 22 16:52:12 2005 Info: 411 Sent: 'QUIT'
Tue Mar 22 16:52:12 2005 Info: 411 Rcvd: '221 mail.example.com'

Christopher C Smith

CSE
Cisco IronPort Customer Support 

[Stephan Bayer]

I noticed the same issue on a customer ESA and wanted to share my findings. 

You will likely see a high Active Recipient count when looking at the ESA GUI > Monitor > Delivery Status or via cli typing >tophosts then Active Recipients.

First locate the Tracking information. Example: 

(Machine myesa)> grep "661943" mail_logs
...

Thu Oct 18 22:31:42 2018 Info: MID 589979 Message-ID '<685.507661943.201810181731408948659.0007476837@email.amf.com>'
Sun Oct 21 17:10:16 2018 Info: Start MID 661943 ICID 1000903
Sun Oct 21 17:10:17 2018 Info: MID 661943 Message-ID '<207965819.2724.1540123818426.JavaMail.SVCE0BI@E1SCG5AP01>'
Sun Oct 21 17:10:17 2018 Info: MID 661943 Subject '**Employee Changes Report **'
Sun Oct 21 17:10:17 2018 Info: MID 661943 ready 8247 bytes from ...
Sun Oct 21 17:10:17 2018 Info: MID 661943 Custom Log Entry: Matched UltiPro_Reports filter
Sun Oct 21 17:10:17 2018 Info: MID 661943 matched all recipients for per-recipient policy DEFAULT in the inbound table
Sun Oct 21 17:10:17 2018 Info: MID 661943 interim AV verdict using Sophos CLEAN
Sun Oct 21 17:10:17 2018 Info: MID 661943 AMP file reputation verdict : UNKNOWN
Sun Oct 21 17:10:17 2018 Info: MID 661943 Custom Log Entry: Header has been stripped due to requested read receipt
Sun Oct 21 17:10:17 2018 Info: MID 661943 attachment 'Employee=20Changes=20Report=20-=20Scheduled.xlsx'
Sun Oct 21 17:10:17 2018 Info: MID 661944 was generated based on MID 661943 by duplicate-quarantine filter 'ultipro_troubleshooting'
Sun Oct 21 17:10:17 2018 Info: MID 661943 Outbreak Filters: verdict negative
Sun Oct 21 17:10:17 2018 Info: MID 661943 queued for delivery
Wed Oct 24 17:42:25 2018 Info: Bounced: DCID 153567 MID 661943 to RID 0 - Bounced by destination server with response: 5.4.7 - Delivery expired (message too old) ('000', ['timeout'])
Wed Oct 24 17:42:25 2018 Info: MID 768246 was generated for bounce of MID 661943
Wed Oct 24 17:42:25 2018 Info: Message finished MID 661943 done

Locate the DCID portion and take note of the number. 

Via CLI again grep for that DCID 

(Machine myesa)> grep "DCID 153567" mail_logs

Wed Oct 24 17:42:25 2018 Info: Connection Error: DCID 153567 domain: [10.130.187.243] IP: 10.130.187.243 port: 25 details: timeout interface: 216.71.148.107 reason: connection timed out
Wed Oct 24 17:42:25 2018 Info: Bounced: DCID 153567 MID 661943 to RID 0 - Bounced by destination server with response: 5.4.7 - Delivery expired (message too old) ('000', ['timeout'])

This led me to locate the IP address in a message filter from the DCID in the configuration file. 

UltiPro_Reports: if (sendergroup == "UltiPro_Exception") {
alt-mailhost ("[10.130.187.243]");
log-entry("Matched UltiPro_Reports filter");

skip-filters();
}

Also performing a CLI telnet test confirmed the connectivity issue to this IP. 

(Machine myesa)> telnet 10.130.187.243 25

Trying 10.130.187.243...

[connection does not open indicating routing issue]

Deactivating this filter fixed the issue! I know 7 years later...Hope it helps others!