Cisco Iron port mail queued for delivery(only for office 365)

[AbcKdc31268]

Everything was working fine until today morning when mails to Office 365 domain did not got delivered. Upon investigation we found that every thing is working fine except for certain domain whose email are hosted of Office 365(**.mail.protection.outlook.com) the emailgateway stuck on line mail delivery queued. It does not even establishes SMTP connection to recipient domain. Tried to forcefully deliver all  mail using deliverall and delivernow command but nothing works. Checked for blacklisting of our email gateway IP in office 365 but it isn't blacklisted. Contacted the recipient side and they told us that SMTP connection has not been established to their mail server(office 365). Checked for network errors but there are not any. Viewed Delivery Status in GUI and the domains where emails are not sent has host status set as  Down. Tried to telnet form iron port to recipient mx and its is successful.

Update: Upon further investigation found the following error in mail log:

Info: Connection Error: DCID 101607 domain: xyz.com IP: x.x.x.x port: 25 details: [Errno 0] Error interface: y.y.y.y reason: network error

But checked connection to recipient mail server from email gateway and everything is fine telnet,ping,trace etc.

Update: Upon checking the domain debug logs to that particular domain

Rcvd: '220 C01FT018.mail.protection.outlook.com Microsoft ESMTP MAIL Service ready at xxxx
Sent: 'EHLO yy.yy.com'
Rcvd: '250-PU1APC01FT018.mail.protection.outlook.com Hello [x.x.x.x]'
Rcvd: '250-SIZE 157286400'
Rcvd: '250-PIPELINING'
Rcvd: '250-DSN'
Rcvd: '250-ENHANCEDSTATUSCODES'
Rcvd: '250-STARTTLS'
Rcvd: '250-8BITMIME'
Rcvd: '250-BINARYMIME'
Rcvd: '250-CHUNKING'
Rcvd: '250 SMTPUTF8'
Sent: 'STARTTLS'
Rcvd: '220 2.0.0 SMTP server ready'

[SaschaHoppe0579]

Hi,

any firewall with inspection in front of the ironport?
We had the issue with our firewall that blocks access to some 'bad IP ranges' not mail specific but provider / country specific.

Other case was with our the public DNS server we used (9.9.9.9) that blocked some addresses too.

a friend had issues with 1.1.1.1 cloudflare dns servers too.

regards 

Sascha

[AbcKdc31268]

Thanks for the reply Sascha. Tried changing the DNS server. But no luck. There is not any kind of firewall policy inspection for the email gateway. All the smtp connection to the domain are successfully established. But Email Gateway is not able to send the data(I think but not sure). 

Rcvd: '220 C01FT018.mail.protection.outlook.com Microsoft ESMTP MAIL Service ready at xxxx
Sent: 'EHLO yy.yy.com'
Rcvd: '250-PU1APC01FT018.mail.protection.outlook.com Hello [x.x.x.x]'
Rcvd: '250-SIZE 157286400'
Rcvd: '250-PIPELINING'
Rcvd: '250-DSN'
Rcvd: '250-ENHANCEDSTATUSCODES'
Rcvd: '250-STARTTLS'
Rcvd: '250-8BITMIME'
Rcvd: '250-BINARYMIME'
Rcvd: '250-CHUNKING'
Rcvd: '250 SMTPUTF8'
Sent: 'STARTTLS'
Rcvd: '220 2.0.0 SMTP server ready'

[svgeorgi]

Looks like a TLS related error - you can check the TLS settings on the ESA's and O365's sides - they should be configured in a compatible way. May want to perform a packet capture to really capture the details of the connection error seen on the ESA.

[AbcKdc31268]

Thanks for the reply @svgeorgi. Yes it was the problem with TLS connection being established from Iron Port to Office 365. The mail works if TLS is disabled. But did not find any thing on our side that is preventing a successful TLS connection. We have not received any response from Office 365 side regarding the issue. So, currently we have disabled TLS connection on Cisco Iron Port.

[svgeorgi]

Glad to hear that it got narrowed down. Check your settings under ESA's GUI>System Administration>SSL Configuration. Also check your certificates under GUI>Network>Certificates, and which certificate is selected for outgoing connections under GUI>Mail Policies>Destination Controls.

If everything is looking good, you will have to involve Microsoft's support as well to check on the O365's side.