FYI: Potential Windows 2008 communication problem

[bpoyner_ironport]

This is mostly a heads-up and a request for anybody that can replicate this problem. We've opened a support request with IronPort and they are looking into it. I'm hoping this might help a few people that have been banging their heads trying to figure out what's going on. :?

We've identified a problem where our C350 is unable to communicate with any Windows 2008 server if specific firewalls are in the middle. This includes some very common firewalls such as Cisco PIX. I don't believe it matters where the incompatible firewall lies -- on your network or a remote network. You need "W2K8<->Incompatible Firewall<->IronPort appliance" in this scenario. Any other configuration seems to work, including Windows 2008 on the same LAN as the IronPort appliance.

It seems to be directly related to TCP Window Scaling that Microsoft has enabled by default in Windows 2008/Vista. When a connection is attempted the TCP handshake completes but all following packets sent by the IronPort appliance are silently dropped by the firewall. If you disable Window Scaling on the Windows 2008 server, the connection works as expected.

Microsoft has a list of incompatible firewalls and steps to disable Windows Scaling here. Cisco also makes note of this problem in a troubleshooting document.

[steven_geerts]

Hi bpoyner,

What protocols are you talking about?
just "plain" SMTP trafic, LDAP traffic, management traffic? of maybe a combination of those?

I'm interested in some more details.

Steven

[bpoyner_ironport]

What protocols are you talking about?
just "plain" SMTP trafic, LDAP traffic, management traffic? of maybe a combination of those? 

Good question, as far as I know it only applies to any TCP traffic originating on the Windows 2008 server, passing through an incompatible firewall, and terminating at the C350. If the TCP traffic originates with the C350 destined for a Windows 2008 server (even through said firewall), we don't see any problem. I think this is because the C350 does not request TCP Window Scaling.

Originally this appeared as a problem where one specific school using Windows 2008 & Exchange 2007 reported that mail destined for us was getting stuck in their queue, but it turned out they also couldn't view the IronPort end user quarantine or any other public port.

We did some testing in house and were easily able to replicate the problem, TCP connections would only handshake and nothing more if an incompatible firewall was in the middle. Once we figured out what was going on, the work-around in the Microsoft technet article restored all communication. We could make the problem appear and go away at will by just toggling the setting.