Duplicate ACK's on network...

[netguyz08]

When transferring a 2GB file between two Windows computers across cable connections, connected by an IPSec tunnel, I see a ton of duplicate ACK's show up from the source IP to the destination IP.

There are no retransmissions, or errors coming up (but a ton of TCP window updates).

Why I posted it to the LAN side, is that the there are Cisco (a 3750G and a 3550) switches involved on either side too, that had a routing table setup to send private IP addresses to the right place. Specifically, the private IPs for the IPSec tunnel go to the firewall and out the tunnel, and the remaining ones go across an MPLS network.

On one side, there is a stack of 3750G's that has the routing table, and a lone 3750G which routes 0.0.0.0 to the IP of the 3750G stack. There is also a Netgear switch in the mix whose configuration is unknown. The main firewall connection was unfortunately plugged into the lone 3750G so it was moved to the stack of 3750G's where all the routing is performed.

There is a QoS policy in place in the switches for Cisco VoIP but it may need to be fixed since it existed before the cable connection came in...

However the goal is to speed up the transfer, because it seems that the duplicate ACK's may be exposing the slowness in transfer. Just seeing what part to attack next.

[lgijssel]

From your description, I noticed there are no dual links in the path between the two hosts.

One possible source remaining for this phenomenon is a duplex mismatch and I would search where I made changes.

You changed the firewall from one port to another, right?

When there is a duplex mismatch between the FW and the switch and the FW is operating in half duplex, then it is possible that it thinks a collision has occurred and needs to resend the frame. As there is mainly ACK traffic coming back from the file transfer it is logical to presume that many of these will be duplicated in this way.

You may have made other changes to the network as well so the root cause might be elsewhere but a scenario like this is a plausible explanation for your issue.

regards,

Leo

[netguyz08]

Leo,

i moved the connection for the LAN to the firewall, from one switch to another, since it seemed to me it should be at the core switch (the two 3750G's linked together) since the routing is there. Nothing was changed on the firewall. Other network changes were moving the routing table from the firewall to the switch. And it is true that it is just one link and IPSec tunnel between the two sites experiencing duplicate ACKs. They were connected over MPLS as well and would continue to be were it not for the IPSec tunnel.

I checked duplex at the firewall and at the switch level, and it is set to full/auto throughout. And I checked in MTU per Edison, and noticed everything is set to MTU 1500 on the WAN side. That might be it right there.. haven't changed it yet.

[Edison Ortiz]

The IPSec device may be sending PMTU information back to the source since packets to avoid packet fragmentation.

As you know, IPSec adds additional information to the IP Header which reduces the size of the IP Packet.

If you want to avoid PMTU, then you need to adjust the MTU at the source.

Regards,

Edison.

[netguyz08]

Tried reducing the MTU on the WAN port on either side of the IPSec tunnel from 1500 to 1364, renegotiated the tunnel and tried again, but still received the same amount of duplicate ack's.

[dixho]

can you post the sniffer trace please? I would like to look at the ID field in IP header and find out if the duplicate ACKs are the same packet or different packet. Also, want to look at ttl and find out if there is a routing loop.

From the TCP's perspective, the only reason I can think of for source sending TCP acks to the destination is that the source wants to send TCP flag push or urgent to the destination

[netguyz08]

Alright, I've attached a Wireshark capture. It is between 192.168.101.5 and 192.168.100.225.

The duplicate ack's start at packet 1946, then 2558, 2566, 2585, 2591, 2596, 2601, 2608, etc.

[dixho]

Thanks for the sniffer trace. I would like to clarify the network topoloyg. Data is transferred from 192.168.101.5 to 192.168.100.225.The TCP ack is from 192.168.100.225 to 192.168.101.5.

I cannot think of any reason why 192.168.100.225. It is getting late. I will research on the problem tomorrow. May I ask why you are concerned on the duplicate acks. Most TCP stack just ignore these TCP acks.

[netguyz08]

My goal here is to speed up the transfer over the WAN link. Seems it should be faster with two fast cable connections between the two locations. I did speed things up by 2x recently by removing an old D-Link 10/100 switch and putting a Cisco 3750G in its place.

So I am working to correct any issues that come up, including potential VLAN or QoS implementations that may also be causing this.

[dixho]

I do not think that the duplicate Acks will improve any throughput. From the sniffer trace, it looks like that there is high round trip time; however, I do not see the sender exhaust TCP window. I have serious doubt that QoS will improve the throughput.

[netguyz08]

So the duplicate ACKs are OK then, and it seems there is nothing more I can do to speed up transfer based on the high round-trip time, correct?