duplicate ACK, out-of-order segment, retransmission and fast-retransmission

[Tshi M]

Hi all,

I am troubleshooting some queueing problem with one of our vendor. I SPAN the port on the switch that connect to their router and I have attached a capture file. The IP address of 10.33.85.24 is one of my local server and the other is the vendor IP. From what I can see, my local server sends ACK but the vendor server ACK seems to show a loss of segment.

Could someone please take a look at the capture file and let me know what they think the problem might and where?

Regards,

[michael.leblanc]

This appears to be an upload from client to server, with the capture taken at the server location.

No. Time Source Protocol Info

10 0.020832 server TCP 36146 > 8252 [ACK] Seq=1 Ack=2493 Win=32767 Len=0

11 0.038359 client TCP 8252 > 36146 [ACK] Seq=2493 Ack=1 Win=65364 Len=1288

12 0.038535 server TCP 36146 > 8252 [ACK] Seq=1 Ack=3781 Win=32767 Len=0

13 0.039805 client TCP [TCP Previous segment lost] 8252 > 36146 [PSH, ACK] Seq=4329 Ack=1 Win=65364 Len=166

The server is expecting a packet with Seq=3781 (conveyed by the ACK in packet 12), but the next client packet has Seq=4329.

Packet 15 was expected to arrive before packet 13 but did not. They likely took different paths, or were influenced by queues.

14 0.039982 server TCP [TCP Dup ACK 12#1] 36146 > 8252 [ACK] Seq=1 Ack=3781 Win=32767 Len=0

The server is ACKing what it has received "in-order" (i.e.: packet 15 is still outstanding).

15 0.041446 client TCP [TCP Out-Of-Order] 8252 > 36146 [PSH, ACK] Seq=3781 Ack=1 Win=65364 Len=548

The delinquent packet with Seq=3781 arrives.

16 0.041617 server TCP 36146 > 8252 [ACK] Seq=1 Ack=4495 Win=32767 Len=0

The server is ACKing what it has received "in-order" (i.e.: packet 13 had highest Seq#).

Packet 13 Seq=4329 + Len=166 = the most recent ACK=4495

17 0.043313 client TCP [TCP Previous segment lost] 8252 > 36146 [PSH, ACK] Seq=5783 Ack=1 Win=65364 Len=389

The server is expecting a packet with Seq=4495 (conveyed by the ACK in packet 16), but the next client packet has Seq=5783.

Packet 19 was expected to arrive before packet 17 but did not. They likely took different paths, or were influenced by queues.

18 0.043491 server TCP [TCP Dup ACK 16#1] 36146 > 8252 [ACK] Seq=1 Ack=4495 Win=32767 Len=0

The server is ACKing what it has received "in-order" (i.e.: packet 19 is still outstanding).

19 0.048370 client TCP [TCP Retransmission] 8252 > 36146 [ACK] Seq=4495 Ack=1 Win=65364 Len=1288

The delinquent packet with Seq=4495 arrives.

All of your data is being delivered. However, packets are arriving out of order because they are taking different paths, or they are being influenced by queueing (at the router, or closer to the client).

I wouldn't think that QoS on the switch would affect output to the SPAN destination port. Using a network tap instead of SPAN would confirm that.

[Tshi M]

Hi Michael,

Thanks for the quick reply. The capture was done on the client side. I put a SPAN on the port that connects to the vendor's router. So I am capturing from my side (client) going to the server side .

[michael.leblanc]

I'm confused by your statement. Your initial post indicated:

The IP address of 10.33.85.24 is one of my local server and the other is the vendor IP.

If the server is local to you, and you are capturing on your side of the vendor's router, would that not mean you are capturing on the server side?

[Tshi M]

I am sorry. I should have pointed out that the client IP address is 10.33.85.24 and the vendor server IP address is 208.x.x.x.

I am capturing from the client server side. So, in your interpretation of the capture which I thank you much for, frame 12 is an ACK from the client. If I understand it.

[michael.leblanc]

When you say "I am capturing from the client server side." you are introducing more confusion. You are at the client side (one or the other, not both).

Packet 12 is the client ACKing the server, based on your correction of the identities.

*** Based on the correction to your initial post (10.33.85.24 is the server). ***

This is a download from the server, with the capture taken at the client location.

No. Time Source Protocol Info

10 0.020832 client TCP 36146 > 8252 [ACK] Seq=1 Ack=2493 Win=32767 Len=0

11 0.038359 server TCP 8252 > 36146 [ACK] Seq=2493 Ack=1 Win=65364 Len=1288

12 0.038535 client TCP 36146 > 8252 [ACK] Seq=1 Ack=3781 Win=32767 Len=0

13 0.039805 server TCP [TCP Previous segment lost] 8252 > 36146 [PSH, ACK] Seq=4329 Ack=1 Win=65364 Len=166

The client is expecting a packet with Seq=3781 (conveyed by the ACK in packet 12), but the next server packet has Seq=4329.

Packet 15 was expected to arrive before packet 13 but did not. They likely took different paths, or were influenced by queues.

14 0.039982 client TCP [TCP Dup ACK 12#1] 36146 > 8252 [ACK] Seq=1 Ack=3781 Win=32767 Len=0

The client is ACKing what it has received "in-order" (i.e.: packet 15 is still outstanding).

15 0.041446 server TCP [TCP Out-Of-Order] 8252 > 36146 [PSH, ACK] Seq=3781 Ack=1 Win=65364 Len=548

The delinquent packet with Seq=3781 arrives.

16 0.041617 client TCP 36146 > 8252 [ACK] Seq=1 Ack=4495 Win=32767 Len=0

The client is ACKing what it has received "in-order" (i.e.: packet 13 had highest Seq#).

Packet 13 Seq=4329 + Len=166 = the most recent ACK=4495

17 0.043313 server TCP [TCP Previous segment lost] 8252 > 36146 [PSH, ACK] Seq=5783 Ack=1 Win=65364 Len=389

The client is expecting a packet with Seq=4495 (conveyed by the ACK in packet 16), but the next server packet has Seq=5783.

Packet 19 was expected to arrive before packet 17 but did not. They likely took different paths, or were influenced by queues.

18 0.043491 client TCP [TCP Dup ACK 16#1] 36146 > 8252 [ACK] Seq=1 Ack=4495 Win=32767 Len=0

The client is ACKing what it has received "in-order" (i.e.: packet 19 is still outstanding).

19 0.048370 server TCP [TCP Retransmission] 8252 > 36146 [ACK] Seq=4495 Ack=1 Win=65364 Len=1288

The delinquent packet with Seq=4495 arrives.

All of your data is being delivered. However, packets are arriving out of order because they are taking different paths, or they are being influenced by queueing (at the router, or closer to the server).

I wouldn't think that QoS on the switch would affect output to the SPAN destination port. Using a network tap instead of SPAN would confirm that.

[Tshi M]

Thanks much. I actually rated your earlier posting.

[michael.leblanc]

Your welcome, and thank you.