IPSEC causing connection problems

[mathewh]

I have set up an encryption link at work and everything works fine apart from one application. It works without encryption but when the encryption is on it does not.

The encryption session is up and active and so i wont go into detail about the configuratiob but FYI it is set up as below and mirrored on the other router:-

crypto isakmp policy 1

authentication pre-share

crypto isakmp key xxx address 11.x.x.5 255.255.255.252

!

!

crypto ipsec transform-set esamo2 esp-3des esp-md5-hmac

!

crypto map esamo 10 ipsec-isakmp

description VPN to estvpn2

set peer 11.11.11.5

set transform-set esamo2

match address ESA-INTERNAL

When i captured some packets from ethereal, i saw that the conversation to this particular server stopped with DUP ACK packet returns. The fact that you're seeing the data sent correctly indicates that nothing is wrong with the sender, nor with the network equipment up to the point of the sniffer, but that packets (segments) aren't correctly

arriving at the receiver. I am therefore stuck at where i can take this. Any feedback would be great!

[sunilc]

Hello,

If you think the config is fine, the tunnel is up, end-end routing is fine; but there is packet loss in one direction for one application - in most cases this could be fragmentation issues.

Try a ping with different packet sizes to see if this is true.

Can mitigate this by setting mtu, using ip tcp-mss-adjust, by using df-bit override on the crypto platform etc.. depending on where the problem is.

Hope this helps.

Regards,

-Sunil.

[mathewh]

Hello Sunil

It worked by setting the df-bit override to clear. However does this mean that the router will just fragment to whatever size is needed? If this is the case then it would be more preferable than setting the MTU with the ip tcp-mss-adjust command. Im not 100% sure what the df-bit actually did :) It did solve the problem though.