rv082 VPN remote desktop encryption error

[Xandertje]

We have two rv082 routers. One at our main oice and the other at a remote office. Both have the latest firmware installed. They are connected through a VPN tunnel. All our computers have WinXP on them. I often use remote desktop through the VPN from my laptop to my desktop computer at the main office with no problems.

Now, I installed Win7 on both my machines and now keep getting an error after a few seconds to a minute after I start a remote desktop session through the VPN. "Because of an error in data encryption, this session will end." If I try a remote desktop session when I am at the main office (not through the VPN), there is no problem. I am using the same computers as before. The only thing that has changed is the operating system.

Remote desktop between WinXP computers through the VPN still works fine. For now, I remote desktop to a Win2003 R2 server through the VPN and then remote desktop from the server to my Win7 machine, which is slow to say the least.

Both computers are connected to the network directly, not wirelessly. I tried turning off Jumbo packets on both machine, but that did not help.

This is driving me crazy, any ideas?

[jtejavanija]

not familiar with shrew soft. i'm on a gateway-to-gateway ipsec vpn connecting home and work, and i use my home xp and vista pc's to rdp into my work win7 pc. hope that answers your question.

[Alejandro Gallego]

That doesn't make any sense!

Really wish we had some logs to look at, as I have not been able to reproduce the problem or even come close to see an error. I will take a closer look at the RV042 firmware and will use that router for the IPSec tunnel, but really there are no big differences between the RV0xx routers.

I am glad you were able to correct the issue, but replacing the product is not what I was hoping for. Thank you very much for the input and I am sorry I was not able to find the problem but will continue to look at this issue. If you were able to open a case with Small Business Support could you please IM me the case number? I would like to take a closer look at the case notes and possibly speak with you about your configuration.

Thank you,

Alejandro

[pgordon]

Okay.  Using Win7Pro, Shrew, and RV042, I am able to get to the point where "tunnel enabled."  I then try to open a remote desktop connection.  No answer from my office computer, which is also using Win7Pro and has been set to accept incoming remote desktop connection.  I then try to set up new connection to my office. The closest I can get is an error code saying DNS not responding or remote server not responding, but I don't have a remote server as such.  Here's the topology:

HomeComputer >> CableModem >> Internet >> DSLModem(bridged) >> RV042 >> Switch(Linksys) >> OfficeComputer(x5)

My particular office computer is set to listen to a particular port.  When I try to establish a remote desktop connection, I type in the IP address and the port number (e.g. 12.34.56.789:12345).  Still, I get no response.  Assuming all of the configurations are correct, should I get a new router, or do I need to dedicate one of the five office computer to being a server with Win Server 2008?

[Xandertje]

I think that pgorden's post is off topic. These posts have been about an encryption error that occurs after a remote desktop connection is made. Please start a new post.

[David Hornstein]

Hi pgordon,

Check out this posting below as well.  Well worth a look at using the native IPSec client in windows 7.  Just wish I had a powerful enough PC that would accept  windows 7 so i could validate the procedure in the posting below;

https://www.myciscocommunity.com/docs/DOC-13592#cf

[Xandertje]

I am still having the same problem. I am using rv082 on both sides with the latest frimware (2.0.0.19-tm). My settings are the same as you describe. This is really driving me crazy!

Which logs would you like to see specifically?

I would like to mention that it does not happen right away. You need to create some traffic by opening windows through the remote desktop.

[Alejandro Gallego]

I would like to see the VPN, and Firewall logs; keep in mind that they be very lengthy and would have all your IP information and also ports being used for connections. If possible please call the Small Business center and open a case or send me an IM via your Cisco Community account.

SBSC: 1.866.606.1866 (USA)

[Xandertje]

I looked in the logs on the target machine and found something interesting:

"encountered a bad packet from xxx.xxx.xxx.xxx. Packet processing leads beyond packet length".

Googling the error lead me to the following post:

http://social.technet.microsoft.com/Forums/en-US/winservergen/thread/51d005e2-8ac4-4fa8-bbf7-f8c2e3f4dce4

I read the post and we don't have Broadcom network cards, so that can't be it. The cards are Intel on both sides. I disabled "Jumbo packets" and  "IPv4 Large Send Offload" on both, but it did not help. Besides I don't have any problems when I RDP locally.

Then he goes on to suggest enabling "Rate Control" on the  "Bandwidth Management" page. I tried it on both sides and it changed nothing.

Any ideas now?

[Alejandro Gallego]

From that line in  your log

"encountered a bad packet from xxx.xxx.xxx.xxx. Packet processing leads beyond packet length".

I would think that the problem is not due to jumbo frames or anyting like that. It seems like we are getting CRC errors. You mentioned that you turned off jumbo frames on your NIC, are they enabled on all your machines? Also is the router connected to a switch with jumbo frames enabled? In your log did you happen to notice anything saying "Policy Violation" or any logged events on either W7 machine (Application Event Logs or System Event Logs)?

[Xandertje]

The router is connected to a Cisco SR2024 Gigabit Switch.

[Xandertje]

I found something that seems to work for now!

Under the 'setup' tab of rvo82, I set the MTU size for WAN1 to 'Manual' 1500 bytes. Since then, I have not seen the encryption error once!

Does this settings have any other consequences?

[mihagan · Accepted solution]

Setting the WAN1 interface to manual 1500 bytes just forces that WAN interface to operator at that specific MTU as opposed to allowing to to negotiate that automatically.

You asked whether or not there are any consequences for changing this setting and the answer really depends on what the correct MTU should be for your connection. Forcing it to this MTU may be causing you to fragment packets. You can run a test from your network to help determine what would be the most ideal setting for your connection.

Open up a command prompt on a machine on your LAN and get an external address to ping out on the internet. Run the following command where x.x.x.x is the external IP address.

ping -f -l 1500 x.x.x.x

This will send a ping with an packet size of 1500. If that MTU is not correct you will get a message that reads "Packet needs to be fragmented but DF set". If the MTU is fine then you will get your standard reply results. If you get the fragmentation error just keep testing by running the same command but lowering the packet size until you get the normal replies. I hope this helps and you may find that 1500 is the perfect MTU for your connection.

[Xandertje]

Good idea. I played around with the ping using a fixed packet size and I found that 1472 was the right size for my connection. So that is the solution encrption error.

The target computer was receiving packets which were too large. This caused CRC errors which the RDP saw as encryption errors.

Under the 'setup' tab of the rv082 router, I set the MTU size for WAN1 to 'Manual' 1472 bytes. I did this on both routers. Since then I have not had one encryption error.

The correct packet size will depend on your internet connection. Use Mihagen's idea to check the correct packet size for each router: ping -f -l 1500 x.x.x.x

Thnaks to Alegalle and Mihagen.

Problem solved!!

[jtejavanija]

i tried doing the ping test. on my home router it works just fine- highest mtu with normal ping results is 1430. changing this value allowed me to properly use quickvpn from home into my office- it didn't work when mtu was set to auto (this is with the ipsec tunnel disabled of course).

however, on my work router (rvs4000) at mtu 1473 it says it has to break up the packets. but at mtu 1472 it says timed out for all 4 attempts. (all mtu values lower than 1472 also say timed out, and all values higher than 1473 also says it has to break up the packets).

the rvs4000 at the office does seem to work fine right now though, with mtu at "auto" (the field says "1500" grayed out, in case that matters). i just don't know whether i need to manually configure it to have an optimized connection.

any advice would be appreciated.

[Alejandro Gallego]

As Michael stated you may have found the problem. In TCP if we send a packet that is malformed or has to be fragmented, the receiving end will take care of it by either requesting a new packet or if fragmented, waits until all fragments are received and then puts the data back together. Now if we add on top of that encryption for the VPN tunnel and Encryption for the RDP session and we are constantly having to resend packets, it is easy to see why the connection would drop. I do most of my work from a mac, and in the Mac version of Command Prompt we are able to get a really good idea as to what the packet size should be. Here is a little example I did from home: (cable connection)

>>> The options for "ping" in Mac are different than on Windows, but the test is the same. We don't see "Reply from" just the data is returned which is more useful in my opinion and shows more clearly what happens to the data. Take note of the colored numbers. <<<

LayOff2:~ agallego$ ping -s 1500 google.com
PING google.com (64.233.169.103): 1500 data bytes
1480 bytes from 64.233.169.103: icmp_seq=0 ttl=243 time=37.670 ms
wrong total length 1500 instead of 1528

Now I reduced the MTU to 1480

LayOff2:~ agallego$ ping -s 1480 google.com
PING google.com (64.233.169.106): 1480 data bytes
1480 bytes from 64.233.169.106: icmp_seq=0 ttl=243 time=33.656 ms
wrong total length 1500 instead of 1508

Now to 1478

LayOff2:~ agallego$ ping -s 1478 google.com
PING google.com (64.233.169.106): 1478 data bytes
1480 bytes from 64.233.169.106: icmp_seq=0 ttl=243 time=39.252 ms
wrong total length 1500 instead of 1506 <=== Still off by 6

Finally

LayOff2:~ agallego$ ping -s 1472 google.com <=== This is the Max I can transmit. Anything less will be just fine.
PING google.com (64.233.169.147): 1472 data bytes
1480 bytes from 64.233.169.147: icmp_seq=0 ttl=243 time=33.566 ms
1480 bytes from 64.233.169.147: icmp_seq=1 ttl=243 time=37.852 ms
1480 bytes from 64.233.169.147: icmp_seq=2 ttl=243 time=35.010 ms