RVS4000 --> Fortigate 200A

[postfalls]

I'm attempting to set up an IPSEC VPN connection between my RVS4000 at home and my Fortigate 200A at work.  I've verified all Phase 1 and Phase 2 settings and checked to make sure the shared key is identical on both units.  When I try to initiate a connection, the log shows the following:

Feb 25 10:50:54 - [VPN Log]: "Fortigate" #61: next payload type of ISAKMP Hash Payload has an unknown value: 172
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #61: malformed payload in packet
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #61: sending notification PAYLOAD_MALFORMED to {REMOTE_IPADDRESS}:500
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: received Vendor ID payload [RFC 3947] method set to=109
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: ignoring unknown Vendor ID payload [8f8d83826d246b6fc7a8a6a428c11de8]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: ignoring unknown Vendor ID payload [439b59f8ba676c4c7737ae22eab8f582]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: ignoring unknown Vendor ID payload [4d1e0e136deafa34c4f3ea9f02ec7285]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: ignoring unknown Vendor ID payload [80d0bb3def54565ee84645d4c85ce3ee]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: ignoring unknown Vendor ID payload [9909b64eed937c6573de52ace952fa6b]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-03] meth=108, but already using method 109
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02] meth=107, but already using method 109
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n] meth=106, but already using method 109
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: ignoring unknown Vendor ID payload [16f6ca16e4a4066d83821a0f0aeaa862]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-00]
Feb 25 10:50:54 - [VPN Log]: packet from {REMOTE_IPADDRESS}:500: received Vendor ID payload [Dead Peer Detection]
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #62: Aggressive mode peer ID is ID_IPV4_ADDR: {REMOTE_IPADDRESS}'
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #62: responding to Aggressive Mode, state #62, connection "Fortigate" from {REMOTE_IPADDRESS}
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #62: transition from state STATE_AGGR_R0 to state STATE_AGGR_R1
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #62: STATE_AGGR_R1: sent AR1, expecting AI2
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #62: packet rejected: should have been encrypted
Feb 25 10:50:54 - [VPN Log]: "Fortigate" #62: sending notification INVALID_FLAGS to {REMOTE_IPADDRESS}:500
Feb 25 10:50:55 - [VPN Log]: "Fortigate" #63: initiating Aggressive Mode #63, connection "Fortigate"
Feb 25 10:50:56 - [VPN Log]: "Fortigate" #63: message ignored because it contains an unknown or unexpected payload type (ISAKMP_NEXT_NAT-D) at the outermost level
Feb 25 10:50:56 - [VPN Log]: "Fortigate" #63: sending notification INVALID_PAYLOAD_TYPE to {REMOTE_IPADDRESS}:500
Feb 25 10:50:58 - [VPN Log]: "Fortigate" #63: message ignored because it contains an unknown or unexpected payload type (ISAKMP_NEXT_NAT-D) at the outermost level
Feb 25 10:50:58 - [VPN Log]: "Fortigate" #63: sending notification INVALID_PAYLOAD_TYPE to {REMOTE_IPADDRESS}:500

No connection is ever made.  Does anyone know what I should be looking at to fix this???

[alissitz]

Hello,

I am not sure of what the problem is, however there are several messages below for why a packet or communication is denied. "packet should be encrypted, message ignored because it contains an unknown or unexpected payload type (ISAKMP_NEXT_NAT-D) at the outermost level, etc ..."

It looks like something in the config parameters is not matching.

Does Fortigate have any literature for how you can connect different vendor's VPN?  Is there a 'standardized' method for connecting multiple vendors with the Fortigate?

Perhaps someone much smarter than me on this community has some additioanl suggestions, however I would suggest checking with Fortigate and seeing if they have a config guide that explains how to connect to 3rd party vendors and which settings will be standard and accepted.

Also, you may consider posting your config here.  Just be sure to remove any information that is sensitive and you don't want to be shared.

HTH,

Andrew Lissitz

[postfalls]

Actually, I got it working.  The Fortigate has an advanced setting under the Phase 2 configuration like this:

Quick Mode Selector

Source address WORK_INTERNAL_SUBNET Source port Destination address HOME_INTERNAL_SUBNET Destination port Protocol

I had left these at all zeros which was supposed to allow all subs on either side, but once I specified my main one, it started up.

Thanks for your help.

[alissitz]

Glad to hear it is up and going, and I appreciate the update and letting us know how you solved this!

Good stuff.

Best wishes for a great weekend,

Andrew Lissitz