EasyVPN Server and Remote help

[levin.davie]

Hello. I’ve been trying to figure this out for couple of days now and not getting much success. The situation is we have a need to two things: Client based VPN access for one person that is not able to use our corporate VPN solution, and a remote VPN from a site that is blocking all but 80 and 443 (so we will use cTCP on port 443). We have two 871’s, one at our main office and the other will be at the remote site, both on 12.4(24)T8 Advanced IP Services. The one at the main office I have configured for client based using the Cisco VPN Client 5.0.07.0290 and that works perfectly. I am now working on the 871 that will be deployed at the remote location and be set up as an EasyVPN Remote to connect to the 871 at the main office. As some background I have not worked on a lot on Cisco firewalls and Security devices so I am going the easy route and have been using a combination of CLI and CCP 2.8, CCP for all VPN setting and firewall. I have created two Easy VPN server entries for each solution and the one for the Remote is using a local user and a VPN group for authorization. I have enabled firewall rule hit logs and Cyrpto logs so I can see the connection and this works for the client based but I never see any logs when testing the Remote. One of the ways I am testing is using the client on a laptop and using the VPN group for the Remote and change to the “IPSec over TCP” on port 443 but never get any logs or response from the 871. Packet captures show no returned packets. I think the 871 at the home office is not listening on 443 or dropping it because if I change the client from “IPSec over UDP” to over TCP I also see no logs or returned traffic. Also, if I change the entry on the client that I am using to test with using the username and VPN group for the remote office from “IPSec over TCP” to UDP I see these logs on the 871 at the home office “%CRYPTO-6-IKMP_NOT_ENCRYPTED: IKE packet from xx.xx.xx.xx was not encrypted and it should've been.” I have verified that the home office 871 has this config line “crypto ctcp port 443”. I must be missing something with TCP/443 but I can’t figure it out.

I've attached a screen shot of the firewall polices, obviously CCP built. I removed most of the allow rules except for the 1st rule which is what I think this traffic would hit.

Thank you for any help you can provide.

Levin

[Philip D'Ath]

Check out this guide (using the CLI):

http://www.cisco.com/c/en/us/products/collateral/security/ios-easy-vpn/prod_white_paper0900aecd8061e2b3.html

[levin.davie]

Thank you for that. It was helpful in that it seems my config is good, at least that I can find but I am still missing something somwhere. The server does see the remote talking to it:

sc-vpn-rtr-01#sh crypto ctcp
        Remote                   Local               VRF            Status

   XX.XXX.XXX.25:64382    192.168.XXX.XX:443                        CTCP_ACK_R

And the remote shows this:

mnr-fw-01#sh crypto ctcp
        Remote                   Local               VRF            Status

   XX.XXX.XXX.79:443        192.168.XXX.XX:26348                      CTCP_SYN_S

The internet routable IP's are what I would expect to see. Other then the above commands and our perimeter firewall logs I can't find any other logs or errors on the server to help me understand what I am missing here.  I'm going to post scrubbed configs, maybe someone wouldn't mind taking a look. I think it has something to do with the IPsec settings, might be missing something? Thanks.

Config of EzVPN REMOTE 871

!
hostname mnr-fw-01
!
boot-start-marker
boot system flash flash:/c870-advipservicesk9-mz.124-24.T8.bin
boot-end-marker
!
aaa new-model
!
!
aaa authorization exec default local
!
!
aaa session-id common
clock timezone pst -8
clock summer-time PDT recurring
!
crypto pki trustpoint TP-self-signed-

 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-
 revocation-check none
 rsakeypair TP-self-signed-
!
!
crypto pki certificate chain TP-self-signed-
 certificate self-signed 01
  quit
dot11 syslog
ip source-route
!
!
!
!
ip cef
ip domain name XXX.org
ip name-server XX.XX.XXX.203
login on-failure log
login on-success log
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
vtp mode transparent
!
crypto logging session
crypto logging ezvpn
!
!
!
crypto ipsec client ezvpn CISCOCP_EZVPN_CLIENT_1
 connect auto
 ctcp port 443
 group MNR key secret-key
 mode network-extension
 peer 65.127.10.79
 virtual-interface 1
 username username password secret-password
 xauth userid mode local
!
crypto ctcp port 443
!
vlan 116
 name MNR-User_XXX.XX.XX.0/24
!
ip ssh time-out 60
ip ssh logging events
ip ssh version 2
!
class-map type inspect match-any SDM_IP
 match access-group name SDM_IP
!
!
policy-map type inspect sdm-permit-ip
 class type inspect SDM_IP
  pass
 class class-default
  drop log
!
zone security Untrust
zone security Trust
zone security ezvpn-zone
zone-pair security sdm-zp-out-ezpn1 source Untrust destination ezvpn-zone
 service-policy type inspect sdm-permit-ip
zone-pair security sdm-zp-ezvpn-out1 source ezvpn-zone destination Untrust
 service-policy type inspect sdm-permit-ip
zone-pair security sdm-zp-ezvpn-in1 source ezvpn-zone destination Trust
 service-policy type inspect sdm-permit-ip
zone-pair security sdm-zp-in-ezvpn1 source Trust destination ezvpn-zone
 service-policy type inspect sdm-permit-ip
!
!
!
interface FastEthernet0
 switchport access vlan 116
 spanning-tree portfast
!
interface FastEthernet1
 switchport access vlan 116
 spanning-tree portfast
!
interface FastEthernet2
 switchport access vlan 116
 spanning-tree portfast
!
interface FastEthernet3
 switchport access vlan 116
 spanning-tree portfast
!
interface FastEthernet4
 description $ETH-WAN$
 ip address dhcp client-id FastEthernet4 hostname mnr-fw-01
 zone-member security Untrust
 duplex auto
 speed auto
 crypto ipsec client ezvpn CISCOCP_EZVPN_CLIENT_1
!
interface Virtual-Template1 type tunnel
 no ip address
 zone-member security ezvpn-zone
 tunnel mode ipsec ipv4
!
interface Vlan1
 no ip address
 shutdown
!
interface Vlan116
 ip address XXX.XX.XXX.1 255.255.255.0
 zone-member security Trust
 crypto ipsec client ezvpn CISCOCP_EZVPN_CLIENT_1 inside
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
!
!
!
ip access-list extended SDM_IP
 remark CCP_ACL Category=1
 permit ip any any
!
access-list 100 remark CCP_ACL Category=4
access-list 100 permit ip XXX.XX.XXX.0 0.0.0.255 any
!
control-plane
!

end

Config of EzVPN Server 871:

hostname sc-vpn-rtr-01
!
boot-start-marker
boot-end-marker
!
aaa new-model
!
!
aaa authentication login default group Duo local
aaa authentication login ciscocp_vpn_xauth_ml_1 local
aaa authentication login ciscocp_vpn_xauth_ml_2 group dc-03 local
aaa authorization console
aaa authorization exec default local
aaa authorization network ciscocp_vpn_group_ml_1 local
aaa authorization network ciscocp_vpn_group_ml_2 local
!
!
aaa session-id common
clock timezone PCTime -8
clock summer-time PCTime date Apr 6 2003 2:00 Oct 26 2003 2:00
!
crypto pki trustpoint TP-self-signed-
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-
 revocation-check none
 rsakeypair TP-self-signed-
!
!
crypto pki certificate chain TP-self-signed-
 certificate self-signed 01
 quit
dot11 syslog
ip source-route
!
!
!
!
ip cef
ip domain name XXX.org
ip name-server XX.XXX.X.203
login on-failure log
login on-success log
no ipv6 cef
!
multilink bundle-name authenticated
!
parameter-map type regex ccp-regex-nonascii
 pattern [^\x00-\x80]

parameter-map type protocol-info yahoo-servers
 server name scs.msg.yahoo.com
 server name scsa.msg.yahoo.com
 server name scsb.msg.yahoo.com
 server name scsc.msg.yahoo.com
 server name scsd.msg.yahoo.com
 server name cs16.msg.dcn.yahoo.com
 server name cs19.msg.dcn.yahoo.com
 server name cs42.msg.dcn.yahoo.com
 server name cs53.msg.dcn.yahoo.com
 server name cs54.msg.dcn.yahoo.com
 server name ads1.vip.scd.yahoo.com
 server name radio1.launch.vip.dal.yahoo.com
 server name in1.msg.vip.re2.yahoo.com
 server name data1.my.vip.sc5.yahoo.com
 server name address1.pim.vip.mud.yahoo.com
 server name edit.messenger.yahoo.com
 server name messenger.yahoo.com
 server name http.pager.yahoo.com
 server name privacy.yahoo.com
 server name csa.yahoo.com
 server name csb.yahoo.com
 server name csc.yahoo.com

parameter-map type protocol-info msn-servers
 server name messenger.hotmail.com
 server name gateway.messenger.hotmail.com
 server name webmessenger.msn.com

parameter-map type protocol-info aol-servers
 server name login.oscar.aol.com
 server name toc.oscar.aol.com
 server name oam-d09a.blue.aol.com

!
!
vtp mode transparent
username username secret 5 secret
!
crypto logging session
crypto logging ezvpn
!
crypto isakmp policy 1
 encr aes 256
 authentication pre-share
 group 2
 lifetime 480
!
crypto isakmp policy 2
 lifetime 480
!
crypto isakmp policy 3
 encr 3des
 authentication pre-share
 group 2
!
crypto isakmp client configuration group Telecom
 key secret
 dns XX.XXX.XX.X
 domain XXXXXX.XXX
 pool SDM_POOL_1
 netmask 255.255.255.224
!
crypto isakmp client configuration group MNR
 key secret
crypto isakmp profile ciscocp-ike-profile-1
   match identity group Telecom
   client authentication list ciscocp_vpn_xauth_ml_2
   isakmp authorization list ciscocp_vpn_group_ml_2
   client configuration address respond
   virtual-template 1
crypto isakmp profile ciscocp-ike-profile-2
   match identity group MNR
   client authentication list ciscocp_vpn_xauth_ml_2
   isakmp authorization list ciscocp_vpn_group_ml_1
   client configuration address initiate
   client configuration address respond
   keepalive 20 retry 2
   virtual-template 2
!
crypto ipsec security-association lifetime seconds 86400
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP-ASE_256_SHA1 esp-aes 256 esp-sha-hmac
!
crypto ipsec profile CiscoCP_Profile1
 set security-association lifetime seconds 3600
 set security-association idle-time 3600
 set transform-set ESP-ASE_256_SHA1
 set isakmp-profile ciscocp-ike-profile-1
!
crypto ipsec profile CiscoCP_Profile2
 set security-association lifetime seconds 3600
 set transform-set ESP-ASE_256_SHA1
 set isakmp-profile ciscocp-ike-profile-2
!
!
crypto dynamic-map dynmap 1
 set transform-set ESP-ASE_256_SHA1
 reverse-route
!
!
crypto map dynmap isakmp authorization list ciscocp_vpn_xauth_ml_1
crypto map dynmap client configuration address respond
!
crypto map static-map 1 ipsec-isakmp dynamic dynmap
!
crypto ctcp port 443
!
!
vlan 852
 name Cisco-VPN_XX.X.XXX.128/26
!
ip ssh time-out 60
ip ssh logging events
ip ssh version 2
!
class-map type inspect match-any ccp-cls-protocol-p2p
 match protocol edonkey signature
 match protocol gnutella signature
 match protocol kazaa2 signature
 match protocol fasttrack signature
 match protocol bittorrent signature
class-map type inspect match-any SDM_AH
 match access-group name SDM_AH
class-map type inspect http match-any ccp-app-nonascii
 match  req-resp header regex ccp-regex-nonascii
class-map type inspect match-any ccp-cls-insp-traffic
 match protocol dns
 match protocol https
 match protocol icmp
 match protocol imap
 match protocol pop3
 match protocol tcp
 match protocol udp
class-map type inspect match-all ccp-insp-traffic
 match class-map ccp-cls-insp-traffic
class-map type inspect match-all SDM_IP
 match access-group name SDM_IP
class-map type inspect match-any SDM_ESP
 match access-group name SDM_ESP
class-map type inspect match-any SDM_EASY_VPN_SERVER_TRAFFIC
 match protocol isakmp
 match protocol ipsec-msft
 match class-map SDM_AH
 match class-map SDM_ESP
class-map type inspect match-all SDM_EASY_VPN_SERVER_PT
 match class-map SDM_EASY_VPN_SERVER_TRAFFIC
class-map type inspect match-any ccp-cls-icmp-access
 match protocol icmp
 match protocol tcp
 match protocol udp
class-map type inspect match-any ccp-cls-protocol-im
 match protocol ymsgr yahoo-servers
 match protocol msnmsgr msn-servers
 match protocol aol aol-servers
class-map type inspect match-all ccp-icmp-access
 match class-map ccp-cls-icmp-access
class-map type inspect match-all ccp-invalid-src
 match access-group 100
class-map type inspect http match-any ccp-app-httpmethods
 match  request method bcopy
 match  request method bdelete
 match  request method bmove
 match  request method bpropfind
 match  request method bproppatch
 match  request method connect
 match  request method copy
 match  request method delete
 match  request method edit
 match  request method getattribute
 match  request method getattributenames
 match  request method getproperties
 match  request method index
 match  request method lock
 match  request method mkcol
 match  request method mkdir
 match  request method move
 match  request method notify
 match  request method options
 match  request method poll
 match  request method post
 match  request method propfind
 match  request method proppatch
 match  request method put
 match  request method revadd
 match  request method revlabel
 match  request method revlog
 match  request method revnum
 match  request method save
 match  request method search
 match  request method setattribute
 match  request method startrev
 match  request method stoprev
 match  request method subscribe
 match  request method trace
 match  request method unedit
 match  request method unlock
 match  request method unsubscribe
class-map type inspect http match-any ccp-http-blockparam
 match  request port-misuse im
 match  request port-misuse p2p
 match  request port-misuse tunneling
 match  req-resp protocol-violation
class-map type inspect match-all ccp-protocol-http
 match protocol http
!
!
policy-map type inspect ccp-permit-icmpreply
 class type inspect ccp-icmp-access
  inspect
 class class-default
  pass
policy-map type inspect http ccp-action-app-http
 class type inspect http ccp-http-blockparam
  log
  reset
 class type inspect http ccp-app-httpmethods
  log
  reset
 class type inspect http ccp-app-nonascii
  log
  reset
policy-map type inspect ccp-inspect
 class type inspect ccp-invalid-src
  drop log
 class type inspect ccp-protocol-http
  inspect
  service-policy http ccp-action-app-http
 class type inspect ccp-insp-traffic
  inspect
 class class-default
  drop
policy-map type inspect ccp-permit
 class type inspect SDM_EASY_VPN_SERVER_PT
  pass
 class class-default
  drop
policy-map type inspect sdm-permit-ip
 class type inspect SDM_IP
  pass
 class class-default
  drop log
!
zone security in-zone
zone security ezvpn-zone
zone security out-zone
zone-pair security sdm-zp-in-ezvpn1 source in-zone destination ezvpn-zone
 service-policy type inspect sdm-permit-ip
zone-pair security ccp-zp-out-self source out-zone destination self
 service-policy type inspect ccp-permit
zone-pair security sdm-zp-out-ezpn1 source out-zone destination ezvpn-zone
 service-policy type inspect sdm-permit-ip
zone-pair security ccp-zp-self-out source self destination out-zone
 service-policy type inspect ccp-permit-icmpreply
zone-pair security sdm-zp-ezvpn-in1 source ezvpn-zone destination in-zone
 service-policy type inspect sdm-permit-ip
zone-pair security ccp-zp-in-out source in-zone destination out-zone
 service-policy type inspect ccp-inspect
zone-pair security sdm-zp-ezvpn-out1 source ezvpn-zone destination out-zone
 service-policy type inspect sdm-permit-ip
!
!
!
interface Loopback0
 no ip address
!
interface FastEthernet0
 switchport access vlan 852
 shutdown
 spanning-tree portfast
!
interface FastEthernet1
 switchport access vlan 852
 shutdown
 spanning-tree portfast
!
interface FastEthernet2
 description switch Gi2/0/1
 switchport access vlan 852
 spanning-tree portfast
!
interface FastEthernet3
 switchport access vlan 852
 shutdown
 spanning-tree portfast
!
interface FastEthernet4
 description switch Fa2/0/18$FW_OUTSIDE$
 ip address XXX.XXX.XXX.31 255.255.255.0
 zone-member security out-zone
 duplex auto
 speed auto
 crypto map static-map
!
interface Virtual-Template1 type tunnel
 ip unnumbered FastEthernet4
 zone-member security ezvpn-zone
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile CiscoCP_Profile1
!
interface Virtual-Template2 type tunnel
 description For ManorCare VPN
 ip unnumbered FastEthernet4
 zone-member security ezvpn-zone
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile CiscoCP_Profile2
!
interface Vlan1
 no ip address
 shutdown
!
interface Vlan852
 description $FW_INSIDE$
 ip address XX.X.XXX.131 255.255.255.192
 zone-member security in-zone
!
ip local pool SDM_POOL_1 XXX.XX.XXX.99 XXX.XX.XXX.XXX
ip default-gateway XX.X.XXX.128
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 XXX.XXX.XXX.1
ip route 10.0.0.0 255.0.0.0 XX.X.XXX.129
ip route 172.16.0.0 255.255.0.0 XX.X.XXX.129
ip route 192.168.157.0 255.255.255.0 XXX.XXX.XXX.1
ip http server
ip http authentication local
ip http secure-server
!
!
ip dns server
!
ip access-list extended SDM_AH
 remark CCP_ACL Category=1
 permit ahp any any
ip access-list extended SDM_ESP
 remark CCP_ACL Category=1
 permit esp any any
ip access-list extended SDM_IP
 remark CCP_ACL Category=1
 permit ip any any
!
access-list 100 remark CCP_ACL Category=128
access-list 100 permit ip host 255.255.255.255 any
access-list 100 permit ip 127.0.0.0 0.255.255.255 any
access-list 100 permit ip 192.168.157.0 0.0.0.255 any
!
!
control-plane
!
end

[Philip D'Ath]

Are you sure port 443 is allowed straight out?  Any chance they have an IPS inspecting the traffic to make sure it is actually https?

Could you try getting something like the below from one of the ends:

debug crypto isakmp

debug crypto ipsec

debug crypto ctcp

[levin.davie]

Thanks p.dath. Right now it is on my desk connected to a cable modem, much like it will be when I deploy it. I have our corporate firewall (that is in front of the 871 in the server role) allowing all ports for the nat on the way in and when I can see by the firewall logs that it is letting the traffic through. Do I need to do something extra because of the nats that are involved?

I've attached a drawing and the debugs from the remote side. There were no debug logs generated at all on the server side, though I know the traffic is reaching it. Also remember the Cisco VPN client works just fine on a laptop. I think there is something missing on the server 871 so it is not listening for 443.

[Philip D'Ath]

This is starting to sound like an IOS bug.  What version of software are you running on the 871?

[levin.davie]

12.4(24)T8 Advanced IP Services on both of them.

[levin.davie]

When trying to load in a different image (c870-advsecurityk9-mz.124-24.T4.bin) the router failed and I now get this error:

Booting flash:/c870-advipservicesk9-mz.124-24.T8.bin
Self decompressing the image : ########################################################################################### [OK]

Error : pre and post compression image sizes disagree

*** System received a Software forced crash ***
signal= 0x17, code= 0x8, context= 0x0
PC = 0x0, Vector = 0x0, SP = 0x0

So I am now stuck in rommon. I have tried three different IOS versions, formatted flash, reseated the flash and ram modules, and I always get that same error. Is there any way to bypass flash from rommon and load an image directly from tftp or into ram or something like that? I think flash may need to be squeezed but I can only use that command from an IOS image.

[levin.davie]

Sorry for the quick double post, but I found the -r switch to bypass flash "tftpdnld -r" but I still get the same error "Error : pre and post compression image sizes disagree". Does this indicate bad ram? This is very puzzling.

[Philip D'Ath]

It sounds like the image you downloaded is corrupted.

[levin.davie]

No, I've tried several other images including one currently running on a functional 871. Has to be hardware.

[levin.davie]

I know everyone is in suspense, I found that it is a bad RAM module. One I removed it boots fine.