Solved: VPN return traffic not flowing over the tunnel

bautsche123 · ‎10-12-2010

Hello.

I've tried to find something on the internet to solve this, but am failing miserably. I guess, I really don't understand how the cisco decides on routing.

Anyway, I have a Cisco 837 which I'm using for internet access and which I would like to be able to terminate a VPN on. When I vpn in (using vpnc from a Solaris box as it happens which is connected to the ethernet interface of the cisco), I can establish an VPN and when I ping a host on the inside, I can see that ping packet arrive, however the return packet, the cisco 837 tries to send over the public internet facing interface Dialer1 without encryption. I can't for the life of me work out why.

(Also note: I can also establish a tunnel from the public internet, but again, I can't get any traffic back through the tunnel. I assume that I'm having the same issue, ie return packets aren't going where they should, but I don't know that for certain, on the host being pinged though, I can see the ping packets arriving and the host responding with an ICMP Echo reply).

here is the cisco version:

adsl#show version
Cisco IOS Software, C850 Software (C850-ADVSECURITYK9-M), Version 12.4(15)T5, RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 01-May-08 02:07 by prod_rel_team

ROM: System Bootstrap, Version 12.3(8r)YI4, RELEASE SOFTWARE

adsl uptime is 1 day, 19 hours, 27 minutes
System returned to ROM by power-on
System restarted at 17:20:56 bst Sun Oct 10 2010
System image file is "flash:c850-advsecurityk9-mz.124-15.T5.bin"

Cisco 857 (MPC8272) processor (revision 0x300) with 59392K/6144K bytes of memory.
Processor board ID FCZ122391F5
MPC8272 CPU Rev: Part Number 0xC, Mask Number 0x10
4 FastEthernet interfaces
1 ATM interface
128K bytes of non-volatile configuration memory.
20480K bytes of processor board System flash (Intel Strataflash)

Configuration register is 0x2102

And here is the cisco's configuration (IP address, etc changed of course):

Current configuration : 7782 bytes
!
! Last configuration change at 11:57:21 bst Mon Oct 11 2010 by bautsche
! NVRAM config last updated at 11:57:22 bst Mon Oct 11 2010 by bautsche
!
version 12.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service sequence-numbers
!
hostname adsl
!
boot-start-marker
boot-end-marker
!
logging buffered 4096
enable secret 5 <secret>
!
aaa new-model
!
!
aaa authentication login local_authen local
aaa authentication login sdm_vpn_xauth_ml_1 local
aaa authorization exec local_author local
aaa authorization network sdm_vpn_group_ml_1 local
!
!
aaa session-id common
clock timezone gmt 0
clock summer-time bst recurring last Sun Mar 1:00 last Sun Oct 1:00
!
!
dot11 syslog
no ip source-route
ip dhcp database dhcpinternal
no ip dhcp use vrf connected
ip dhcp excluded-address 10.10.7.1 10.10.7.99
ip dhcp excluded-address 10.10.7.151 10.10.7.255
!
ip dhcp pool dhcpinternal
   import all
   network 10.10.7.0 255.255.255.0
   default-router 10.10.7.1
   dns-server 212.159.6.9 212.159.6.10 212.159.13.49 212.159.13.50
!
!
ip cef
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
no ip bootp server
ip host nfs1 10.10.140.207
ip name-server 212.159.11.150
ip name-server 212.159.13.150
!
!
!
username cable password 7 <password>
username bautsche password 7 <password>
username vpnuser password 7 <password>
!
!
crypto isakmp policy 1
encr 3des
hash md5
authentication pre-share
group 2
!
crypto isakmp policy 2
encr aes 256
authentication pre-share
group 2
!
crypto isakmp policy 3
encr 3des
authentication pre-share group 2
crypto isakmp client configuration address-pool local SDM_POOL_1
!
crypto isakmp client configuration group groupname2
key <key>
dns 10.10.140.201 10.10.140.202
domain swangage.co.uk
pool SDM_POOL_1
max-users 3
netmask 255.255.255.0
!
crypto isakmp client configuration group groupname1
key <key>
dns 10.10.140.201 10.10.140.202
domain swangage.co.uk
pool SDM_POOL_1
max-users 3
netmask 255.255.255.0
crypto isakmp profile sdm-ike-profile-1
   match identity group groupname2
   client authentication list sdm_vpn_xauth_ml_1
   isakmp authorization list sdm_vpn_group_ml_1
   client configuration address respond
crypto isakmp profile sdm-ike-profile-2
   match identity group groupname1
   isakmp authorization list sdm_vpn_group_ml_1
   client configuration address respond
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP_MD5_3DES esp-3des esp-md5-hmac
crypto ipsec transform-set ESP-AES-256-SHA esp-aes esp-sha-hmac
!
crypto dynamic-map SDM_DYNMAP_1 1
set security-association idle-time 3600
set transform-set ESP-AES-256-SHA
reverse-route
crypto dynamic-map SDM_DYNMAP_1 2
set security-association idle-time 3600
set transform-set ESP-AES-256-SHA
reverse-route
!
!
crypto map SDM_CMAP_1 client authentication list sdm_vpn_xauth_ml_1
crypto map SDM_CMAP_1 isakmp authorization list sdm_vpn_group_ml_1
crypto map SDM_CMAP_1 65535 ipsec-isakmp dynamic SDM_DYNMAP_1
!
crypto ctcp port 10000
archive
log config
hidekeys
!
!
ip tcp synwait-time 10
!
!
!
interface Null0
no ip unreachables
!
interface ATM0
no ip address
no ip redirects
no ip unreachables
no ip proxy-arp
ip route-cache flow
no atm ilmi-keepalive
pvc 0/38
encapsulation aal5mux ppp dialer
dialer pool-member 1
!
dsl operating-mode auto
hold-queue 224 in
!
interface FastEthernet0
!
interface FastEthernet1
!
interface FastEthernet2
!
interface FastEthernet3
!
interface Vlan1
description $FW_INSIDE$
ip address 10.10.7.1 255.255.255.0
ip access-group 121 in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat inside
ip virtual-reassembly
ip route-cache flow
crypto map SDM_CMAP_1
hold-queue 100 out
!
interface Dialer1
description $FW_OUTSIDE$
ip address negotiated
ip access-group 121 in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat outside
ip virtual-reassembly
encapsulation ppp
ip route-cache flow
no ip split-horizon
dialer pool 1
dialer idle-timeout 0
dialer persistent
dialer-group 1
no cdp enable
ppp authentication chap callin
ppp chap hostname <hostname>
ppp chap password 7 <password>
crypto map SDM_CMAP_1
!
ip local pool SDM_POOL_1 10.10.148.11 10.10.148.20
ip local pool public_184 123.12.12.184
ip local pool public_186 123.12.12.186
ip local pool public_187 123.12.12.187
ip local pool internal_9 10.10.7.9
ip local pool internal_8 10.10.7.8
ip local pool internal_223 10.10.7.223
ip local pool internal_47 10.10.7.47
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.10.140.0 255.255.255.0 10.10.7.2
!
no ip http server
no ip http secure-server
ip nat inside source route-map SDM_RMAP_1 interface Dialer1 overload
ip nat inside source static 10.10.7.9 123.12.12.184
ip nat inside source static tcp 10.10.7.8 22 123.12.12.185 22 extendable
ip nat inside source static tcp 10.10.7.8 25 123.12.12.185 25 extendable
ip nat inside source static tcp 10.10.7.8 80 123.12.12.185 80 extendable
ip nat inside source static tcp 10.10.7.8 443 123.12.12.185 443 extendable
ip nat inside source static tcp 10.10.7.8 993 123.12.12.185 993 extendable
ip nat inside source static tcp 10.10.7.8 1587 123.12.12.185 1587 extendable
ip nat inside source static tcp 10.10.7.8 8443 123.12.12.185 8443 extendable
ip nat inside source static 10.10.7.223 123.12.12.186
ip nat inside source static 10.10.7.47 123.12.12.187
!
logging 10.10.140.213
access-list 18 permit any
access-list 23 permit 10.10.140.0 0.0.0.255
access-list 23 permit 10.10.7.0 0.0.0.255
access-list 100 remark SDM_ACL Category=2
access-list 100 deny   ip any 10.10.148.0 0.0.0.255
access-list 100 permit ip any any
access-list 121 remark SDM_ACL Category=17
access-list 121 deny   udp any eq netbios-dgm any
access-list 121 deny   udp any eq netbios-ns any
access-list 121 deny   udp any eq netbios-ss any
access-list 121 deny   tcp any eq 137 any
access-list 121 deny   tcp any eq 138 any
access-list 121 deny   tcp any eq 139 any
access-list 121 permit ip any any
access-list 125 permit tcp any any eq www
access-list 125 permit udp any eq isakmp any
access-list 125 permit udp any any eq isakmp
access-list 194 deny   udp any eq isakmp any
access-list 194 deny   udp any any eq isakmp
access-list 194 permit ip host 123.12.12.184 any
access-list 194 permit ip any host 123.12.12.184
access-list 194 permit ip host 10.10.7.9 any
access-list 194 permit ip any host 10.10.7.9
access-list 195 deny   udp any eq isakmp any
access-list 195 deny   udp any any eq isakmp
access-list 195 permit ip host 123.12.12.185 any
access-list 195 permit ip any host 123.12.12.185
access-list 195 permit ip host 10.10.7.8 any
access-list 195 permit ip any host 10.10.7.8
no cdp run
route-map public_185 permit 10
match ip address 195
!
route-map public_184 permit 10
match ip address 194
!
route-map SDM_RMAP_1 permit 1
match ip address 100
!
!
control-plane
!
!
line con 0
login authentication local_authen
no modem enable
transport preferred none
transport output telnet
stopbits 1
line aux 0
login authentication local_authen
transport output telnet
stopbits 1
line vty 0 4
access-class 23 in
privilege level 15
authorization exec local_author
login authentication local_authen
length 0
transport preferred none
transport input telnet ssh
!
scheduler max-task-time 5000
scheduler allocate 4000 1000
scheduler interval 500
sntp server 130.88.202.49
sntp server 130.88.200.98
sntp server 130.88.200.6
sntp server 130.88.203.64
end

Any help would be appreciated.

Thanks a lot.

Ciao,

Eric

Herbert Baerten · ‎11-08-2010

Hi Eric,

(sorry for the delayed response - was in need of some vacation )

So I see you got a few steps farther now. I think there are 2 things we can try:

1)

I suppose you have put back this:

ip nat inside source route-map SDM_RMAP_1 interface Dialer1 overload

Since the routemap refers to ACL 100 to define the traffic to be translated, we can exclude the traffic that initiates from the router:

access-list 100 remark SDM_ACL Category=2

access-list 100 deny ip host 123.12.12.185 any
access-list 100 deny ip any 10.10.148.0 0.0.0.255
access-list 100 permit ip any any

That should prevent the udp source port from changing from 4500 to 1029

OR

2)

if you prefer to use another ip address for VPN,

then you can use a loopback like this:

interface loopback 0

ip address 123.12.12.187 255.255.255.255

no shut

crypto map SDM_CMAP_1 local-address loopback 0

I don't think you need to apply the crypto map to the loopback interface, but it's been a while since I configured something like this, so if you have any trouble try that first, and if still not working get the crypto debugs again (isakmp+ipsec on the vpn router, nat+packet on the client router).

hth

Herbert

View solution in original post

praprama · ‎10-12-2010

Hey Eric,

When connected to the VPN, please post the output of "show crypto isa sa" and "show crypto ipsec sa". Also, what ip addresses are you trying to connect to when connected over VPN and how exactly are you trying to connect to them? Are you able to ping the router's IP address of 10.10.7.1 form the VPN clients?

Thanks and Regards,

Prapanch

bautsche123 · ‎10-12-2010

Hi Prapanch.

Thanks for your help.

Here are the outputs from show crypto isa sa:

adsl#show crypto isa sa
IPv4 Crypto ISAKMP SA
dst src state conn-id slot status
123.12.12.185 10.10.7.115 QM_IDLE 2030 0 ACTIVE

IPv6 Crypto ISAKMP SA

adsl#

And the output from show crypto ipsec sa:

adsl#show crypto ipsec sa

interface: Dialer1
Crypto map tag: SDM_CMAP_1, local addr 123.12.12.185

   protected vrf: (none)
   local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (10.10.148.14/255.255.255.255/0/0)
   current_peer 10.10.7.115 port 500
     PERMIT, flags={}
    #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
    #pkts decaps: 21, #pkts decrypt: 21, #pkts verify: 21
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 8

     local crypto endpt.: 123.12.12.185, remote crypto endpt.: 10.10.7.115
     path mtu 1500, ip mtu 1500, ip mtu idb Dialer1
     current outbound spi: 0xFF2842BF(4280828607)

     inbound esp sas:
      spi: 0x7C9990A0(2090438816)
        transform: esp-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 285, flow_id: Motorola SEC 1.0:285, crypto map: SDM_CMAP_1
        sa timing: remaining key lifetime (k/sec): (4514693/3440)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

inbound ah sas:

inbound pcp sas:

     outbound esp sas:
      spi: 0xFF2842BF(4280828607)
        transform: esp-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 286, flow_id: Motorola SEC 1.0:286, crypto map: SDM_CMAP_1
        sa timing: remaining key lifetime (k/sec): (4514698/3440)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

outbound ah sas:

outbound pcp sas:
adsl#

I'm trying to connect to 10.10.140.93 by using ping. The ping (Echo request) packet arrives at that node and the node responds with an echo reply but that is then transmitted in the clear out of Vlan1 (when the VPN client is on the 10.10.7.0/24 network, I can't see what happens when the client is on the public internet side).

Re pinging 10.10.7.1, that address pings when the client is on the 10.10.7.0/24 network (obviously) whether VPN'ed in or not, but it does not ping when VPN'ed in from the public internet.

Thanks again for your help.

Eric

Namit Agarwal · ‎10-13-2010

Hi ,

I see that there is a crypto map applied on the interface VLAN 1. Are you trying to terminate any tunnel on that interface ? If no please take out that command and try once more.

thanks,

Namit

bautsche123 · ‎10-13-2010

I've removed the crypto map on Vlan1 but that hasn't changed the behaviour, unfortunately. ICMP pings still arrive at the host being pinged, but the replys never make it to the VPN client. :-(

Eric

Herbert Baerten · ‎10-13-2010

Eric,

I see you have 2 groups, one with and one without authentication - is that intentional? If not, remove one of the groups ance everything associated with it.

If it is your intention to have the 2 groups side by side, then you'll want to reference your isakmp profiles in your crypto map:

crypto dynamic-map SDM_DYNMAP_1 1
set isakmp profile sdm-ike-profile-1

crypto dynamic-map SDM_DYNMAP_1 2

set isakmp profile sdm-ike-profile-2

Still, this does not explain that you see the packets going out on the inside interface without encryption...

What is the source and destination IP address and MAC address of those reply packets?

Then check:

show ip route

Assuming you hace CEF enabled:

show ip cef detail

And apart from all that, you may want to upgrade to a more recent IOS version like 12.4(15)T14 - I'm not saying this will solve this problem but it wouldn't hurt to exclude the possibility you're running into a known bug.

hth

Herbert

bautsche123 · ‎10-14-2010

Hallo Herbert.

You are quite correct about useless stuff hanging around in the configuration, there were a few left overs from testing that I did, I have now cleaned up my configuration but unfortunately, it's still now working.

Here's the configuration as it is now and then I'll put the output from the show ip commands.

Configuration:

Current configuration : 7323 bytes
!
! Last configuration change at 07:58:13 bst Thu Oct 14 2010 by bautsche
! NVRAM config last updated at 07:58:14 bst Thu Oct 14 2010 by bautsche
!
version 12.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service sequence-numbers
!
hostname adsl
!
boot-start-marker
boot-end-marker
!
logging buffered 4096
enable secret 5
!
aaa new-model
!
!
aaa authentication login local_authen local
aaa authentication login sdm_vpn_xauth_ml_1 local
aaa authorization exec local_author local
aaa authorization network sdm_vpn_group_ml_1 local
!
!
aaa session-id common
clock timezone gmt 0
clock summer-time bst recurring last Sun Mar 1:00 last Sun Oct 1:00
!
!
dot11 syslog
no ip source-route
ip dhcp database dhcpinternal
no ip dhcp use vrf connected
ip dhcp excluded-address 10.10.7.1 10.10.7.99
ip dhcp excluded-address 10.10.7.151 10.10.7.255
!
ip dhcp pool dhcpinternal
   import all
   network 10.10.7.0 255.255.255.0
   default-router 10.10.7.1
   dns-server 212.159.6.9 212.159.6.10 212.159.13.49 212.159.13.50
!
!
ip cef
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
no ip bootp server
ip host nfs1 10.10.140.207
ip name-server 212.159.11.150
ip name-server 212.159.13.150
!
!
!
username cable password 7 091E4A210051
username bautsche password 7 0832495C1F4D0812
username vpnuser password 7 04764A051D2E7B161B1C
!
!
crypto isakmp policy 1
encr 3des
hash md5
authentication pre-share
group 2
crypto isakmp client configuration address-pool local SDM_POOL_1
!
crypto isakmp client configuration group
key
dns 10.10.140.201 10.10.140.202
domain
pool SDM_POOL_1
max-users 3
netmask 255.255.255.0
crypto isakmp profile sdm-ike-profile-1
   match identity group
   client authentication list sdm_vpn_xauth_ml_1
   isakmp authorization list sdm_vpn_group_ml_1
   client configuration address respond
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP_MD5_3DES esp-3des esp-md5-hmac
crypto ipsec transform-set ESP-AES-256-SHA esp-aes esp-sha-hmac
!
crypto dynamic-map SDM_DYNMAP_1 1
set security-association idle-time 3600
set transform-set ESP-AES-256-SHA
reverse-route
!
!
crypto map SDM_CMAP_1 client authentication list sdm_vpn_xauth_ml_1
crypto map SDM_CMAP_1 isakmp authorization list sdm_vpn_group_ml_1
crypto map SDM_CMAP_1 65535 ipsec-isakmp dynamic SDM_DYNMAP_1
!
crypto ctcp port 10000
archive
log config
hidekeys
!
!
ip tcp synwait-time 10
!
!
!
interface Null0
no ip unreachables
!
interface ATM0
no ip address
no ip redirects
no ip unreachables
no ip proxy-arp
ip route-cache flow
no atm ilmi-keepalive
pvc 0/38
encapsulation aal5mux ppp dialer
dialer pool-member 1
!
dsl operating-mode auto
hold-queue 224 in
!
interface FastEthernet0
!
interface FastEthernet1
!
interface FastEthernet2
!
interface FastEthernet3
!
interface Vlan1
description $FW_INSIDE$
ip address 10.10.7.1 255.255.255.0
ip access-group 121 in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat inside
ip virtual-reassembly
ip route-cache flow
hold-queue 100 out
!
interface Dialer1
description $FW_OUTSIDE$
ip address negotiated
ip access-group 121 in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat outside
ip virtual-reassembly
encapsulation ppp
ip route-cache flow
no ip split-horizon
dialer pool 1
dialer idle-timeout 0
dialer persistent
dialer-group 1
no cdp enable
ppp authentication chap callin
ppp chap hostname
ppp chap password 7
crypto map SDM_CMAP_1
!
ip local pool SDM_POOL_1 10.10.148.11 10.10.148.20
ip local pool public_184 123.10.10.184
ip local pool public_186 123.10.10.186
ip local pool public_187 123.10.10.187
ip local pool internal_9 10.10.7.9
ip local pool internal_8 10.10.7.8
ip local pool internal_223 10.10.7.223
ip local pool internal_47 10.10.7.47
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.10.140.0 255.255.255.0 10.10.7.2
!
no ip http server
no ip http secure-server
ip nat inside source route-map SDM_RMAP_1 interface Dialer1 overload
ip nat inside source static 10.10.7.9 123.10.10.184
ip nat inside source static tcp 10.10.7.8 22 123.10.10.185 22 extendable
ip nat inside source static tcp 10.10.7.8 25 123.10.10.185 25 extendable
ip nat inside source static tcp 10.10.7.8 80 123.10.10.185 80 extendable
ip nat inside source static tcp 10.10.7.8 443 123.10.10.185 443 extendable
ip nat inside source static tcp 10.10.7.8 993 123.10.10.185 993 extendable
ip nat inside source static tcp 10.10.7.8 1587 123.10.10.185 1587 extendable
ip nat inside source static tcp 10.10.7.8 8443 123.10.10.185 8443 extendable
ip nat inside source static 10.10.7.223 123.10.10.186
ip nat inside source static 10.10.7.47 123.10.10.187
!
logging 10.10.140.213
access-list 18 permit any
access-list 23 permit 10.10.140.0 0.0.0.255
access-list 23 permit 10.10.7.0 0.0.0.255
access-list 100 remark SDM_ACL Category=2
access-list 100 deny   ip any 10.10.148.0 0.0.0.255
access-list 100 permit ip any any
access-list 101 permit ip 10.10.148.0 0.0.0.255 any
access-list 101 permit ip any 10.10.148.0 0.0.0.255
access-list 101 deny   ip any any
access-list 121 remark SDM_ACL Category=17
access-list 121 deny   udp any eq netbios-dgm any
access-list 121 deny   udp any eq netbios-ns any
access-list 121 deny   udp any eq netbios-ss any
access-list 121 deny   tcp any eq 137 any
access-list 121 deny   tcp any eq 138 any
access-list 121 deny   tcp any eq 139 any
access-list 121 permit ip any any
access-list 125 permit tcp any any eq www
access-list 125 permit udp any eq isakmp any
access-list 125 permit udp any any eq isakmp
access-list 194 deny   udp any eq isakmp any
access-list 194 deny   udp any any eq isakmp
access-list 194 permit ip host 123.10.10.184 any
access-list 194 permit ip any host 123.10.10.184
access-list 194 permit ip host 10.10.7.9 any
access-list 194 permit ip any host 10.10.7.9
access-list 195 deny   udp any eq isakmp any
access-list 195 deny   udp any any eq isakmp
access-list 195 permit ip host 123.10.10.185 any
access-list 195 permit ip any host 123.10.10.185
access-list 195 permit ip host 10.10.7.8 any
access-list 195 permit ip any host 10.10.7.8
no cdp run
route-map public_185 permit 10
match ip address 195
!
route-map public_184 permit 10
match ip address 194
!
route-map SDM_RMAP_1 permit 1
match ip address 100
!
!
control-plane
!
!
line con 0
login authentication local_authen
no modem enable
transport preferred none
transport output telnet
stopbits 1
line aux 0
login authentication local_authen
transport output telnet
stopbits 1
line vty 0 4
access-class 23 in
privilege level 15
authorization exec local_author
login authentication local_authen
length 0
transport preferred none
transport input telnet ssh
!
scheduler max-task-time 5000
scheduler allocate 4000 1000
scheduler interval 500
sntp server 130.88.202.49
sntp server 130.88.200.98
sntp server 130.88.200.6
sntp server 130.88.203.64
end

And here the output from the show route commands (ip addresses change again, 122.1.1.142 is the IP of the client on the public internet):

adsl#show ip route 10.10.148.12
Routing entry for 10.10.148.12/32
Known via "static", distance 1, metric 0
Routing Descriptor Blocks:
* 122.1.1.142
Route metric is 0, traffic share count is 1

adsl#show ip route 10.10.140.93
Routing entry for 10.10.140.0/24
Known via "static", distance 1, metric 0
Routing Descriptor Blocks:
* 10.10.7.2
Route metric is 0, traffic share count is 1

adsl#sh ip cef 10.10.148.12 detail
10.10.148.12/32, version 27, epoch 0, cached adjacency to Dialer1
0 packets, 0 bytes
via 122.1.1.142, 0 dependencies, recursive
    next hop 122.1.1.142, Dialer1 via 0.0.0.0/0
    valid cached adjacency
adsl#show ip cef 10.10.140.93 detail
10.10.140.0/24, version 11, epoch 0, cached adjacency 10.10.7.2
0 packets, 0 bytes
via 10.10.7.2, 0 dependencies, recursive
    next hop 10.10.7.2, Vlan1 via 10.10.7.2/32
    valid cached adjacency
adsl#

I can still see the pings arrive and being responded to on the target system 10.10.140.93, by the way.

Thanks again for everyone's help, it's really appreciated.

Eric

Herbert Baerten · ‎10-14-2010

Eric,

could you still tie the ike profile to the crypto map :

crypto dynamic-map SDM_DYNMAP_1 1
set isakmp profile sdm-ike-profile-1

If that does not help, can you confirm that

- the echo reply packets arrive at the router, and are sent back out the vlan1 interface? If so what is the source and destination IP address and MAC address of the reply packets you see going out of vlan1

- "show crypto ipsec sa" shows no packets are being encrypted

Herbert

bautsche123 · ‎10-14-2010

Hello Herbert.

I have added set iskmp sdm-ike-profile-1 to the SDM_DYNMAP_1 1.

It's not fixed the issue, so re your other questions:

> can you confirm that the echo reply packets arrive at the router, and are sent back out the vlan1 interface?

I can't really as I don't know how to make the router tell me. What I can tell you is that my firewall passes them back to the router:

This is a snoop of the interface of the firewall connected to vlan 10.10.7.0/24:

[...]

vpn-adsl-01 -> oberon ICMP Echo request (ID: 949 Sequence number: 49)
oberon -> vpn-adsl-01 ICMP Echo reply (ID: 949 Sequence number: 49)

[...]

And this is the firewall's routing table (extract):

root@tethys # netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
[....]

10.10.7.0 10.10.7.2 U 1 0 xnf1
[....]

> If so what is the source and destination IP address and MAC address of the reply packets you see going out of vlan1

I exect that the reply packets go out of Dialer1 rather than vlan1 given the routing configuration on the router:

adsl#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

     123.0.0.0/32 is subnetted, 1 subnets
C       123.1.1.185 is directly connected, Dialer1
     10.10.148.0/32 is subnetted, 1 subnets
S       10.10.148.11 [1/0] via 83.217.167.142
     180.16.128.0/32 is subnetted, 1 subnets
C       180.16.128.227 is directly connected, Dialer1
S    10.10.140.0/24 [1/0] via 10.10.7.2
C    10.10.7.0/24 is directly connected, Vlan1
S*   0.0.0.0/0 is directly connected, Dialer1

> "show crypto ipsec sa" shows no packets are being encrypted

adsl#show crypto ipsec sa

interface: Dialer1
Crypto map tag: SDM_CMAP_1, local addr 84.92.202.185

   protected vrf: (none)
   local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (10.10.148.11/255.255.255.255/0/0)
   current_peer port 500
     PERMIT, flags={}
    #pkts encaps: 529, #pkts encrypt: 529, #pkts digest: 529
    #pkts decaps: 529, #pkts decrypt: 529, #pkts verify: 529
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 123.1.1.185, remote crypto endpt.:
     path mtu 1500, ip mtu 1500, ip mtu idb Dialer1
     current outbound spi: 0x125571BF(307589567)

     inbound esp sas:
      spi: 0x49714BA8(1232161704)
        transform: esp-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 1, flow_id: Motorola SEC 1.0:1, crypto map: SDM_CMAP_1
        sa timing: remaining key lifetime (k/sec): (4501781/3216)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

inbound ah sas:

inbound pcp sas:

     outbound esp sas:
      spi: 0x125571BF(307589567)
        transform: esp-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2, flow_id: Motorola SEC 1.0:2, crypto map: SDM_CMAP_1
        sa timing: remaining key lifetime (k/sec): (4501770/3214)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

outbound ah sas:

outbound pcp sas:
adsl#

I'm afraid I'm not clear on how to read the output of that command though...

Sorry.

Eric

Herbert Baerten · ‎10-14-2010

Hi Eric

well I think the problem with the external client may be different than the one with the client on the inside.

In the latest output of the external client we see:

#pkts encaps: 529, #pkts encrypt: 529, #pkts digest: 529

#pkts decaps: 529, #pkts decrypt: 529, #pkts verify: 529

I.e. it is encrypting as many packets as it is decrypting - which seems to indicate that the echo replies get encrypted ok.

So can you please check your client, does it have similar counters for packets sent/received (encrypted/decrypted)?

Or could you sniff the traffic on the client to see if it is receiving IPsec (ESP) packets?

Is the external client behind a NAT device?

Could it be that the ISP to which the VPNrouter is connected, does not allow outbound IPsec packets?

Herbert

bautsche123 · ‎10-14-2010

Thanks for your help.

You have made me check again something I had checked earlier, ie. the snoop on the client. Earlier, the client simply didn't get any packets back, but now it seems it does but doesn't appear know what to do with them (so we - you really - have fixed something, which is encouraging):

10.10.40.104 -> 123.1.1.185 ESP SPI=0x824af60f Replay=56
123.1.1.185 -> 10.10.40.104 ESP SPI=0x583b2299 Replay=55
10.10.40.104 -> 123.1.1.185 ICMP Destination unreachable (Bad protocol 50)
10.10.40.104 -> 123.1.1.185 ESP SPI=0x824af60f Replay=57
123.1.1.185 -> 10.10.40.104 ESP SPI=0x583b2299 Replay=56
10.10.40.104 -> 123.1.1.185 ICMP Destination unreachable (Bad protocol 50)
10.10.40.104 -> 123.1.1.185 ESP SPI=0x824af60f Replay=58
10.10.40.104 -> 123.1.1.185 ESP SPI=0x824af60f Replay=59
123.1.1.185 -> 10.10.40.104 ESP SPI=0x583b2299 Replay=57
10.10.40.104 -> 123.1.1.185 ICMP Destination unreachable (Bad protocol 50)
123.1.1.185 -> 10.10.40.104 ESP SPI=0x583b2299 Replay=58
10.10.40.104 -> 123.1.1.185 ICMP Destination unreachable (Bad protocol 50)

The ICMP Destination unreachable packets basically complain about the ESP packet that just came in with "Bad protocol".

The client is behind a NAT device (actually a Cisco 871 ADSL router). It's not likely that the ISP is having an issue with IPsec though as I can VPN into my employer's VPN service using the same tools just fine.

Thanks again for your persistence in helping me to debug this.

Eric

Herbert Baerten · ‎10-14-2010

Ok, glad to see we're making progress

Now I asked about the ISP and the NAT device because I thought they might be blackholing the return packets, but your snoop confirms that they effectively arrive on the client, so this is now definitely a client issue.

Googling the unreachable message, I found this:

http://thestaticvoid.com/post/2010/07/22/fun-with-vpnc/

So could you please try to run vpnc with --natt-mode force-natt

hth

Herbert

bautsche123 · ‎10-14-2010

Hi Herbert.

Thanks again for your help.

I was using cisco-udp as the NAT mode. I have changed it to force-natt but that unfortunately doesn't establish the tunnel. Here is the debug output from the Cisco router when trying with force-natt (IP addresses altered again). vpnc exits with "no response from target" after waiting about 30secs:

000631: Oct 14 12:49:35.282 bst: ISAKMP:(2006):purging node 427214974
000632: Oct 14 12:49:35.282 bst: ISAKMP:(2006):purging node 2060102796
000633: Oct 14 12:49:35.282 bst: ISAKMP:(2006):purging node -1191592158
000634: Oct 14 12:49:35.282 bst: ISAKMP:(2006):purging node 228
000635: Oct 14 12:49:35.282 bst: ISAKMP:(2006):purging node 195
000636: Oct 14 12:49:37.465 bst: ISAKMP (0:0): received packet from 120.3.3.142 dport 500 sport 500 Global (N) NEW SA
000637: Oct 14 12:49:37.465 bst: ISAKMP: Created a peer struct for 120.3.3.142, peer port 500
000638: Oct 14 12:49:37.465 bst: ISAKMP: New peer created peer = 0x82F6A850 peer_handle = 0x80000009
000639: Oct 14 12:49:37.465 bst: ISAKMP: Locking peer struct 0x82F6A850, refcount 1 for crypto_isakmp_process_block
000640: Oct 14 12:49:37.465 bst: ISAKMP:(0):Setting client config settings 82C3B344
000641: Oct 14 12:49:37.465 bst: ISAKMP:(0):(Re)Setting client xauth list and state
000642: Oct 14 12:49:37.465 bst: ISAKMP/xauth: initializing AAA request
000643: Oct 14 12:49:37.465 bst: ISAKMP: local port 500, remote port 500
000644: Oct 14 12:49:37.469 bst: ISAKMP: Find a dup sa in the avl tree during calling isadb_insert sa = 8210A588
000645: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing SA payload. message ID = 0
000646: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing ID payload. message ID = 0
000647: Oct 14 12:49:37.469 bst: ISAKMP (0:0): ID payload
        next-payload : 13
        type         : 11
        group id     : groupname
        protocol     : 17
        port         : 500
        length       : 18
000648: Oct 14 12:49:37.469 bst: ISAKMP:(0):: peer matches sdm-ike-profile-1 profile
000649: Oct 14 12:49:37.469 bst: ISAKMP:(0):(Re)Setting client xauth list sdm_vpn_xauth_ml_1 and state
000650: Oct 14 12:49:37.469 bst: ISAKMP/xauth: initializing AAA request
000651: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing vendor id payload
000652: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID seems Unity/DPD but major 242 mismatch
000653: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID is XAUTH
000654: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing vendor id payload
000655: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID is Unity
000656: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing vendor id payload
000657: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch
000658: Oct 14 12:49:37.469 bst: ISAKMP (0:0): vendor ID is NAT-T RFC 3947
000659: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing vendor id payload
000660: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID seems Unity/DPD but major 123 mismatch
000661: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID is NAT-T v2
000662: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing vendor id payload
000663: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID seems Unity/DPD but major 164 mismatch
000664: Oct 14 12:49:37.469 bst: ISAKMP:(0): processing vendor id payload
000665: Oct 14 12:49:37.469 bst: ISAKMP:(0): vendor ID seems Unity/DPD but major 168 mismatch
000666: Oct 14 12:49:37.473 bst: ISAKMP:(0): processing vendor id payload
000667: Oct 14 12:49:37.473 bst: ISAKMP:(0): vendor ID seems Unity/DPD but major 221 mismatch
000668: Oct 14 12:49:37.473 bst: ISAKMP:(0): processing vendor id payload
000669: Oct 14 12:49:37.473 bst: ISAKMP:(0): vendor ID is DPD
000670: Oct 14 12:49:37.473 bst: ISAKMP:(0): Authentication by xauth preshared
000671: Oct 14 12:49:37.473 bst: ISAKMP:(0):Checking ISAKMP transform 0 against priority 1 policy
000672: Oct 14 12:49:37.473 bst: ISAKMP:      keylength of 256
000673: Oct 14 12:49:37.473 bst: ISAKMP:      encryption AES-CBC
000674: Oct 14 12:49:37.473 bst: ISAKMP:      hash SHA
000675: Oct 14 12:49:37.473 bst: ISAKMP:      default group 2
000676: Oct 14 12:49:37.473 bst: ISAKMP:      auth XAUTHInitPreShared
000677: Oct 14 12:49:37.473 bst: ISAKMP:      life type in seconds
000678: Oct 14 12:49:37.473 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000679: Oct 14 12:49:37.473 bst: ISAKMP:(0):Encryption algorithm offered does not match policy!
000680: Oct 14 12:49:37.473 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000681: Oct 14 12:49:37.473 bst: ISAKMP:(0):Checking ISAKMP transform 1 against priority 1 policy
000682: Oct 14 12:49:37.473 bst: ISAKMP:      keylength of 256
000683: Oct 14 12:49:37.473 bst: ISAKMP:      encryption AES-CBC
000684: Oct 14 12:49:37.473 bst: ISAKMP:      hash MD5
000685: Oct 14 12:49:37.473 bst: ISAKMP:      default group 2
000686: Oct 14 12:49:37.473 bst: ISAKMP:      auth XAUTHInitPreShared
000687: Oct 14 12:49:37.473 bst: ISAKMP:      life type in seconds
000688: Oct 14 12:49:37.473 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000689: Oct 14 12:49:37.473 bst: ISAKMP:(0):Encryption algorithm offered does not match policy!
000690: Oct 14 12:49:37.473 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000691: Oct 14 12:49:37.473 bst: ISAKMP:(0):Checking ISAKMP transform 2 against priority 1 policy
000692: Oct 14 12:49:37.473 bst: ISAKMP:      keylength of 192
000693: Oct 14 12:49:37.473 bst: ISAKMP:      encryption AES-CBC
000694: Oct 14 12:49:37.473 bst: ISAKMP:      hash SHA
000695: Oct 14 12:49:37.473 bst: ISAKMP:      default group 2
000696: Oct 14 12:49:37.473 bst: ISAKMP:      auth XAUTHInitPreShared
000697: Oct 14 12:49:37.477 bst: ISAKMP:      life type in seconds
000698: Oct 14 12:49:37.477 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000699: Oct 14 12:49:37.477 bst: ISAKMP:(0):Encryption algorithm offered does not match policy!
000700: Oct 14 12:49:37.477 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000701: Oct 14 12:49:37.477 bst: ISAKMP:(0):Checking ISAKMP transform 3 against priority 1 policy
000702: Oct 14 12:49:37.477 bst: ISAKMP:      keylength of 192
000703: Oct 14 12:49:37.477 bst: ISAKMP:      encryption AES-CBC
000704: Oct 14 12:49:37.477 bst: ISAKMP:      hash MD5
000705: Oct 14 12:49:37.477 bst: ISAKMP:      default group 2
000706: Oct 14 12:49:37.477 bst: ISAKMP:      auth XAUTHInitPreShared
000707: Oct 14 12:49:37.477 bst: ISAKMP:      life type in seconds
000708: Oct 14 12:49:37.477 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000709: Oct 14 12:49:37.477 bst: ISAKMP:(0):Encryption algorithm offered does not match policy!
000710: Oct 14 12:49:37.477 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000711: Oct 14 12:49:37.477 bst: ISAKMP:(0):Checking ISAKMP transform 4 against priority 1 policy
000712: Oct 14 12:49:37.477 bst: ISAKMP:      keylength of 128
000713: Oct 14 12:49:37.477 bst: ISAKMP:      encryption AES-CBC
000714: Oct 14 12:49:37.477 bst: ISAKMP:      hash SHA
000715: Oct 14 12:49:37.477 bst: ISAKMP:      default group 2
000716: Oct 14 12:49:37.477 bst: ISAKMP:      auth XAUTHInitPreShared
000717: Oct 14 12:49:37.477 bst: ISAKMP:      life type in seconds
000718: Oct 14 12:49:37.477 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000719: Oct 14 12:49:37.477 bst: ISAKMP:(0):Encryption algorithm offered does not match policy!
000720: Oct 14 12:49:37.477 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000721: Oct 14 12:49:37.477 bst: ISAKMP:(0):Checking ISAKMP transform 5 against priority 1 policy
000722: Oct 14 12:49:37.477 bst: ISAKMP:      keylength of 128
000723: Oct 14 12:49:37.477 bst: ISAKMP:      encryption AES-CBC
000724: Oct 14 12:49:37.477 bst: ISAKMP:      hash MD5
000725: Oct 14 12:49:37.477 bst: ISAKMP:      default group 2
000726: Oct 14 12:49:37.477 bst: ISAKMP:      auth XAUTHInitPreShared
000727: Oct 14 12:49:37.477 bst: ISAKMP:      life type in seconds
000728: Oct 14 12:49:37.477 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000729: Oct 14 12:49:37.477 bst: ISAKMP:(0):Encryption algorithm offered does not match policy!
000730: Oct 14 12:49:37.481 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000731: Oct 14 12:49:37.481 bst: ISAKMP:(0):Checking ISAKMP transform 6 against priority 1 policy
000732: Oct 14 12:49:37.481 bst: ISAKMP:      encryption 3DES-CBC
000733: Oct 14 12:49:37.481 bst: ISAKMP:      hash SHA
000734: Oct 14 12:49:37.481 bst: ISAKMP:      default group 2
000735: Oct 14 12:49:37.481 bst: ISAKMP:      auth XAUTHInitPreShared
000736: Oct 14 12:49:37.481 bst: ISAKMP:      life type in seconds
000737: Oct 14 12:49:37.481 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000738: Oct 14 12:49:37.481 bst: ISAKMP:(0):Hash algorithm offered does not match policy!
000739: Oct 14 12:49:37.481 bst: ISAKMP:(0):atts are not acceptable. Next payload is 3
000740: Oct 14 12:49:37.481 bst: ISAKMP:(0):Checking ISAKMP transform 7 against priority 1 policy
000741: Oct 14 12:49:37.481 bst: ISAKMP:      encryption 3DES-CBC
000742: Oct 14 12:49:37.481 bst: ISAKMP:      hash MD5
000743: Oct 14 12:49:37.481 bst: ISAKMP:      default group 2
000744: Oct 14 12:49:37.481 bst: ISAKMP:      auth XAUTHInitPreShared
000745: Oct 14 12:49:37.481 bst: ISAKMP:      life type in seconds
000746: Oct 14 12:49:37.481 bst: ISAKMP:      life duration (VPI) of 0x0 0x20 0xC4 0x9B
000747: Oct 14 12:49:37.481 bst: ISAKMP:(0):atts are acceptable. Next payload is 3
000748: Oct 14 12:49:37.481 bst: ISAKMP:(0):Acceptable atts:actual life: 86400
000749: Oct 14 12:49:37.481 bst: ISAKMP:(0):Acceptable atts:life: 0
000750: Oct 14 12:49:37.481 bst: ISAKMP:(0):Fill atts in sa vpi_length:4
000751: Oct 14 12:49:37.481 bst: ISAKMP:(0):Fill atts in sa life_in_seconds:2147483
000752: Oct 14 12:49:37.481 bst: ISAKMP:(0):Returning Actual lifetime: 86400
000753: Oct 14 12:49:37.481 bst: ISAKMP:(0)::Started lifetime timer: 86400.

000754: Oct 14 12:49:37.481 bst: ISAKMP:(0): processing KE payload. message ID = 0
000755: Oct 14 12:49:37.525 bst: ISAKMP:(0): processing NONCE payload. message ID = 0
000756: Oct 14 12:49:37.525 bst: ISAKMP (0:0): vendor ID is NAT-T RFC 3947
000757: Oct 14 12:49:37.525 bst: ISAKMP:(0): vendor ID is NAT-T v2
000758: Oct 14 12:49:37.529 bst: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_AM_EXCH
000759: Oct 14 12:49:37.529 bst: ISAKMP:(0):Old State = IKE_READY New State = IKE_R_AM_AAA_AWAIT

000760: Oct 14 12:49:37.529 bst: ISAKMP:(2007): constructed NAT-T vendor-rfc3947 ID
000761: Oct 14 12:49:37.529 bst: ISAKMP:(2007):SA is doing pre-shared key authentication plus XAUTH using id type ID_IPV4_ADDR
000762: Oct 14 12:49:37.529 bst: ISAKMP (0:2007): ID payload
        next-payload : 10
        type         : 1
        address      : 123.1.1.185
        protocol     : 17
        port         : 0
        length       : 12
000763: Oct 14 12:49:37.529 bst: ISAKMP:(2007):Total payload length: 12
000764: Oct 14 12:49:37.533 bst: ISAKMP:(2007): sending packet to 120.3.3.142 my_port 500 peer_port 500 (R) AG_INIT_EXCH
000765: Oct 14 12:49:37.533 bst: ISAKMP:(2007):Sending an IKE IPv4 Packet.
000766: Oct 14 12:49:37.533 bst: ISAKMP:(2007):Input = IKE_MESG_FROM_AAA, PRESHARED_KEY_REPLY
000767: Oct 14 12:49:37.533 bst: ISAKMP:(2007):Old State = IKE_R_AM_AAA_AWAIT New State = IKE_R_AM2

000768: Oct 14 12:49:37.713 bst: ISAKMP (0:2007): received packet from 120.3.3.142 dport 4500 sport 4500 Global (R) AG_INIT_EXCH
000769: Oct 14 12:49:37.717 bst: ISAKMP:(2007): processing HASH payload. message ID = 0
000770: Oct 14 12:49:37.717 bst: ISAKMP:(2007): processing NOTIFY INITIAL_CONTACT protocol 1
        spi 0, message ID = 0, sa = 8210A588
000771: Oct 14 12:49:37.717 bst: ISAKMP:received payload type 20
000772: Oct 14 12:49:37.717 bst: ISAKMP (0:2007): NAT found, the node inside NAT
000773: Oct 14 12:49:37.717 bst: ISAKMP:received payload type 20
000774: Oct 14 12:49:37.717 bst: ISAKMP (0:2007): NAT found, both nodes are all located inside NAT
000775: Oct 14 12:49:37.717 bst: ISAKMP:(2007):SA authentication status:
        authenticated
000776: Oct 14 12:49:37.717 bst: ISAKMP:(2007):SA has been authenticated with 120.3.3.142
000777: Oct 14 12:49:37.717 bst: ISAKMP:(2007):Detected port,floating to port = 4500
000778: Oct 14 12:49:37.717 bst: ISAKMP: Trying to find existing peer 123.1.1.185/120.3.3.142/4500/ and found existing peer 82E01CCC to reuse, free 82F6A850
000779: Oct 14 12:49:37.717 bst: ISAKMP: Unlocking peer struct 0x82F6A850 Reuse existing peer, count 0
000780: Oct 14 12:49:37.717 bst: ISAKMP: Deleting peer node by peer_reap for 120.3.3.142: 82F6A850
000781: Oct 14 12:49:37.717 bst: ISAKMP: Locking peer struct 0x82E01CCC, refcount 2 for Reuse existing peer
000782: Oct 14 12:49:37.717 bst: ISAKMP:(2007):SA authentication status:
        authenticated
000783: Oct 14 12:49:37.717 bst: ISAKMP:(2007): Process initial contact,
bring down existing phase 1 and 2 SA's with local 123.1.1.185 remote 120.3.3.142 remote port 4500
000784: Oct 14 12:49:37.717 bst: ISAKMP:(2007):returning IP addr to the address pool: 10.10.148.11
000785: Oct 14 12:49:37.717 bst: ISAKMP (0:2007): returning address 10.10.148.11 to pool
000786: Oct 14 12:49:37.721 bst: ISAKMP:(2005):received initial contact, deleting SA
000787: Oct 14 12:49:37.721 bst: ISAKMP:(2005):peer does not do paranoid keepalives.

000788: Oct 14 12:49:37.721 bst: ISAKMP:(2005):peer does not do paranoid keepalives.

000789: Oct 14 12:49:37.721 bst: ISAKMP:(2005):deleting SA reason "Receive initial contact" state (R) CONF_XAUTH (peer 120.3.3.142)
000790: Oct 14 12:49:37.721 bst: ISAKMP:(2007):Setting UDP ENC peer struct 0x0 sa= 0x8210A588
000791: Oct 14 12:49:37.721 bst: ISAKMP:(2007):Returning Actual lifetime: 86400
000792: Oct 14 12:49:37.721 bst: ISAKMP: set new node 38746371 to CONF_XAUTH
000793: Oct 14 12:49:37.721 bst: ISAKMP:(2007):Sending NOTIFY RESPONDER_LIFETIME protocol 1
spi 2192666280, message ID = 38746371
000794: Oct 14 12:49:37.721 bst: ISAKMP:(2007): sending packet to 120.3.3.142 my_port 4500 peer_port 4500 (R) QM_IDLE
000795: Oct 14 12:49:37.721 bst: ISAKMP:(2007):Sending an IKE IPv4 Packet.
000796: Oct 14 12:49:37.721 bst: ISAKMP:(2007):purging node 38746371
000797: Oct 14 12:49:37.721 bst: ISAKMP: Sending phase 1 responder lifetime 86400

000798: Oct 14 12:49:37.725 bst: ISAKMP:(2007):Input = IKE_MESG_FROM_PEER, IKE_AM_EXCH
000799: Oct 14 12:49:37.725 bst: ISAKMP:(2007):Old State = IKE_R_AM2 New State = IKE_P1_COMPLETE

000800: Oct 14 12:49:37.725 bst: IPSEC(key_engine): got a queue event with 1 KMI message(s)
000801: Oct 14 12:49:37.725 bst: IPSEC(key_engine): got a queue event with 1 KMI message(s)
000802: Oct 14 12:49:37.725 bst: IPSEC(key_engine): got a queue event with 1 KMI message(s)
000803: Oct 14 12:49:37.725 bst: IPSEC(key_engine_delete_sas): rec'd delete notify from ISAKMP
000804: Oct 14 12:49:37.725 bst: IPSEC(key_engine_delete_sas): delete all SAs shared with peer 120.3.3.142
000805: Oct 14 12:49:37.725 bst: ISAKMP: set new node -497880832 to CONF_XAUTH
000806: Oct 14 12:49:37.729 bst: ISAKMP:(2005): sending packet to 120.3.3.142 my_port 4500 peer_port 4500 (R) CONF_XAUTH
000807: Oct 14 12:49:37.729 bst: ISAKMP:(2005):Sending an IKE IPv4 Packet.
000808: Oct 14 12:49:37.729 bst: ISAKMP:(2005):purging node -497880832
000809: Oct 14 12:49:37.729 bst: ISAKMP:(2005):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL
000810: Oct 14 12:49:37.729 bst: ISAKMP:(2005):Old State = IKE_XAUTH_REQ_SENT New State = IKE_DEST_SA

000811: Oct 14 12:49:37.729 bst: ISAKMP:(2007):Need XAUTH
000812: Oct 14 12:49:37.729 bst: ISAKMP: set new node -1159538072 to CONF_XAUTH
000813: Oct 14 12:49:37.729 bst: ISAKMP/xauth: request attribute XAUTH_USER_NAME_V2
000814: Oct 14 12:49:37.729 bst: ISAKMP/xauth: request attribute XAUTH_USER_PASSWORD_V2
000815: Oct 14 12:49:37.729 bst: ISAKMP:(2007): initiating peer config to 120.3.3.142. ID = -1159538072
000816: Oct 14 12:49:37.729 bst: ISAKMP:(2007): sending packet to 120.3.3.142 my_port 4500 peer_port 4500 (R) CONF_XAUTH
000817: Oct 14 12:49:37.729 bst: ISAKMP:(2007):Sending an IKE IPv4 Packet.
000818: Oct 14 12:49:37.733 bst: ISAKMP:(2007):Input = IKE_MESG_INTERNAL, IKE_PHASE1_COMPLETE
000819: Oct 14 12:49:37.733 bst: ISAKMP:(2007):Old State = IKE_P1_COMPLETE New State = IKE_XAUTH_REQ_SENT

000820: Oct 14 12:49:37.733 bst: ISAKMP:(2005):deleting SA reason "Receive initial contact" state (R) CONF_XAUTH (peer 120.3.3.142)
000821: Oct 14 12:49:37.733 bst: ISAKMP:(0):Can't decrement IKE Call Admission Control stat incoming_active since it's already 0.
000822: Oct 14 12:49:37.733 bst: ISAKMP: Unlocking peer struct 0x82E01CCC for isadb_mark_sa_deleted(), count 1
000823: Oct 14 12:49:37.733 bst: ISAKMP:(2005):deleting node -565811019 error FALSE reason "IKE deleted"
000824: Oct 14 12:49:37.733 bst: ISAKMP:(2005):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH
000825: Oct 14 12:49:37.733 bst: ISAKMP:(2005):Old State = IKE_DEST_SA New State = IKE_DEST_SA

000826: Oct 14 12:49:39.724 bst: ISAKMP (0:2007): received packet from 120.3.3.142 dport 4500 sport 4500 Global (R) CONF_XAUTH
000827: Oct 14 12:49:39.724 bst: ISAKMP:(2007): phase 1 packet is a duplicate of a previous packet.
000828: Oct 14 12:49:39.724 bst: ISAKMP:(2007): retransmitting due to retransmit phase 1
000829: Oct 14 12:49:39.724 bst: ISAKMP:(2007): no outgoing phase 1 packet to retransmit. CONF_XAUTH
000830: Oct 14 12:49:40.248 bst: ISAKMP:(2001):purging SA., sa=82C3A740, delme=82C3A740
000831: Oct 14 12:49:40.248 bst: ISAKMP:(2001):purging node 420788257
000832: Oct 14 12:49:40.248 bst: ISAKMP:(2001):purging node -330594349
000833: Oct 14 12:49:43.734 bst: ISAKMP (0:2007): received packet from 120.3.3.142 dport 4500 sport 4500 Global (R) CONF_XAUTH
000834: Oct 14 12:49:43.734 bst: ISAKMP:(2007): phase 1 packet is a duplicate of a previous packet.
000835: Oct 14 12:49:43.734 bst: ISAKMP:(2007): retransmitting due to retransmit phase 1
000836: Oct 14 12:49:43.734 bst: ISAKMP:(2007): no outgoing phase 1 packet to retransmit. CONF_XAUTH
000837: Oct 14 12:49:45.278 bst: ISAKMP:(2006):purging SA., sa=82F39980, delme=82F39980
000838: Oct 14 12:49:45.278 bst: ISAKMP:(2006):purging node 1765509531
000839: Oct 14 12:49:45.278 bst: ISAKMP:(2006):purging node -1220737993
000840: Oct 14 12:49:51.746 bst: ISAKMP (0:2007): received packet from 120.3.3.142 dport 4500 sport 4500 Global (R) CONF_XAUTH
000841: Oct 14 12:49:51.746 bst: ISAKMP:(2007): phase 1 packet is a duplicate of a previous packet.
000842: Oct 14 12:49:51.746 bst: ISAKMP:(2007): retransmitting due to retransmit phase 1
000843: Oct 14 12:49:51.746 bst: ISAKMP:(2007): no outgoing phase 1 packet to retransmit. CONF_XAUTH

Herbert Baerten · ‎10-14-2010

Ok, this is getting more and more intriguing

I see from the debugs that vpnc is using AM (agressive mode), and at http://www.unix-ag.uni-kl.de/~massar/vpnc/ I read that it does not support MM (main mode).

Now what should happen with AM, is:

Initiator sends AM1 on UDP 500

Responder sends AM2 on UDP 500

Initiator sends AM3 on UDP 4500

=> phase 1 complete

Responder sends Xauth request on UDP 4500

Initiator sends Xauth response etc. etc.

However here we see:

Initiator sends AM1 on UDP 500 - ok

Responder sends AM2 on UDP 500 - ok

Initiator sends AM3 on UDP 4500 - ok

=> phase 1 complete - ok

Responder sends Xauth request on UDP 4500 - ok

Initiator re-sends AM3 .... not ok

So it looks like the client (initiator) receives AM2 (on port 500) but not the Xauth request (on port 4500).

Another snoop on the client would confirm if it is receiving any UDP 4500 ?

Maybe your router is blocking this port?

Also, if you say you can connect to another VPN router then perhaps snooping a working connection there can learn us more. Are you also using cisco-udp for that connection? Is it also an IOS router you connect to?

Herbert

bautsche123 · ‎10-14-2010

Hi Herbert.

When vpnc is configured with force-natt I can see exactly these packets:

10.10.40.106 -> 123.1.1.185 UDP D=500 S=500 LEN=1294

123.1.1.185 -> 10.10.40.106 UDP D=500 S=500 LEN=412

10.10.40.106 -> 123.1.1.185 UDP D=4500 S=4500 LEN=168

The last packet repeats a few times more and then vpnc gives up.

Interestingly, if I leave the nat option off the vpnc config it doesn't act any differently to the above.

I see nothing in the other Cisco's router that blocks port 4500 anywhere.

When snooping the interface observing a successful connection to my company's VPN, I can see this (IP addresses altered):

10.10.40.106 -> 210.21.21.123 UDP D=500 S=500 LEN=842
210.21.21.123 -> 10.10.40.106 UDP IP fragment ID=151 Offset=0 MF=1 TOS=0x0 TTL=246
210.21.21.123 -> 10.10.40.106 UDP IP fragment ID=151 Offset=1480 MF=0 TOS=0x0 TTL=246
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=232
210.21.21.123 -> 10.10.40.106 UDP D=4500 S=4500 LEN=88
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=104
210.21.21.123 -> 10.10.40.106 UDP D=4500 S=4500 LEN=80
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=80
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=184
210.21.21.123 -> 10.10.40.106 UDP D=4500 S=4500 LEN=224

There is no option covering NAT in that config file at all. I don't know what the hardware I'm terminating on is, but rumour has it it's CISCO kit, but what it is, I couldn't say (or even find out).

Eric