cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
16427
Views
0
Helpful
40
Replies

VPN return traffic not flowing over the tunnel

bautsche123
Level 1
Level 1

Hello.

I've tried to find something on the internet to solve this, but am failing miserably. I guess, I really don't understand how the cisco decides on routing.

Anyway, I have a Cisco 837 which I'm using for internet access and which I would like to be able to terminate a VPN on. When I vpn in (using vpnc from a Solaris box as it happens which is connected to the ethernet interface of the cisco), I can establish an VPN and when I ping a host on the inside, I can see that ping packet arrive, however the return packet, the cisco 837 tries to send over the public internet facing interface Dialer1 without encryption. I can't for the life of me work out why.

(Also note: I can also establish a tunnel from the public internet, but again, I can't get any traffic back through the tunnel. I assume that I'm having the same issue, ie return packets aren't going where they should, but I don't know that for certain, on the host being pinged though, I can see the ping packets arriving and the host responding with an ICMP Echo reply).

here is the cisco version:

adsl#show version
Cisco IOS Software, C850 Software (C850-ADVSECURITYK9-M), Version 12.4(15)T5, RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 01-May-08 02:07 by prod_rel_team

ROM: System Bootstrap, Version 12.3(8r)YI4, RELEASE SOFTWARE

adsl uptime is 1 day, 19 hours, 27 minutes
System returned to ROM by power-on
System restarted at 17:20:56 bst Sun Oct 10 2010
System image file is "flash:c850-advsecurityk9-mz.124-15.T5.bin"

Cisco 857 (MPC8272) processor (revision 0x300) with 59392K/6144K bytes of memory.
Processor board ID FCZ122391F5
MPC8272 CPU Rev: Part Number 0xC, Mask Number 0x10
4 FastEthernet interfaces
1 ATM interface
128K bytes of non-volatile configuration memory.
20480K bytes of processor board System flash (Intel Strataflash)

Configuration register is 0x2102

And here is the cisco's configuration (IP address, etc changed of course):

Current configuration : 7782 bytes
!
! Last configuration change at 11:57:21 bst Mon Oct 11 2010 by bautsche
! NVRAM config last updated at 11:57:22 bst Mon Oct 11 2010 by bautsche
!
version 12.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service sequence-numbers
!
hostname adsl
!
boot-start-marker
boot-end-marker
!
logging buffered 4096
enable secret 5 <secret>
!
aaa new-model
!
!
aaa authentication login local_authen local
aaa authentication login sdm_vpn_xauth_ml_1 local
aaa authorization exec local_author local
aaa authorization network sdm_vpn_group_ml_1 local
!
!
aaa session-id common
clock timezone gmt 0
clock summer-time bst recurring last Sun Mar 1:00 last Sun Oct 1:00
!
!
dot11 syslog
no ip source-route
ip dhcp database dhcpinternal
no ip dhcp use vrf connected
ip dhcp excluded-address 10.10.7.1 10.10.7.99
ip dhcp excluded-address 10.10.7.151 10.10.7.255
!
ip dhcp pool dhcpinternal
   import all
   network 10.10.7.0 255.255.255.0
   default-router 10.10.7.1
   dns-server 212.159.6.9 212.159.6.10 212.159.13.49 212.159.13.50
!
!
ip cef
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
no ip bootp server
ip host nfs1 10.10.140.207
ip name-server 212.159.11.150
ip name-server 212.159.13.150
!
!
!
username cable password 7 <password>
username bautsche password 7 <password>
username vpnuser password 7 <password>
!
!
crypto isakmp policy 1
encr 3des
hash md5
authentication pre-share
group 2
!
crypto isakmp policy 2
encr aes 256
authentication pre-share
group 2
!
crypto isakmp policy 3
encr 3des
authentication pre-share group 2
crypto isakmp client configuration address-pool local SDM_POOL_1
!
crypto isakmp client configuration group groupname2
key <key>
dns 10.10.140.201 10.10.140.202
domain swangage.co.uk
pool SDM_POOL_1
max-users 3
netmask 255.255.255.0
!
crypto isakmp client configuration group groupname1
key <key>
dns 10.10.140.201 10.10.140.202
domain swangage.co.uk
pool SDM_POOL_1
max-users 3
netmask 255.255.255.0
crypto isakmp profile sdm-ike-profile-1
   match identity group groupname2
   client authentication list sdm_vpn_xauth_ml_1
   isakmp authorization list sdm_vpn_group_ml_1
   client configuration address respond
crypto isakmp profile sdm-ike-profile-2
   match identity group groupname1
   isakmp authorization list sdm_vpn_group_ml_1
   client configuration address respond
!
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP_MD5_3DES esp-3des esp-md5-hmac
crypto ipsec transform-set ESP-AES-256-SHA esp-aes esp-sha-hmac
!
crypto dynamic-map SDM_DYNMAP_1 1
set security-association idle-time 3600
set transform-set ESP-AES-256-SHA
reverse-route
crypto dynamic-map SDM_DYNMAP_1 2
set security-association idle-time 3600
set transform-set ESP-AES-256-SHA
reverse-route
!
!
crypto map SDM_CMAP_1 client authentication list sdm_vpn_xauth_ml_1
crypto map SDM_CMAP_1 isakmp authorization list sdm_vpn_group_ml_1
crypto map SDM_CMAP_1 65535 ipsec-isakmp dynamic SDM_DYNMAP_1
!
crypto ctcp port 10000
archive
log config
  hidekeys
!
!
ip tcp synwait-time 10
!
!
!
interface Null0
no ip unreachables
!
interface ATM0
no ip address
no ip redirects
no ip unreachables
no ip proxy-arp
ip route-cache flow
no atm ilmi-keepalive
pvc 0/38
  encapsulation aal5mux ppp dialer
  dialer pool-member 1
!
dsl operating-mode auto
hold-queue 224 in
!
interface FastEthernet0
!
interface FastEthernet1
!
interface FastEthernet2
!
interface FastEthernet3
!
interface Vlan1
description $FW_INSIDE$
ip address 10.10.7.1 255.255.255.0
ip access-group 121 in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat inside
ip virtual-reassembly
ip route-cache flow
crypto map SDM_CMAP_1
hold-queue 100 out
!
interface Dialer1
description $FW_OUTSIDE$
ip address negotiated
ip access-group 121 in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat outside
ip virtual-reassembly
encapsulation ppp
ip route-cache flow
no ip split-horizon
dialer pool 1
dialer idle-timeout 0
dialer persistent
dialer-group 1
no cdp enable
ppp authentication chap callin
ppp chap hostname <hostname>
ppp chap password 7 <password>
crypto map SDM_CMAP_1
!
ip local pool SDM_POOL_1 10.10.148.11 10.10.148.20
ip local pool public_184 123.12.12.184
ip local pool public_186 123.12.12.186
ip local pool public_187 123.12.12.187
ip local pool internal_9 10.10.7.9
ip local pool internal_8 10.10.7.8
ip local pool internal_223 10.10.7.223
ip local pool internal_47 10.10.7.47
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.10.140.0 255.255.255.0 10.10.7.2
!
no ip http server
no ip http secure-server
ip nat inside source route-map SDM_RMAP_1 interface Dialer1 overload
ip nat inside source static 10.10.7.9 123.12.12.184
ip nat inside source static tcp 10.10.7.8 22 123.12.12.185 22 extendable
ip nat inside source static tcp 10.10.7.8 25 123.12.12.185 25 extendable
ip nat inside source static tcp 10.10.7.8 80 123.12.12.185 80 extendable
ip nat inside source static tcp 10.10.7.8 443 123.12.12.185 443 extendable
ip nat inside source static tcp 10.10.7.8 993 123.12.12.185 993 extendable
ip nat inside source static tcp 10.10.7.8 1587 123.12.12.185 1587 extendable
ip nat inside source static tcp 10.10.7.8 8443 123.12.12.185 8443 extendable
ip nat inside source static 10.10.7.223 123.12.12.186
ip nat inside source static 10.10.7.47 123.12.12.187
!
logging 10.10.140.213
access-list 18 permit any
access-list 23 permit 10.10.140.0 0.0.0.255
access-list 23 permit 10.10.7.0 0.0.0.255
access-list 100 remark SDM_ACL Category=2
access-list 100 deny   ip any 10.10.148.0 0.0.0.255
access-list 100 permit ip any any
access-list 121 remark SDM_ACL Category=17
access-list 121 deny   udp any eq netbios-dgm any
access-list 121 deny   udp any eq netbios-ns any
access-list 121 deny   udp any eq netbios-ss any
access-list 121 deny   tcp any eq 137 any
access-list 121 deny   tcp any eq 138 any
access-list 121 deny   tcp any eq 139 any
access-list 121 permit ip any any
access-list 125 permit tcp any any eq www
access-list 125 permit udp any eq isakmp any
access-list 125 permit udp any any eq isakmp
access-list 194 deny   udp any eq isakmp any
access-list 194 deny   udp any any eq isakmp
access-list 194 permit ip host 123.12.12.184 any
access-list 194 permit ip any host 123.12.12.184
access-list 194 permit ip host 10.10.7.9 any
access-list 194 permit ip any host 10.10.7.9
access-list 195 deny   udp any eq isakmp any
access-list 195 deny   udp any any eq isakmp
access-list 195 permit ip host 123.12.12.185 any
access-list 195 permit ip any host 123.12.12.185
access-list 195 permit ip host 10.10.7.8 any
access-list 195 permit ip any host 10.10.7.8
no cdp run
route-map public_185 permit 10
match ip address 195
!
route-map public_184 permit 10
match ip address 194
!
route-map SDM_RMAP_1 permit 1
match ip address 100
!
!
control-plane
!
!
line con 0
login authentication local_authen
no modem enable
transport preferred none
transport output telnet
stopbits 1
line aux 0
login authentication local_authen
transport output telnet
stopbits 1
line vty 0 4
access-class 23 in
privilege level 15
authorization exec local_author
login authentication local_authen
length 0
transport preferred none
transport input telnet ssh
!
scheduler max-task-time 5000
scheduler allocate 4000 1000
scheduler interval 500
sntp server 130.88.202.49
sntp server 130.88.200.98
sntp server 130.88.200.6
sntp server 130.88.203.64
end

Any help would be appreciated.

Thanks a lot.

Ciao,

Eric

40 Replies 40

Hi Eric,

When vpnc is configured with force-natt I can see exactly these packets:

10.10.40.106 -> 123.1.1.185 UDP D=500 S=500 LEN=1294

123.1.1.185 -> 10.10.40.106 UDP D=500 S=500 LEN=412

10.10.40.106 -> 123.1.1.185 UDP D=4500 S=4500 LEN=168

10.10.40.106 -> 123.1.1.185 UDP D=4500 S=4500 LEN=168

10.10.40.106 -> 123.1.1.185 UDP D=4500 S=4500 LEN=168

ok so this confirms my suspicion, the VPNrouter sends the

Xauth-request on port 4500 but your client does not see it, so it keeps resending AM3 (and after a while gives up).

When snooping the interface observing a successful connection to my company's VPN, I can see this (IP addresses altered):

10.10.40.106 -> 210.21.21.123 UDP D=500 S=500 LEN=842
210.21.21.123 -> 10.10.40.106 UDP IP fragment ID=151 Offset=0    MF=1 TOS=0x0 TTL=246
210.21.21.123 -> 10.10.40.106 UDP IP fragment ID=151 Offset=1480 MF=0 TOS=0x0 TTL=246
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=232
210.21.21.123 -> 10.10.40.106 UDP D=4500 S=4500 LEN=88
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=104
210.21.21.123 -> 10.10.40.106 UDP D=4500 S=4500 LEN=80
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=80
10.10.40.106 -> 210.21.21.123 UDP D=4500 S=4500 LEN=184
210.21.21.123 -> 10.10.40.106 UDP D=4500 S=4500 LEN=224

Ok so now at least we know that with NAT-T it should work.

So the question is, why do the UDP4500 packets from your company's vpn arrive (cfr the line in bold), but not the ones from your VPNrouter?

I see nothing in the other Cisco's router that blocks port 4500 anywhere.

Which router do you mean ?

I meant on the NAT router that your vpnc is behind. Is it configured with PAT / port forwarding for port 4500 ? I guess it is otherwise the connection to your company wouldn't work. Is there an ACL maybe that allows UDP 4500 from your company but not from the other vpn router?

The only other explanation I can think of is that the ISP that your VPNrouter connects to, drops the UDP 4500...

Herbert

Ok so now at least we know that with NAT-T it should work.

So the question is, why do the UDP4500 packets from your company's vpn arrive (cfr the line in bold), but not the ones from your VPNrouter?


Do we know though that my vpn gateway is sending it? Is there any way to "snoop" the router's Dialer1 interface?

I see nothing in the other Cisco's router that blocks port 4500 anywhere.

Which router do you mean ?

I meant on the NAT router that your vpnc is behind. Is it configured with PAT / port forwarding for port 4500 ? I guess it is otherwise the connection to your company wouldn't work. Is there an ACL maybe that allows UDP 4500 from your company but not from the other vpn router?

I did mean the same router you are referring to, there is no reference to port 4500 in that router config at all.

The only other explanation I can think of is that the ISP that your VPNrouter connects to, drops the UDP 4500...


Hmm, I can ask them, but given that I can VPN into my company's VPN router from that internet connection, too, I don't think so.

The other option of course still is that I simply need to update the firmware on my VPN gateway/router, but the last time I tried to buy a Cisco support contract, I gave up after several weeks of trying to find someone willing to sell me anything for less than several thousand pound.... :-(

Eric

Ok so now at least we know that with NAT-T it should work.

So the question is, why do the UDP4500 packets from your company's vpn arrive (cfr the line in bold), but not the ones from your VPNrouter?


Do we know though that my vpn gateway is sending it? Is there any way to "snoop" the router's Dialer1 interface?

I don't remember if you mentioned the IOS version; if it runs IOS 12.3(4)T or later you could try RITE; I've never used it myself so not sure how well it works on a dialer interface:

http://www.cisco.com/en/US/docs/ios/sec_user_services/configuration/guide/sec_ip_traff_export.html

I did mean the same router you are referring to, there is no reference to port 4500 in that router config at all.

Ok; just to be sure, would you mind posting the config here? Or just the acl and nat?

The only other explanation I can think of is that the ISP that your VPNrouter connects to, drops the UDP 4500...


Hmm, I can ask them, but given that I can VPN into my company's VPN router from that internet connection, too, I don't think so.

If that connection also uses nat-t, then you are right. And I think it is very unlikely anyway that it would drop 4500 and not 500. I only mentioned it because I like to consider all options.

The other option of course still is that I simply need to update the firmware on my VPN gateway/router, but the last time I tried to buy a Cisco support contract, I gave up after several weeks of trying to find someone willing to sell me anything for less than several thousand pound.... :-(

Well, having just said I consider all options I can't fully exclude the vpnrouter as the culprit, however I do think it is very unlikely that the vpnrouter is sending the udp500 packet but not the udp4500 packet.

Herbert

Ok; just to be sure, would you mind posting the config here? Or just the acl and nat?

Here is the config of the outgoing router:

Current configuration : 2683 bytes
!
! No configuration change since last restart
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname cisco-vispa
!
logging queue-limit 100
enable secret 5
!
username password 7
username password 7
clock timezone gmt 0
clock summer-time bst recurring last Sun Mar 1:00 last Sun Oct 1:00
ip subnet-zero
no ip source-route
ip host nfs1 10.10.140.207
ip name-server 208.67.222.222
ip name-server 208.67.220.220
ip dhcp database dhcpinternal
ip dhcp excluded-address 10.10.40.1 10.10.40.99
ip dhcp excluded-address 10.10.40.151 10.10.40.255
!
ip dhcp pool dhcpinternal
   import all
   network 10.10.40.0 255.255.255.0
   default-router 10.10.40.1
   dns-server 208.67.222.222 208.67.220.220
!
!
ip audit notify log
ip audit po max-events 100
no ftp-server write-enable
!
!
!
!
!
!
!
interface Ethernet0
ip address 10.10.40.1 255.255.255.0
ip access-group 121 in
no ip redirects
no ip proxy-arp
ip nat inside
hold-queue 100 out
!
interface ATM0
no ip address
no ip mroute-cache
no atm ilmi-keepalive
pvc 0/38
  encapsulation aal5mux ppp dialer
  dialer pool-member 1
!
dsl operating-mode auto
hold-queue 224 in
!
interface Dialer1
ip address negotiated
ip access-group 121 in
no ip redirects
no ip proxy-arp
ip nat outside
encapsulation ppp
no ip split-horizon
dialer pool 1
dialer idle-timeout 0
dialer persistent
dialer-group 1
ppp authentication chap callin
ppp chap hostname
ppp chap password 7
!
ip nat inside source list 18 interface Dialer1 overload
ip nat inside source static 10.10.40.8 interface Dialer1
ip classless
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.10.50.0 255.255.255.0 10.10.40.2
ip route 10.10.140.0 255.255.255.0 10.10.40.2
no ip http server
no ip http secure-server
!
logging 10.10.140.213
access-list 18 permit any
access-list 121 deny   udp any eq netbios-dgm any
access-list 121 deny   udp any eq netbios-ns any
access-list 121 deny   udp any eq netbios-ss any
access-list 121 deny   tcp any eq 137 any
access-list 121 deny   tcp any eq 138 any
access-list 121 deny   tcp any eq 139 any
access-list 121 permit ip any any
!
line con 0
exec-timeout 120 0
no modem enable
transport preferred none
stopbits 1
line aux 0
stopbits 1
line vty 0 4
access-class 23 in
exec-timeout 120 0
password 7 09610F0A0B0A204A1909
login local
length 0
transport preferred none
!
scheduler max-task-time 5000
sntp server 130.88.202.49
sntp server 130.88.200.98
sntp server 130.88.200.6
sntp server 130.88.203.64
!
end

The only other explanation I can think of is that the ISP that your VPNrouter connects to, drops the UDP 4500...

Hmm, I can ask them, but given that I can VPN into my company's VPN router from that internet connection, too, I don't think so.

If that connection also uses nat-t, then you are right. And I think it is very unlikely anyway that it would drop 4500 and not 500. I only mentioned it because I like to consider all options.


A snoop when connected to my employer's VPN across the same network shows both the UDP 500 packets and UDP 4500 packets aplenty.

Ok so now at least we know that with NAT-T it should work.

So the question is, why do the UDP4500 packets from your company's vpn arrive (cfr the line in bold), but not the ones from your VPNrouter?


Do we know though that my vpn gateway is sending it? Is there any way to "snoop" the router's Dialer1 interface?

I don't remember if you mentioned the IOS version; if it runs IOS 12.3(4)T or later you could try RITE; I've never used it myself so not sure how well it works on a dialer interface:

http://www.cisco.com/en/US/docs/ios/sec_user_services/configuration/guide/sec_ip_traff_export.html

Unfortunately, my 837 doesn't appear to support ip traffic-export :-(

Thanks again for your help.

Eric

I'm slowly running out of ideas here I'm afraid...

Maybe instead of RITE you can do:

  access-list 199 permit ip host x.x.x.x host y.y.y.y

where X is the vpnrouter and Y is the client.

then

  debug ip packet detail 199

You could do the same on the NAT router but use access-list 199 permit host x.x.x.x any.

And/or on the NAT router modify the ACL so you have "permit udp host x.x.x.x any eq 4500" before the "permit ip any any" and then check show access-list for any hits.

hth

Herbert

I'm slowly running out of ideas here I'm afraid...


I'm glad it's not just me... ;-)


Maybe instead of RITE you can do:

  access-list 199 permit ip host x.x.x.x host y.y.y.y

where X is the vpnrouter and Y is the client.

then

  debug ip packet detail 199


That worked! Thanks. I'll have to remember that.

Here's the output:

003661: Oct 15 13:04:50.804 bst: IP: tableid=0, s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), routed via RIB
003662: Oct 15 13:04:50.804 bst: IP: s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), len 1314, rcvd 3
003663: Oct 15 13:04:50.804 bst:     UDP src=500, dst=500
003664: Oct 15 13:04:50.852 bst: IP: tableid=0, s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), routed via FIB
003665: Oct 15 13:04:50.852 bst: IP: s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), len 432, sending
003666: Oct 15 13:04:50.852 bst:     UDP src=500, dst=500
003667: Oct 15 13:04:51.036 bst: IP: tableid=0, s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), routed via RIB
003668: Oct 15 13:04:51.036 bst: IP: s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), len 188, rcvd 3
003669: Oct 15 13:04:51.036 bst:     UDP src=4500, dst=4500
003670: Oct 15 13:04:51.040 bst: IP: tableid=0, s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), routed via FIB
003671: Oct 15 13:04:51.040 bst: IP: s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), len 124, sending
003672: Oct 15 13:04:51.040 bst:     UDP src=4500, dst=4500
003673: Oct 15 13:04:51.040 bst: IP: tableid=0, s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), routed via FIB
003674: Oct 15 13:04:51.044 bst: IP: s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), len 116, sending
003675: Oct 15 13:04:51.044 bst:     UDP src=4500, dst=4500
003676: Oct 15 13:04:51.044 bst: IP: tableid=0, s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), routed via FIB
003677: Oct 15 13:04:51.044 bst: IP: s=123.1.1.185 (local), d=120.20.20.142 (Dialer1), len 100, sending
003678: Oct 15 13:04:51.044 bst:     UDP src=4500, dst=4500
003679: Oct 15 13:04:53.039 bst: IP: tableid=0, s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), routed via RIB
003680: Oct 15 13:04:53.039 bst: IP: s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), len 188, rcvd 3
003681: Oct 15 13:04:53.039 bst:     UDP src=4500, dst=4500
003682: Oct 15 13:04:57.045 bst: IP: tableid=0, s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), routed via RIB
003683: Oct 15 13:04:57.049 bst: IP: s=120.20.20.142 (Dialer1), d=123.1.1.185 (Dialer1), len 188, rcvd 3
003684: Oct 15 13:04:57.049 bst:     UDP src=4500, dst=4500

We now know that the router is indeed sending the UDP 4500 packet but doing a snoop on the client at the same time, I can't see it arriving.

We also see that the UDP 4500 packet sent from the client arrives.

I'm starting to wonder if I don't have a NAT problem after all, ie. that the 871 that the client is connected to doesn't know where to send the 4500 packet. Unfortunlately the same packet debug doesn't appear to work on the 871. I will have another go....

Thanks again for your help.

Ciao,

Eric

It seems that the debug ip packet detail on the 837 is a bit, well, buggy (the last post should have read ...doesn't appear to work on the 837....).

The debug setting seems to capture some but not all packets and not repeatably.

I logged a call with my ISP earlier this morning to clarify the UDP port 4500 thing and am waiting for a response.

The not-so-encouraging thing is that I tried via a third ISP (Vodafone USB mobile modem) earlier today with the same results as testing from my second ISP connection: no UDP 4500 reply packet... Having said that of course, Vodafone does also do NAT for their mobile addresses.

I have also noticed something else that's odd: I have a hardware VPN client (for which I can't see any debug output at all) and that's configured to VPN into my company VPN concentrator and it only appears to work on one of my ISP connections, so there is the possibility that both Vodafone and my ISP are doing something "odd" that causes the VPN solution to break.

Either way, I'm down to waiting for my ISP to get back to me now.

Thanks for all your help.

Eric

My ISP says they don't block anything, which leaves me pretty stuck.

I'm in two different locations early next week and will try a few more ISPs for good measure...

Eric

Well, maybe one last thing to try:

on your NAT router (the one your vpnc is behind), change your acl from:

access-list 121 deny   udp any eq netbios-dgm any
access-list 121 deny   udp any eq netbios-ns any
access-list 121 deny   udp any eq netbios-ss any
access-list 121 deny   tcp any eq 137 any
access-list 121 deny   tcp any eq 138 any
access-list 121 deny   tcp any eq 139 any
access-list 121 permit ip any any

to

access-list 121 deny   udp any eq netbios-dgm any
access-list 121 deny   udp any eq netbios-ns any
access-list 121 deny   udp any eq netbios-ss any
access-list 121 deny   tcp any eq 137 any
access-list 121 deny   tcp any eq 138 any
access-list 121 deny   tcp any eq 139 any

access-list 121 permit udp any an eq 4500
access-list 121 permit ip any any

then try to connect and check "show access-list 121" and see if you get any hits on that new line...

BTW which IOS version is it running? I see 12.2 in the config, so not that recent I guess

That's interesting. The access list shows 7 matches, the debug ip packet detail for the same attempt showed 2 packets (clearly that isn't working properly as I already suspected) and the snoop on the client shows 4 packets.

That would lead me to conclude that the packets do arrive at the NAT router but for some reason the router doesn't know that they ought to be forwarded on to the client. This may be a stupid question: how would the router know where to forward them on to in the first place since we are doing NAT overload? They are after all only UDP packets.

The NAT router's version is:

cisco-vispa#show version
Cisco Internetwork Operating System Software
IOS (tm) C837 Software (C837-K9O3Y6-M), Version 12.2(13)ZH4, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)
Synched to technology version 12.2(14.5)T
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Wed 24-Mar-04 19:00 by ealyon
Image text-base: 0x800131E8, data-base: 0x80AA1DF0

ROM: System Bootstrap, Version 12.2(11r)YV1, RELEASE SOFTWARE (fc1)
ROM: C837 Software (C837-K9O3Y6-M), Version 12.2(13)ZH4, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)

cisco-vispa uptime is 10 hours, 0 minutes
System returned to ROM by power-on
System restarted at 22:29:46 bst Fri Oct 15 2010
System image file is "flash:c837-k9o3y6-mz.122-13.ZH4.bin"

CISCO C837 (MPC857DSL) processor (revision 0x501) with 44237K/4915K bytes of memory.
Processor board ID FOC08511D6P (1759460522), with hardware revision 0000
CPU rev number 7
Bridging software.
1 Ethernet/IEEE 802.3 interface(s)
1 ATM network interface(s)
128K bytes of non-volatile configuration memory.
12288K bytes of processor board System flash (Read/Write)
2048K bytes of processor board Web flash (Read/Write)

Configuration register is 0x2102

cisco-vispa#

I'll take any hints and tips by the way on where to get a Cisco support agreement that allows me to update the versions on my various pieces of Cisco kit that doesn't cost me thousands of pounds....

Eric

Hi Eric

to get a service contract (or maybe just to inquire about the price of an IOS upgrade), check the partner locator tool to fin a Cisco partner near you:

http://tools.cisco.com/WWChannels/LOCATR/openBasicSearch.do

To be honest I have no idea about pricing...

To answer the question about how the router knows how to forward the incoming UDP packets: when the client first sends a UDP packet, the router will create an entry in its NAT table, for the given source & destination address and ports, and set a timer.

When an inbound packet arrives that matches the entry, the packet is "untranslated" and forwarded on the inside (and the timer reset).

If no traffic is seen before the timer expires, the nat entry is removed.

So one thing to check would be "show ip nat trans" (assuming this command exists in your IOS version...) immediately after attempting the vpn connection.

You can also enable "debug ip nat detail" and see if that tells you anything when the UDP4500 packet comes in.

Should you discover that the nat entry times out, then you can tweak the timer with:

basic(config)#ip nat translation udp-timeout ?
  <0-536870>  Timeout in seconds
  never       Never timeout

or:

basic(config)#ip nat translation port-timeout udp 4500 ?
  <0-536870>  Timeout in seconds
  never       Never timeout

(again, not sure if your IOS version already has these commands)

hth

Herbert

Thanks for the info. I'll try that.

Now back online after two days on the road where I have tried three further public internet connections, same result with all of them. cisco-udp allows the connection but then no return packets flow. force-natt sends the UDP packet on port 4500 but never gets a response.

Note that all three public internet connections were nat'ed. While I think we can rule out that my ISP blocks anything, I don't think we can quite rule out that NAT isn't interfering.

Once I've caught up with all my todos after two days out of the office, I will have another look in detail at your suggestions.

Thanks for bearing with me and for all your good help.

Ciao,

Eric

My apologies for dragging this out, it's one of those weeks....

I've compared the nat tables of the router to which the client is connected before and after attempting to VPN in. The relevant entries appear to be:

Pro Inside global      Inside local       Outside local      Outside global
udp 123.1.1.185:500  123.1.1.185:500  120.20.20.142:500 120.20.20.142:500
udp 123.1.1.185:1029 123.1.1.185:4500 120.20.20.142:4500 120.20.20.142:4500

One does notice the Inside global port on the second line is 1029 and I'm not clear where that would have come from given the snoop does not show any traffice at all on that port.

The debug at the same time show this:

001247: Oct 22 06:40:49.237 bst: NAT: i: udp (123.1.1.185, 500) -> (120.20.20.142, 500) [36660]
001248: Oct 22 06:40:49.417 bst: NAT: i: udp (123.1.1.185, 4500) -> (120.20.20.142, 4500) [36661]
001249: Oct 22 06:40:49.417 bst: NAT: UDP s=4500->1029, d=4500
001250: Oct 22 06:40:49.421 bst: NAT: i: udp (123.1.1.185, 4500) -> (120.20.20.142, 4500) [36662]
001251: Oct 22 06:40:49.421 bst: NAT: UDP s=4500->1029, d=4500
001252: Oct 22 06:40:49.421 bst: NAT: i: udp (123.1.1.185, 4500) -> (120.20.20.142, 4500) [36663]
001253: Oct 22 06:40:49.421 bst: NAT: UDP s=4500->1029, d=4500

Again, the port 1029 shows up for some reason.

I'm not sure if that is a red herring or if there is an issue with this....

Eric

Hi eric,

udp 123.1.1.185:1029 123.1.1.185:4500 120.20.20.142:4500 120.20.20.142:4500


could it be that you incorrectly replace the real addresses with fake ones in the output above?


I.e. normally I would expect something like (if the client has internal ip 10.0.0.1):

udp 123.1.1.185:1029 10.0.0.1:4500 120.20.20.142:4500 120.20.20.142:4500

I.e. the inside local address should be the REAL address of the client, not the public ip address.

Now, about the port 1029, that I would normally say is normal, because this router is doing nat overload (aka dynamic nat or PAT).

So the client sends packets with source port 4500, the router translates the source IP to the dialer interface's address and the source port to a random port.

BUT on the other hand,looking at the debugs you took earlier on the VPN router:

000768: Oct 14 12:49:37.713 bst: ISAKMP (0:2007): received packet from 120.3.3.142 dport 4500 sport 4500 Global (R) AG_INIT_EXCH

we see that it is receiving packets with sport (source port) 4500 ...

Now, these were not taken at the same time, so just to be sure I would suggest to test again and get all the debugs and show commands we collected so far again, on both routers at the same time.

If they confirm the above, then I guess this is a bug in NAT on the  client side router since the debugs say it is translating the source  port but it is not really doing that.

BTW the NAT debugs (or at least the part you copy pasted) only show outbound packets (and indeed you see again "s=4500->1029" which means  it is translating the source port, or at least it should be).

Before the part you quoted, were there debugs about inbound  udp500 packets? (just to make sure there is nothing wrong with the  debugs)

Herbert

Thinking about it some more, I had 2 more thoughts:

- my theory doesn't explain why the connection works towards your company vpn

- you could try to add something like this:

ip nat inside source static tcp 10.0.0.1 4500 interface dialer 1 4500

In other words statically nat port 4500, and see if that changes anything.

hth

H

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: