Solved: Re: IPSEC over VTI not working

apstownsend · ‎09-22-2017

Hi

I have a new ISR4321 router which is replacing an ISR877. The ISR4321 has two IPSEC over VTI connections to two other ISR's. The ISR4321 is unable to establish IPSEC over VTI, but simple GRE over VTI works fine.

The ISAKMP response on the remote ISR's is trying to return to port 512 not port 500 on the ISR4321. I think this is the issue, but I don't know how to resolve it. It must be caused by the ISR4321 as it happens on all remote ISRs: -

Sep 22 2017 11:21:15 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (I) MM_SA_SETUP
Sep 22 2017 11:21:15 BST: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.
Sep 22 2017 11:21:15 BST: ISAKMP:(0): retransmitting due to retransmit phase 1
Sep 22 2017 11:21:15 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:15 BST: ISAKMP (0): incrementing error counter on sa, attempt 4 of 5: retransmit phase 1
Sep 22 2017 11:21:15 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP
Sep 22 2017 11:21:15 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (I) MM_SA_SETUP

Here is a full debug from a remote ISR: -

Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing SA payload. message ID = 0
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP (0): vendor ID is NAT-T RFC 3947
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 245 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP (0): vendor ID is NAT-T v7
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 157 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID is NAT-T v3
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 123 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID is NAT-T v2
Sep 22 2017 11:21:08 BST: ISAKMP:(0):found peer pre-shared key matching 1.1.1.1
Sep 22 2017 11:21:08 BST: ISAKMP:(0): local preshared key found
Sep 22 2017 11:21:08 BST: ISAKMP : Scanning profiles for xauth ...
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Checking ISAKMP transform 1 against priority 10 policy
Sep 22 2017 11:21:08 BST: ISAKMP:      encryption AES-CBC
Sep 22 2017 11:21:08 BST: ISAKMP:      keylength of 256
Sep 22 2017 11:21:08 BST: ISAKMP:      hash SHA
Sep 22 2017 11:21:08 BST: ISAKMP:      default group 5
Sep 22 2017 11:21:08 BST: ISAKMP:      auth pre-share
Sep 22 2017 11:21:08 BST: ISAKMP:      life type in seconds
Sep 22 2017 11:21:08 BST: ISAKMP:      life duration (basic) of 3600
Sep 22 2017 11:21:08 BST: ISAKMP:(0):atts are acceptable. Next payload is 0
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Acceptable atts:actual life: 0
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Acceptable atts:life: 0
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Basic life_in_seconds:3600
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Returning Actual lifetime: 3600
Sep 22 2017 11:21:08 BST: ISAKMP:(0)::Started lifetime timer: 3600.

Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 69 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP (0): vendor ID is NAT-T RFC 3947
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 245 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP (0): vendor ID is NAT-T v7
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 157 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID is NAT-T v3
Sep 22 2017 11:21:08 BST: ISAKMP:(0): processing vendor id payload
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID seems Unity/DPD but major 123 mismatch
Sep 22 2017 11:21:08 BST: ISAKMP:(0): vendor ID is NAT-T v2
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Old State = IKE_R_MM1  New State = IKE_R_MM1

Sep 22 2017 11:21:08 BST: ISAKMP:(0): constructed NAT-T vendor-rfc3947 ID
Sep 22 2017 11:21:08 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (R) MM_SA_SETUP
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Sending an IKE IPv4 Packet.
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Old State = IKE_R_MM1  New State = IKE_R_MM2

Sep 22 2017 11:21:08 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:08 BST: ISAKMP:(0):peer does not do paranoid keepalives.

Sep 22 2017 11:21:08 BST: ISAKMP:(0):deleting SA reason "Death by retransmission P1" state (R) MM_SA_SETUP (peer 1.1.1.1)
Sep 22 2017 11:21:08 BST: ISAKMP:(0):deleting SA reason "Death by retransmission P1" state (R) MM_SA_SETUP (peer 1.1.1.1)
Sep 22 2017 11:21:08 BST: ISAKMP: Unlocking peer struct 0x852FE83C for isadb_mark_sa_deleted(), count 0
Sep 22 2017 11:21:08 BST: ISAKMP: Deleting peer node by peer_reap for 1.1.1.1: 852FE83C
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL
Sep 22 2017 11:21:08 BST: ISAKMP:(0):Old State = IKE_R_MM2  New State = IKE_DEST_SA

Sep 22 2017 11:21:15 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (I) MM_SA_SETUP
Sep 22 2017 11:21:15 BST: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.
Sep 22 2017 11:21:15 BST: ISAKMP:(0): retransmitting due to retransmit phase 1
Sep 22 2017 11:21:15 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:15 BST: ISAKMP (0): incrementing error counter on sa, attempt 4 of 5: retransmit phase 1
Sep 22 2017 11:21:15 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP
Sep 22 2017 11:21:15 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (I) MM_SA_SETUP
Sep 22 2017 11:21:15 BST: ISAKMP:(0):Sending an IKE IPv4 Packet.
Sep 22 2017 11:21:18 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (R) MM_SA_SETUP
Sep 22 2017 11:21:18 BST: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.
Sep 22 2017 11:21:18 BST: ISAKMP:(0): retransmitting due to retransmit phase 1
Sep 22 2017 11:21:19 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:19 BST: ISAKMP (0): incrementing error counter on sa, attempt 1 of 5: retransmit phase 1
Sep 22 2017 11:21:19 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP
Sep 22 2017 11:21:19 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (R) MM_SA_SETUP
Sep 22 2017 11:21:19 BST: ISAKMP:(0):Sending an IKE IPv4 Packet.
R1003951#
Sep 22 2017 11:21:25 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (I) MM_SA_SETUP
Sep 22 2017 11:21:25 BST: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.
Sep 22 2017 11:21:25 BST: ISAKMP:(0): retransmitting due to retransmit phase 1
Sep 22 2017 11:21:25 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:25 BST: ISAKMP (0): incrementing error counter on sa, attempt 5 of 5: retransmit phase 1
Sep 22 2017 11:21:25 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP
Sep 22 2017 11:21:25 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (I) MM_SA_SETUP
Sep 22 2017 11:21:25 BST: ISAKMP:(0):Sending an IKE IPv4 Packet.
Sep 22 2017 11:21:28 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (R) MM_SA_SETUP
Sep 22 2017 11:21:28 BST: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.
Sep 22 2017 11:21:28 BST: ISAKMP:(0): retransmitting due to retransmit phase 1
Sep 22 2017 11:21:29 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:29 BST: ISAKMP (0): incrementing error counter on sa, attempt 2 of 5: retransmit phase 1
Sep 22 2017 11:21:29 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP
Sep 22 2017 11:21:29 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (R) MM_SA_SETUP
Sep 22 2017 11:21:29 BST: ISAKMP:(0):Sending an IKE IPv4 Packet.
Sep 22 2017 11:21:35 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:35 BST: ISAKMP:(0):peer does not do paranoid keepalives.

Sep 22 2017 11:21:35 BST: ISAKMP:(0):deleting SA reason "Death by retransmission P1" state (I) MM_SA_SETUP (peer 1.1.1.1)
Sep 22 2017 11:21:35 BST: ISAKMP:(0):deleting SA reason "Death by retransmission P1" state (I) MM_SA_SETUP (peer 1.1.1.1)
Sep 22 2017 11:21:35 BST: ISAKMP: Unlocking peer struct 0x85725B18 for isadb_mark_sa_deleted(), count 0
Sep 22 2017 11:21:35 BST: ISAKMP: Deleting peer node by peer_reap for 1.1.1.1: 85725B18
Sep 22 2017 11:21:35 BST: ISAKMP:(0):deleting node -2087223094 error FALSE reason "IKE deleted"
Sep 22 2017 11:21:35 BST: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL
Sep 22 2017 11:21:35 BST: ISAKMP:(0):Old State = IKE_I_MM3  New State = IKE_DEST_SA

Sep 22 2017 11:21:38 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (R) MM_SA_SETUP
Sep 22 2017 11:21:38 BST: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.
Sep 22 2017 11:21:38 BST: ISAKMP:(0): retransmitting due to retransmit phase 1
Sep 22 2017 11:21:39 BST: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...
Sep 22 2017 11:21:39 BST: ISAKMP (0): incrementing error counter on sa, attempt 3 of 5: retransmit phase 1

Here is a config from the ISR4321 (first time I have used ZBF): -

class-map type inspect match-any CM_ZP_ANY
 match access-group name ACL_ANY
class-map type inspect match-any CM_ZP_IN-OUT
 match protocol dns
 match protocol icmp
 match protocol http
 match protocol https
 match protocol ssh
 match access-group name ACL_ZP_IN-OUT
class-map type inspect match-any CM_L2L
 match access-group name ACL_L2L
class-map type inspect match-any CM_ZP_OUT-LO
 match access-group name ACL_ZP_OUT-LO
class-map type inspect match-any CM_ZP_LO-OUT
 match access-group name ACL_ZP_LO-OUT
class-map type inspect match-any CM_ZP_IN-WAN
 match access-group name ACL_ZP_IN-WAN
class-map type inspect match-any CM_ZP_WAN-IN
 match access-group name ACL_ZP_WAN-IN
!
policy-map type inspect PM_ZP_IN-WAN
 class type inspect CM_ZP_IN-WAN
  inspect
 class class-default
  drop log
policy-map type inspect PM_ZP-LO-OUT
 class type inspect CM_L2L
  pass log
 class type inspect CM_ZP_LO-OUT
  pass log
 class class-default
  drop log
policy-map type inspect PM_ZP-OUT-LO
 class type inspect CM_L2L
  pass log
 class type inspect CM_ZP_OUT-LO
  pass log
 class class-default
  drop log
policy-map type inspect PM_ZP-IN-OUT
 class type inspect CM_ZP_IN-OUT
  inspect
 class class-default
  drop log
policy-map type inspect PM_ZP_WAN-IN
 class type inspect CM_ZP_WAN-IN
  inspect
 class class-default
  drop log
!
zone security Z_IN
zone security Z_OUT
zone security Z_WAN
zone-pair security ZP_IN-OUT source Z_IN destination Z_OUT
 service-policy type inspect PM_ZP-IN-OUT
zone-pair security ZP_IN-WAN source Z_IN destination Z_WAN
 service-policy type inspect PM_ZP_IN-WAN
zone-pair security ZP_LO-OUT source self destination Z_OUT
 service-policy type inspect PM_ZP-LO-OUT
zone-pair security ZP_OUT-LO source Z_OUT destination self
 service-policy type inspect PM_ZP-OUT-LO
zone-pair security ZP_WAN-IN source Z_WAN destination Z_IN
 service-policy type inspect PM_ZP_WAN-IN
!
crypto isakmp policy 10
 encr aes 256
 authentication pre-share
 group 5
 lifetime 3600
crypto isakmp key xxxx address 2.2.2.2
!
!
crypto ipsec transform-set CITS_1 esp-aes esp-sha512-hmac
 mode transport
!
crypto ipsec profile CIP_1
 set transform-set CITS_1
!
!interface Loopback0
 ip address 1.2.50.79 255.255.255.255
!
interface Tunnel1002
 ip address 10.144.226.5 255.255.255.254
 ip mtu 1300
 ip tcp adjust-mss 1260
 tunnel source Dialer1
 tunnel mode ipsec ipv4
 tunnel destination 2.2.2.2
 tunnel protection ipsec profile CIP_1
!
interface Ethernet0/2/0
 no ip address
 no negotiation auto
!
interface Ethernet0/2/0.101
 encapsulation dot1Q 101
 pppoe enable group global
 pppoe-client dial-pool-number 1
!
interface Vlan1
 description LAN/WIFI
 ip address 10.144.144.254 255.255.255.0
 ip nat inside
 zone-member security Z_IN
!
interface Dialer1
 ip address negotiated
 ip nat outside
 zone-member security Z_OUT
 encapsulation ppp
 ip tcp adjust-mss 1452
 dialer pool 1
 ppp mtu adaptive
 ppp authentication chap callin
 ppp chap hostname x
 ppp chap password 7 x
!
ip nat inside source list ACL_NAT interface Dialer1 overload
ip forward-protocol nd
no ip http server
no ip http secure-server
ip tftp source-interface Vlan1
ip route 0.0.0.0 0.0.0.0 Dialer1 permanent
ip route 10.145.0.0 255.255.224.0 Tunnel1001
ip route 192.168.0.0 255.255.254.0 Tunnel1001
ip route 192.168.254.0 255.255.255.0 Tunnel1001
!
ip ssh logging events
ip ssh version 2
ip ssh dh min size 4096
ip ssh server algorithm encryption aes192-ctr aes256-ctr
ip ssh client algorithm encryption aes192-ctr aes256-ctr
!
!
ip access-list extended ACL_L2L
 permit ip any any
ip access-list extended ACL_NAT
 deny   ip any object-group OGN_RFC1918
 permit ip any any
ip access-list extended ACL_VPN_L2L
 permit object-group OGS_VPN_L2L object-group OGN_VPN_L2L any
 permit object-group OGS_VPN_L2L any object-group OGN_VPN_L2L
 permit gre any any
 permit esp any any
 permit udp any eq isakmp any eq isakmp
ip access-list extended ACL_VTY_IN
 permit tcp 82.118.108.48 0.0.0.15 any
 permit tcp host 212.105.163.218 any
 permit tcp host 78.25.251.240 any
 permit tcp host 78.25.251.241 any
 permit tcp 10.144.144.0 0.0.0.255 any
 permit tcp host 10.145.1.111 any
 deny   ip any any log
ip access-list extended ACL_ZP_IN-OUT
 permit ip object-group OGN_LAN object-group OGN_RFC1918 log
 permit ip any any
 permit object-group OGS_IN-OUT object-group OGN_LAN any
ip access-list extended ACL_ZP_IN-WAN
 permit ip any any
ip access-list extended ACL_ZP_LO-OUT
 permit icmp any any
 permit tcp any object-group OGN_DATCOM eq 22
 permit udp any object-group OGN_DNS eq domain
 permit tcp any object-group OGN_DNS eq domain
 permit udp any any eq ntp
 permit object-group OGS_VPN_L2L any object-group OGN_VPN_L2L
 permit udp any eq domain any
ip access-list extended ACL_ZP_OUT-LO
 permit icmp object-group OGN_DATCOM any
 permit tcp object-group OGN_DATCOM any eq 22
 permit object-group OGS_VPN_L2L object-group OGN_VPN_L2L any
ip access-list extended ACL_ZP_WAN-IN
 permit ip any any

Thanks

Andrew

apstownsend · ‎11-17-2017

Hi Igor,

Easy fix in the end. I altered the NAT ACL to just include the subnet of the local LAN: -

Before: -

object-group network OGN_RFC1918
 10.0.0.0 255.0.0.0
 192.168.0.0 255.255.0.0
 172.0.0.0 255.224.0.0

ip nat inside source list ACL_NAT interface Dialer1 overload

ip access-list extended ACL_NAT
 deny ip any object-group OGN_RFC1918
 permit ip any any

After: -

object-group network OGN_RFC1918
 10.0.0.0 255.0.0.0
 192.168.0.0 255.255.0.0
 172.0.0.0 255.224.0.0

ip nat inside source list ACL_NAT interface Dialer1 overload

ip access-list extended ACL_NAT
 deny ip any object-group OGN_RFC1918
 permit ip 192.168.1.0 255.255.255.0 any

This fixed my issue. 'permit ip any any' was always fine on IOS, although not recommended, whereas on IOS-XE it doesn't work (by design.)

Hopefully this helps you.

Thanks,

Andrew

View solution in original post

Francesco Molino · ‎09-22-2017

Hi can you share the config of the remote router?

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

apstownsend · ‎09-25-2017

Hi Francesco,

Here you go: -

! NVRAM config last updated at 04:00:00 BST Sun Sep 24 2017
version 15.1
no service pad
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
!
hostname R1003951
!
boot-start-marker
boot-end-marker
!
logging buffered 4096 informational
!
aaa new-model
!
aaa authentication login default local
aaa authentication ppp default local
aaa authorization exec default local
aaa authorization network default local
!
aaa session-id common
clock timezone GMT 0 0
clock summer-time BST recurring last Sun Mar 1:00 last Sun Oct 2:00
crypto pki token default removal timeout 0
!
!
dot11 syslog
ip source-route
!
ip cef
ip domain name somedomain.com
ip name-server 8.8.8.8
ip name-server 8.8.4.4
login on-failure log
login on-success log
!
archive
path ftp://somedomain.com/upload/cisco/backup/BK1003951
write-memory
time-period 1440
object-group network DNS-SERVERS
host 8.8.8.8
host 8.8.4.4
!
object-group service EX-IN-ALL
tcp eq ftp-data
!
object-group network OGN_COMPANY
host 1.1.1.1

!
no ip ftp passive
ip ftp username sd_ftp_cisco
ip ftp password 7 111111
!
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 5
lifetime 3600
crypto isakmp key xxxx address 3.3.3.3
crypto isakmp key xxxx address 1.1.1.1
!
crypto ipsec transform-set CITS_1 esp-aes esp-sha512-hmac
!
crypto ipsec profile CIP_1
set transform-set CITS_1
!
interface Loopback0
ip address 10.0.39.51 255.255.255.255
!
interface Tunnel1002
description to "L2L"
ip address 10.144.226.4 255.255.255.254
ip mtu 1300
ip tcp adjust-mss 1260
tunnel source 2.2.2.2
tunnel mode ipsec ipv4
tunnel destination 1.1.1.1
tunnel protection ipsec profile CIP_1
!
interface Tunnel1005
description "L2L/R1003326/DSL1007952"
ip address 10.144.226.10 255.255.255.254
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 2.2.2.2
tunnel mode ipsec ipv4
tunnel destination 3.3.3.3
tunnel protection ipsec profile CIP_1
!
interface FastEthernet0
no ip address
!
interface FastEthernet1
no ip address
!
interface FastEthernet2
no ip address
!
interface FastEthernet3
no ip address
!
interface FastEthernet4
ip address 2.2.2.2 255.255.255.248
ip access-group F4-IN in
ip access-group F4-OUT out
duplex auto
speed auto
!
interface Vlan1
ip address 10.9.98.254 255.255.255.0
!
interface Dialer1
no ip address
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 82.118.127.233
ip route 10.1.10.0 255.255.255.0 10.9.98.192 permanent
ip route 10.9.99.0 255.255.255.0 10.9.98.192 permanent
ip route 10.11.4.0 255.255.255.0 10.9.98.192 permanent
ip route 10.144.144.0 255.255.255.0 Tunnel1002
ip route 10.145.0.0 255.255.254.0 Tunnel1005
ip route 192.168.254.0 255.255.255.0 Tunnel1005
!
ip access-list extended F4-IN
permit ip any any
permit ip object-group OGN_COMPANY any
permit udp object-group DNS-SERVERS eq domain any
permit udp any eq ntp any eq ntp
evaluate F4-REFLEX
deny ip any any log
ip access-list extended F4-OUT
permit ip any any reflect F4-REFLEX timeout 300
ip access-list extended VTY-IN
permit tcp object-group OGN_COMPANY any
permit tcp 10.9.99.0 0.0.0.255 any
permit tcp 10.9.98.0 0.0.0.255 any
permit tcp 192.168.254.0 0.0.0.255 any
permit tcp 192.168.252.0 0.0.0.255 any
deny ip any any
!
kron occurrence daily-backup at 4:00 recurring
policy-list daily-backup
!
kron policy-list daily-backup
cli write
!
logging facility local6
logging source-interface Vlan1
logging 10.9.99.1
logging host 10.9.99.1 transport tcp port 3951
!
snmp-server ifindex persist
!
control-plane
!
line con 0
no modem enable
line aux 0
line vty 0 4
access-class VTY-IN in
transport preferred ssh
transport input ssh
transport output ssh
!
ntp server uk.pool.ntp.org
end

Thanks,
Andrew

Francesco Molino · ‎09-25-2017

Thanks. Can you paste the full config of your 1st router? I wanted to validate also all groups used within your ACLs?

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

Francesco Molino · ‎09-25-2017

Can you share the output in a text file for below commands on both routers:

- sh cryp isak sa

- sh cryp ipsec sa

- sh ip int bri | ex unas

On your 1st router, I've seen some ACE without any protocols and those shouldn't work, like:

ip access-list extended ACL_VPN_L2L
permit object-group OGS_VPN_L2L object-group OGN_VPN_L2L any
permit object-group OGS_VPN_L2L any object-group OGN_VPN_L2L

ip access-list extended ACL_ZP_LO-OUT
permit object-group OGS_VPN_L2L any object-group OGN_VPN_L2L

ip access-list extended ACL_ZP_OUT-LO
permit object-group OGS_VPN_L2L object-group OGN_VPN_L2L any

Can you share the output on your 1st router for command:

- sh access-list ACL_VPN_L2L

- sh access-list ACL_ZP_LO-OUT

- sh access-list ACL_ZP_OUT-LO

Maybe it's just a copy/paste issue.

Except that, even with ZBF, you tunnel should be UP.

I'll wait for your object-groups to validate.

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

apstownsend · ‎09-26-2017

R1025079#sh crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
1.1.1.1 2.2.2.2 MM_SA_SETUP 0 ACTIVE
1.1.1.1 2.2.2.2 MM_SA_SETUP 0 ACTIVE
2.2.2.2 1.1.1.1 MM_NO_STATE 0 ACTIVE
2.2.2.2 1.1.1.1 MM_NO_STATE 0 ACTIVE (deleted)

IPv6 Crypto ISAKMP SA

R1025079#sh crypto ipsec sa

interface: Tunnel1002
Crypto map tag: Tunnel1002-head-0, local addr 1.1.1.1

protected vrf: (none)
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer 2.2.2.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0

local crypto endpt.: 1.1.1.1, remote crypto endpt.: 2.2.2.2
plaintext mtu 1492, path mtu 1492, ip mtu 1492, ip mtu idb Dialer1
current outbound spi: 0x0(0)
PFS (Y/N): N, DH group: none

inbound esp sas:

inbound ah sas:

inbound pcp sas:

outbound esp sas:

outbound ah sas:

outbound pcp sas:

R1025079#sh ip int bri | ex unas
Interface IP-Address OK? Method Status Protocol
Dialer1 1.1.1.1 YES IPCP up up
Loopback0 1.2.50.79 YES NVRAM up up
Tunnel1001 10.144.226.3 YES NVRAM up up
Tunnel1002 10.144.226.5 YES NVRAM up down

Vlan1 10.144.144.254 YES NVRAM up up

object-group network OGN_COMPANY
host 2.2.2.2
host 3.3.3.3
!
object-group network OGN_DNS
host 208.67.222.222
host 208.67.220.220
host 8.8.8.8
host 8.8.4.4
!
object-group network OGN_LAN
10.144.144.0 255.255.255.0
!
object-group network OGN_RFC1918
10.0.0.0 255.0.0.0
192.168.0.0 255.255.0.0
172.0.0.0 255.224.0.0
!
object-group network OGN_VPN_L2L
host 2.2.2.2
host 3.3.3.3
!
object-group service OGS_IN-OUT
tcp-udp eq 3389
tcp eq 993
tcp eq 5222
udp eq ntp
tcp eq 5223
!
object-group service OGS_VPN_L2L
gre
udp eq isakmp
udp eq non500-isakmp
icmp
esp
!

R1025079#sh access-l ACL_VPN_L2L
Extended IP access list ACL_VPN_L2L
10 permit object-group OGS_VPN_L2L object-group OGN_VPN_L2L any
20 permit object-group OGS_VPN_L2L any object-group OGN_VPN_L2L
30 permit gre any any
40 permit esp any any
50 permit udp any eq isakmp any eq isakmp

R1025079#sh access-l ACL_ZP_LO-OUT
Extended IP access list ACL_ZP_LO-OUT
10 permit icmp any any
20 permit tcp any object-group OGN_COMPANY eq 22
30 permit udp any object-group OGN_DNS eq domain
40 permit tcp any object-group OGN_DNS eq domain
50 permit udp any any eq ntp
60 permit object-group OGS_VPN_L2L any object-group OGN_VPN_L2L
70 permit udp any eq domain any

R1025079#sh access-l ACL_ZP_OUT-LO
Extended IP access list ACL_ZP_OUT-LO
10 permit icmp object-group OGN_COMPANY any
20 permit tcp object-group OGN_COMPANY any eq 22
30 permit object-group OGS_VPN_L2L object-group OGN_VPN_L2L any

apstownsend · ‎09-26-2017

Hi,

This is the remote side: -

R1003951#sh run
Building configuration...

Current configuration : 5853 bytes
!
! Last configuration change at 10:29:46 BST Fri Sep 22 2017 by COMPANY
! NVRAM config last updated at 04:00:00 BST Wed Sep 27 2017
! NVRAM config last updated at 04:00:00 BST Wed Sep 27 2017
version 15.1
no service pad
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
!
hostname R1003951
!
boot-start-marker
boot-end-marker
!
!
logging buffered 4096 informational
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication ppp default local
aaa authorization exec default local
aaa authorization network default local
!
!
!
!
!
aaa session-id common
clock timezone GMT 0 0
clock summer-time BST recurring last Sun Mar 1:00 last Sun Oct 2:00
crypto pki token default removal timeout 0
!
!
dot11 syslog
ip source-route
!
!
!
ip cef
ip inspect name INSPECT ntp
ip inspect name INSPECT icmp
ip domain name new.uk.COMPANY.co.uk
ip name-server 8.8.8.8
ip name-server 8.8.4.4
login on-failure log
login on-success log
!
!
!
!
archive
path /upload/cisco/backup/BK1003951
write-memory
time-period 1440
object-group network DNS-SERVERS
host 8.8.8.8
host 8.8.4.4
!
object-group service EX-IN-ALL
tcp eq ftp-data
!
object-group network OGN_COMPANY
host 1.1.1.1
host 3.3.3.3
!
!
no ip ftp passive
ip ftp username sd_ftp_cisco
ip ftp password 7 x
!
!
!
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 5
lifetime 3600
crypto isakmp key RgHRUM1oCb4khQJpPG2D address 3.3.3.3
crypto isakmp key JIcv5U3yB8gBg2x33Yfn address 1.1.1.1
!
!
crypto ipsec transform-set CITS_1 esp-aes esp-sha512-hmac
!
crypto ipsec profile CIP_1
set transform-set CITS_1
!
!
!
!
!
interface Loopback0
ip address 10.0.39.51 255.255.255.255
!
interface Tunnel1002
ip address 10.144.226.4 255.255.255.254
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 2.2.2.2
tunnel mode ipsec ipv4
tunnel destination 1.1.1.1
tunnel protection ipsec profile CIP_1
!
interface Tunnel1005
description "L2L/GRANTHAM/R1003326/DSL1007952"
ip address 10.144.226.10 255.255.255.254
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 2.2.2.2
tunnel mode ipsec ipv4
tunnel destination 3.3.3.3
tunnel protection ipsec profile CIP_1
!
interface FastEthernet0
no ip address
!
interface FastEthernet1
no ip address
!
interface FastEthernet2
no ip address
!
interface FastEthernet3
no ip address
!
interface FastEthernet4
ip address 2.2.2.2 255.255.255.248
ip access-group F4-IN in
ip access-group F4-OUT out
duplex auto
speed auto
!
interface Vlan1
ip address 10.9.98.254 255.255.255.0
!
interface Dialer1
no ip address
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 82.118.127.233
ip route 10.1.10.0 255.255.255.0 10.9.98.192 permanent
ip route 10.9.99.0 255.255.255.0 10.9.98.192 permanent
ip route 10.11.4.0 255.255.255.0 10.9.98.192 permanent
ip route 10.144.144.0 255.255.255.0 Tunnel1002
ip route 10.145.0.0 255.255.254.0 Tunnel1005
ip route 192.168.254.0 255.255.255.0 Tunnel1005
!
ip access-list extended F4-IN
permit ip any any
permit ip object-group OGN_COMPANY any
permit udp object-group DNS-SERVERS eq domain any
permit udp any eq ntp any eq ntp
evaluate F4-REFLEX
deny ip any any log
ip access-list extended F4-OUT
permit ip any any reflect F4-REFLEX timeout 300
ip access-list extended VTY-IN
permit tcp object-group OGN_COMPANY any
permit tcp 10.9.99.0 0.0.0.255 any
permit tcp 10.9.98.0 0.0.0.255 any
permit tcp 192.168.254.0 0.0.0.255 any
permit tcp 192.168.252.0 0.0.0.255 any
deny ip any any
!
kron occurrence daily-backup at 4:00 recurring
policy-list daily-backup
!
kron policy-list daily-backup
cli write
!
logging facility local6
logging source-interface Vlan1
logging 10.9.99.1
logging host 10.9.99.1 transport tcp port 3951
!
!
!
snmp-server community prtg RO SNMP
snmp-server community COMPANY RO
snmp-server ifindex persist
!
!
!
control-plane
!
!
line con 0
no modem enable
line aux 0
line vty 0 4
access-class VTY-IN in
transport preferred ssh
transport input ssh
transport output ssh
!
ntp server uk.pool.ntp.org
end

Francesco Molino · ‎09-27-2017

Hi

You shared the full config of remote site but not the local one. Can you drop the config for the local site in a text file please? It will be easier to read it, avoiding scroll down/up.

Anyway, I reproduced your design, just to be sure that there were not missing something (as I said before, the config looks good). The lab of your design works well.

Here the config I used. Let me know for the primary site if I'am in line with your production config?

In the mean time, have your run some debug for crypto ? and did you validate that ZBF isn't dropping anything else? to troubleshoot ZBFW if you don't have any experience in it, take a look at this post: https://supportforums.cisco.com/t5/security-documents/zbfw-troubleshooting-command-list/ta-p/3107683

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

apstownsend · ‎09-27-2017

Hi

Config attached.

I don't think it's an ACL issue as I get the same error when I add 'ip any any'.

The issue seems to be the port number in the debug output, I can't find anyone else who has the same issue! The source/peer port should be 500 for ISAKMP.

Sep 22 2017 11:21:15 BST: ISAKMP (0): received packet from 1.1.1.1 dport 500 sport 512 Global (I) MM_SA_SETUP
Sep 22 2017 11:21:15 BST: ISAKMP:(0): sending packet to 1.1.1.1 my_port 500 peer_port 512 (I) MM_SA_SETUP

The router does seem to have other quirks, it can't resolve names, and when attempting to ping a name it takes over 120 secs to time out. I can't access it remotely via SSH (despite the ACL getting a successful hit.) I just assumed I have made a mistake as it's the first time configuring IOS-XE and ZBF.

Thanks,
Andrew

Francesco Molino · ‎09-27-2017

Hi

Sorry I didn't noticed that before. I red it quickly.
Have you tried to downgrade to a recommended version like 16.3.4 ?

Source port can't be changed on the router.

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

ilkulagin · ‎10-30-2017

Hello Andrew.

I have absolutely the same problem with the same port numbers. If you found the solution could you please share it? I think many people would be really greatful for that.

Best regards,

Igor

apstownsend · ‎11-01-2017

Hello Igor,

I have yet to find a solution, if I do I will post it in this thread. If you have the same problem on the same model router it could be a hardware issue.

Thanks,

Andrew

apstownsend · ‎11-17-2017

Hi Igor,

Easy fix in the end. I altered the NAT ACL to just include the subnet of the local LAN: -

Before: -

object-group network OGN_RFC1918
 10.0.0.0 255.0.0.0
 192.168.0.0 255.255.0.0
 172.0.0.0 255.224.0.0

ip nat inside source list ACL_NAT interface Dialer1 overload

ip access-list extended ACL_NAT
 deny ip any object-group OGN_RFC1918
 permit ip any any

After: -

object-group network OGN_RFC1918
 10.0.0.0 255.0.0.0
 192.168.0.0 255.255.0.0
 172.0.0.0 255.224.0.0

ip nat inside source list ACL_NAT interface Dialer1 overload

ip access-list extended ACL_NAT
 deny ip any object-group OGN_RFC1918
 permit ip 192.168.1.0 255.255.255.0 any

This fixed my issue. 'permit ip any any' was always fine on IOS, although not recommended, whereas on IOS-XE it doesn't work (by design.)

Hopefully this helps you.

Thanks,

Andrew