dynamic ip FlexVPN Hub-and- dynamic ip multi Spoke

[katheer_4u]

Good day

Hi Everyone,
I'm having some trouble with FlexVPN in a Hub & multi Spoke i have both site are dynamic ip. actually the VPN is up but when i trying to configure the 2nd spoke 1 st one getting down and when i restrat the 1st one its getting VPN up but 2nd one its getting down

if anyone can help me i really appreciated

HUB

+++++++++++++++

HO-FLXVPN#show running-config
Building configuration...

Current configuration : 3455 bytes
!
! Last configuration change at 15:26:37 UTC Thu Jan 23 2020 by ciscouser
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname HO-FLXVPN
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-intf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family

!
aaa new-model
!
!
aaa authorization network default local
!
!
!
!
!
!
aaa session-id common
!
!
!
!
!
!
!
!
!

ip domain name mm.com

!
!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
!
!
license udi pid ISR4331/K9 sn FDO20432DC6
license boot level securityk9
spanning-tree extend system-id
!

!
redundancy
mode none
!
crypto ikev2 authorization policy POC-AUTH-POL-01
route set interface
route set access-list POC-ROUTE-ACL-01
!
crypto ikev2 authorization policy FLEX-AUTH-POL-01
route set interface
route set access-list FLEX-ROUTE-ACL-01
!
crypto ikev2 proposal POC-PROP-01
encryption aes-cbc-128
integrity sha256
group 19
!
crypto ikev2 policy POC-POL-01
match fvrf any
proposal POC-PROP-01
!
!
crypto ikev2 profile POC-IKEV2-PROFILE-01
match identity remote address 0.0.0.0
identity local address 192.168.100.222
authentication remote pre-share key 6 cisco123
authentication local pre-share key 6 cisco123
aaa authorization group psk list default FLEX-AUTH-POL-01
virtual-template 23
!
!
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
!
crypto ipsec transform-set POC-IPSEC-TS-01 esp-aes esp-md5-hmac
mode transport
!
crypto ipsec profile POC-IPSEC-PROF-01
set transform-set POC-IPSEC-TS-01
set ikev2-profile POC-IKEV2-PROFILE-01
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0/0
ip address 192.168.100.222 255.255.255.0
ip nat outside
negotiation auto
!
interface GigabitEthernet0/0/1
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet0/0/2
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet0/1/0
!
interface GigabitEthernet0/1/1
!
interface GigabitEthernet0/1/2
!
interface GigabitEthernet0/1/3
!
interface GigabitEthernet0
vrf forwarding Mgmt-intf
no ip address
shutdown
negotiation auto
!
interface Virtual-Template23 type tunnel
ip unnumbered Vlan1
ip nhrp network-id 23
ip nhrp redirect
tunnel source GigabitEthernet0/0/0
tunnel mode ipsec ipv4
tunnel protection ipsec profile POC-IPSEC-PROF-01
!
interface Vlan1
ip address 192.168.250.1 255.255.255.0
ip nat inside
!
ip nat inside source list 20 interface GigabitEthernet0/0/0 overload
ip forward-protocol nd
no ip http server
no ip http secure-server
ip tftp source-interface GigabitEthernet0/1/0
ip route 0.0.0.0 0.0.0.0 192.168.100.1
ip route 192.168.42.0 255.255.255.0 192.168.250.2
ip route 192.168.101.0 255.255.255.0 192.168.250.2
ip route 192.168.203.0 255.255.255.0 192.168.250.2
ip ssh rsa keypair-name mm.com
!
!
ip access-list standard FLEX-ROUTE-ACL-01
permit 192.168.101.0 0.0.0.255
permit 192.168.203.0 0.0.0.255
permit 192.168.42.0 0.0.0.255
ip access-list standard POC-ROUTE-ACL-01
permit any
!
access-list 20 permit 192.168.250.0 0.0.0.255
access-list 21 permit 192.168.203.0 0.0.0.255
!
!
!
!
!
control-plane
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
access-class 21 in
transport input all
!
!
end

Spoke

++++++++++++++++

Branch#show running-config
Building configuration...

Current configuration : 2825 bytes
!
version 15.3
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Branch
!
boot-start-marker
boot-end-marker
!
aqm-register-fnf
!

!
aaa new-model
!
!
aaa authorization network default local
!
!
!
!
!
aaa session-id common
!
!
!
!
!
!

!
!
!
!
ip name-server 216.146.35.35
ip name-server 216.146.36.36
ip cef
no ipv6 cef
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
license udi pid C887VAM-K9 sn FCZ1950929W
!
!

!
crypto ikev2 authorization policy FLEX-AUTH-POL-01
route set interface
route set access-list FLEX-ROUTE-ACL-01
!
crypto ikev2 proposal POC-PROP-01
encryption aes-cbc-128
integrity sha256
group 19
!
crypto ikev2 policy POC-POL-01
match fvrf any
proposal POC-PROP-01
!
!
crypto ikev2 profile POC-IKEV2-PROFILE-01
match identity remote address 0.0.0.0
identity local address 192.168.100.250
authentication remote pre-share key 6 cisco123
authentication local pre-share key 6 cisco123
aaa authorization group psk list default FLEX-AUTH-POL-01
virtual-template 23
!
crypto ikev2 client flexvpn POC-FLEX-CLIENT-01
peer 1 fqdn xxxxxxx.dyndns.org dynamic
client connect Tunnel23
!
!
!
controller VDSL 0
!
!
!
!
crypto ipsec transform-set POC-IPSEC-TS-01 esp-aes esp-md5-hmac
mode transport
!
crypto ipsec profile POC-IPSEC-PROF-01
set transform-set POC-IPSEC-TS-01
set ikev2-profile POC-IKEV2-PROFILE-01
!
!
!
!
!
!
!
!
interface Tunnel23
ip unnumbered Vlan1
ip nhrp network-id 23
ip nhrp redirect
tunnel source Vlan2
tunnel mode ipsec ipv4
tunnel destination dynamic
tunnel protection ipsec profile POC-IPSEC-PROF-01
!
interface ATM0
no ip address
shutdown
no atm ilmi-keepalive
!
interface Ethernet0
no ip address
shutdown
!
interface FastEthernet0
no ip address
!
interface FastEthernet1
no ip address
!
interface FastEthernet2
no ip address
!
interface FastEthernet3
switchport access vlan 2
no ip address
!
interface Vlan1
ip address 192.168.14.1 255.255.255.0
!
interface Vlan2
ip address 192.168.100.250 255.255.255.0
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
!
ip route 0.0.0.0 0.0.0.0 192.168.100.1
ip route 172.16.114.0 255.255.255.0 192.168.14.2
ip route 192.168.114.0 255.255.255.0 192.168.14.2
!
ip access-list standard FLEX-ROUTE-ACL-01
permit 192.168.114.0 0.0.0.255
!
!
!
!
!
control-plane
!
!
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
!
!
!
!
line con 0
no modem enable
line aux 0
line vty 0 4
transport input all
!
scheduler allocate 20000 1000
!
end

Branch#show crypto ikev2 sa detailed
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.250/4500 89.211.152.117/4500 none/none READY
Encr: AES-CBC, keysize: 128, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/17018 sec
CE id: 2001, Session-id: 1
Status Description: Negotiation done
Local spi: 7435398A5B040F61 Remote spi: 9A900A01F9641E5B
Local id: 192.168.100.250
Remote id: 192.168.100.222
Local req msg id: 5 Remote req msg id: 6
Local next msg id: 5 Remote next msg id: 6
Local req queued: 5 Remote req queued: 6
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : Yes
Remote subnets:
192.168.250.1 255.255.255.255
192.168.101.0 255.255.255.0
192.168.203.0 255.255.255.0
192.168.42.0 255.255.255.0

[Rob Ingram · Accepted solution]

Hi,
Can you enable some debugs "debug crypto ikev2" and "debug aaa authorization" on both the hub and spoke prior to establishing a VPN from the other spoke, then upload the output of when the tunnel fails.

On the spoke you are referencing a Virtual-Template which you don't have defined in your configuration, can you remove it and try again.

Can you also provide the configuration of the other spoke.

Can you try establishing the tunnel without authorisation just for testing, this would confirm whether it's authorisation that could be causing the issue.
HTH

[katheer_4u]

Hi
1st of all thanks a lot for the prompt reply
Please see the below Spoke 2 (Branch 2) configuration and now i'm wondering it's given new error even its not getting up and there is not any hit in hub 

New Spoke (branch 2) Error
++++++++++++++++

*Jan 23 18:15:30.000: Domain: query for xxxxx.dyndns.org type 1 to 255.255.255.255timed out
*Jan 23 18:15:42.000: Domain: query for xxxxx.dyndns.org type 1 to 255.255.255.255timed out
*Jan 23 18:15:54.000: Domain: query for xxxxx.dyndns.org type 1 to 255.255.255.255timed out
*Jan 23 18:16:05.999: Domain: query for xxxxx.dyndns.org type 1 to 255.255.255.255timed out
*Jan 23 18:16:17.999: Domain: query for xxxxx.dyndns.org type 1 to 255.255.255.255timed out
*Jan 23 18:16:29.999: Domain: query for xxxxx.dyndns.org type 1 to 255.255.255.255timed out

Branch2
+++++++++++++++++++

Branch2 #show running-config
Building configuration...

Current configuration : 2807 bytes
!
! Last configuration change at 18:14:01 UTC Thu Jan 23 2020
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Branch2
!
boot-start-marker
boot-end-marker
!
aqm-register-fnf
!
!
aaa new-model
!
aaa session-id common
bsd-client server url https://cloudsso.cisco.com/as/token.oauth2
!

ip cef
no ipv6 cef
!

multilink bundle-name authenticated
!

cts logging verbose

!
crypto ikev2 authorization policy FLEX-AUTH-POL-01
route set interface
route set access-list FLEX-ROUTE-ACL-01
!
crypto ikev2 proposal POC-PROP-01
encryption aes-cbc-128
integrity sha256
group 19
!
crypto ikev2 policy POC-POL-01
match fvrf any
proposal POC-PROP-01
!
!
crypto ikev2 profile POC-IKEV2-PROFILE-01
match identity remote address 0.0.0.0
identity local address 192.168.100.250
authentication remote pre-share key 6 cisco123
authentication local pre-share key 6 cisco123
!
crypto ikev2 client flexvpn POC-FLEX-CLIENT-01
peer 1 fqdn xxxxx.dyndns.org dynamic
client connect Tunnel23
!
controller VDSL 0

!
crypto ipsec transform-set POC-IPSEC-TS-01 estimed outp-aes esp-md5-hmac
mode transport
!
crypto ipsec profile POC-IPSEC-PROF-01
set transform-set POC-IPSEC-TS-01
set ikev2-profile POC-IKEV2-PROFILE-01
!
!
interface Tunnel23
ip unnumbered Vlan1
ip nhrp network-id 23
ip nhrp redirect
tunnel source Vlan2
tunnel mode ipsec ipv4
tunnel destination dynamic
tunnel protection ipsec profile POC-IPSEC-PROF-01
!
interface ATM0
no ip address
shutdown
no atm ilmi-keepalive
!
interface Ethernet0
no ip address
shutdown
!
interface FastEthernet0
no ip address
!
interface FastEthernet1
no ip address
!
interface FastEthernet2
no ip address
!
interface FastEthernet3
switchport access vlan 2
no ip address
!
interface Vlan1
ip address 192.168.21.1 255.255.255.0
!
interface Vlan2
ip address 192.168.100.250 255.255.255.0
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 192.168.100.1
ip route 172.16.121.0 255.255.255.0 192.168.21.2
ip route 192.168.121.0 255.255.255.0 192.168.21.2
!
ip access-list standard FLEX-ROUTE-ACL-01
permit 192.168.121.0 0.0.0.255
!

!
control-plane
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
!
line con 0
no modem enable
line aux 0
line vty 0 4
transport input all
!
scheduler allocate 20000 1000
!
end

[Rob Ingram · Accepted solution]

From that error it looks like the router cannot resolve the dyndns name. You don't appear to have your name-servers defined as you do on the other spoke.

ip name-server 216.146.35.35
ip name-server 216.146.36.36

...but if that was the case the VPN would not have established in the first place. In your initial post you mentioned the VPN on this 2nd spoke did briefly establish.

What about providing the debugs for review?

[katheer_4u]

Hi
Please see the below Spoke 2 (Branch 2) configuration and when i enter the ip name-server in branch 2 VPN was UP but branch 1 its down
see those logs and after i restart the branch 1 its up, now its seem both are VPN lights are up but unable to ping form the branch 2 to hub

in Hub there is no debugs for review

++++++++++++++++++++++++++++++++++++++++++++++++++++++

Branch 2 (config)#ip name-server 216.146.35.35
Branch 2 (config)#ip name-server 216.146.36.36

*Jan 23 19:31:07.487: Domain: query for xxxxx.dyndns.org type 1 to 316.146.35.35 Reply received ok
*Jan 23 19:31:07.647: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
*Jan 23 19:31:07.651: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 19:31:07.651: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 19:31:07.655: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH public key, DH Group 19
*Jan 23 19:31:07.655: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
*Jan 23 19:31:07.655: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH key
*Jan 23 19:31:07.655: IKEv2:IKEv2 initiator - no config data to send in IKE_SA_INIT exch
*Jan 23 19:31:07.655: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_SA_INIT message
*Jan 23 19:31:07.655: IKEv2:(SESSION ID = 1,SA ID = 1):IKE Proposal: 1, SPI size: 0 (initial negotiation),
Num. transforms: 4
AES-CBC SHA256 SHA256 DH_GROUP_256_ECP/Group 19

*Jan 23 19:31:07.655: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 190A0556C327099D - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 19:31:07.655: IKEv2:(SESSION ID = 1,SA ID = 1):Insert SA

*Jan 23 19:31:07.707: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:500/To 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 190A0556C327099D - Responder SPI : 83918630D3837E1A Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 19:31:07.707: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
*Jan 23 19:31:07.707: IKEv2:(SESSION ID = 1,SA ID = 1):Verify SA init message
*Jan 23 19:31:07.707: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
*Jan 23 19:31:07.711: IKEv2:(SESSION ID = 1,SA ID = 1):Checking NAT discovery
*Jan 23 19:31:07.711: IKEv2:(SESSION ID = 1,SA ID = 1):NAT INSIDE found
*Jan 23 19:31:07.711: IKEv2:(SESSION ID = 1,SA ID = 1):NAT detected float to init port 4500, resp port 4500
*Jan 23 19:31:07.711: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH secret key, DH Group 19
*Jan 23 19:31:07.731: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH secret
*Jan 23 19:31:07.731: IKEv2:(SA ID = 1):[IKEv2 -> Crypto Engine] Calculate SKEYSEED and create rekeyed IKEv2 SA
*Jan 23 19:31:07.731: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] SKEYSEED calculation and creation of rekeyed IKEv2 SA PASSED
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Completed SA init exchange
*Jan 23 19:31:07.731: IKEv2:Config data to send:
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-request
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-dns, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-dns, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-nbns, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-nbns, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv6-dns, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv6-subnet, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 242, data: Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.5(1)T1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Mon 02-Mar-15 04:13 by prod_rel_team
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: split-dns, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: banner, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: config-url, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: backup-gateway, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: def-domain, length: 0
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Have config mode data to send
*Jan 23 19:31:07.731: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Generate my authentication data
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 192.168.100.250, key len 14
*Jan 23 19:31:07.735: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
*Jan 23 19:31:07.735: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Get my authentication method
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):My authentication method is 'PSK'
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_AUTH message
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Constructing IDi payload: '192.168.100.250' of type 'IPv4 address'
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):ESP Proposal: 1, SPI size: 4 (IPSec negotiation),
Num. transforms: 3
AES-CBC MD596 Don't use ESN
*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Building packet for encryption.
Payload contents:
VID IDi AUTH CFG SA TSi TSr NOTIFY(INITIAL_CONTACT) NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

*Jan 23 19:31:07.735: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:4500/From 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 190A0556C327099D - Responder SPI : 83918630D3837E1A Message id: 1
IKEv2 IKE_AUTH Exchange REQUEST
Payload contents:
ENCR

*Jan 23 19:31:07.815: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:4500/To 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 190A0556C327099D - Responder SPI : 83918630D3837E1A Message id: 1
IKEv2 IKE_AUTH Exchange RESPONSE
Payload contents:
VID IDr AUTH CFG SA TSi TSr NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

*Jan 23 19:31:07.815: IKEv2:(SESSION ID = 1,SA ID = 1):Process auth response notify
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Searching policy based on peer's identity '192.168.100.222' of type 'IPv4 address'
*Jan 23 19:31:07.819: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 19:31:07.819: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's policy
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's policy verified
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's authentication method
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's authentication method is 'PSK'
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's preshared key for 192.168.100.222
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's authentication data
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 192.168.100.222, key len 14
*Jan 23 19:31:07.819: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
*Jan 23 19:31:07.819: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Verification of peer's authenctication data PASSED
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 19:31:07.819: IKEv2:Using mlist default and username FLEX-AUTH-POL-01 for group author request
*Jan 23 19:31:07.819: AAA/BIND(0000000D): Bind i/f
*Jan 23 19:31:07.819: AAA/AUTHOR (0xD): Pick method list 'default'
*Jan 23 19:31:07.819: IKEv2:(SA ID = 1):[IKEv2 -> AAA] Authorisation request sent
*Jan 23 19:31:07.819: IKEv2:(SA ID = 1):[AAA -> IKEv2] Received AAA authorisation response
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Received valid config mode data
*Jan 23 19:31:07.819: IKEv2:Config data recieved:
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-reply
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.250.1 255.255.255.255
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.101.0 255.255.255.0
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.203.0 255.255.255.0
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.42.0 255.255.255.0
*Jan 23 19:31:07.819: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 246, data: Cisco IOS Software, ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(2)S3, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 25-Mar-16 16:59 by mcpre
*Jan 23 19:31:07.823: IKEv2:(SESSION ID = 1,SA ID = 1):Set received config mode data
*Jan 23 19:31:07.823: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_AUTH message
*Jan 23 19:31:07.823: IKEv2:IPSec policy validate request sent for profile POC-IKEV2-PROFILE-01 with psh index 1.

*Jan 23 19:31:07.823: IKEv2:(SA ID = 1):[IPsec -> IKEv2] Callback received for the validate proposal - PASSED.

*Jan 23 19:31:07.823: IKEv2:(SESSION ID = 1,SA ID = 1):IKEV2 SA created; inserting SA into database. SA lifetime timer (86400 sec) started
*Jan 23 19:31:07.823: IKEv2:(SESSION ID = 1,SA ID = 1):Session with IKE ID PAIR (192.168.100.222, 192.168.100.250) is UP
*Jan 23 19:31:07.823: IKEv2:IKEv2 MIB tunnel started, tunnel index 1
*Jan 23 19:31:07.823: IKEv2:(SESSION ID = 1,SA ID = 1):Load IPSEC key material
*Jan 23 19:31:07.823: IKEv2:(SA ID = 1):[IKEv2 -> IPsec] Create IPsec SA into IPsec database
*Jan 23 19:31:07.827: IKEv2:(SA ID = 1):[IPsec -> IKEv2] Creation of IPsec SA into IPsec database PASSED
*Jan 23 19:31:07.827: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-set
*Jan 23 19:31:07.827: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.21.1 255.255.255.255
*Jan 23 19:31:07.827: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.121.0 255.255.255.0
*Jan 23 19:31:07.827: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 242, data: Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.5(1)T1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Mon 02-Mar-15 04:13 by prod_rel_team
*Jan 23 19:31:07.827: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel23, changed state to up
*Jan 23 19:31:07.831: IKEv2:(SESSION ID = 1,SA ID = 1):Checking for duplicate IKEv2 SA
*Jan 23 19:31:07.831: IKEv2:(SESSION ID = 1,SA ID = 1):No duplicate IKEv2 SA found
*Jan 23 19:31:07.831: IKEv2:(SESSION ID = 1,SA ID = 1):Sending info exch config
*Jan 23 19:31:07.831: IKEv2:(SESSION ID = 1,SA ID = 1):Building packet for encryption.
Payload contents:
CFG
*Jan 23 19:31:07.831: IKEv2:(SESSION ID = 1,SA ID = 1):Checking if request will fit in peer window

*Jan 23 19:31:07.831: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:4500/From 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 190A0556C327099D - Responder SPI : 83918630D3837E1A Message id: 2
IKEv2 INFORMATIONAL Exchange REQUEST
Payload contents:
ENCR

*Jan 23 19:31:07.831: %FLEXVPN-6-FLEXVPN_CONNECTION_UP: FlexVPN(POC-FLEX-CLIENT-01) Client_public_addr = 192.168.100.250 Server_public_addr = 37.211.149.33

*Jan 23 19:31:07.871: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:4500/To 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 190A0556C327099D - Responder SPI : 83918630D3837E1A Message id: 2
IKEv2 INFORMATIONAL Exchange RESPONSE
Payload contents:
CFG

*Jan 23 19:31:07.871: IKEv2:(SESSION ID = 1,SA ID = 1):Processing ACK to informational exchange
*Jan 23 19:31:07.871: IKEv2:Config data recieved:
*Jan 23 19:31:07.871: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-ack
*Jan 23 19:31:07.871: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 0
*Jan 23 19:31:07.871: IKEv2:(SESSION ID = 1,SA ID = 1):Set received config mode data

HUB (HO-FLXVPN)
+++++++++++

Gateway of last resort is 192.168.100.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 192.168.100.1
192.168.21.0/32 is subnetted, 1 subnets
S 192.168.21.1 is directly connected, Virtual-Access2
S 192.168.42.0/24 [1/0] via 192.168.250.2
192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.100.0/24 is directly connected, GigabitEthernet0/0/0
L 192.168.100.222/32 is directly connected, GigabitEthernet0/0/0
S 192.168.101.0/24 [1/0] via 192.168.250.2
S 192.168.121.0/24 is directly connected, Virtual-Access2
S 192.168.203.0/24 [1/0] via 192.168.250.2
192.168.250.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.250.0/24 is directly connected, Vlan1
L 192.168.250.1/32 is directly connected, Vlan1

But now still branch 2 VPN lights are UP but in hub there is no routing table or unable to ping hub

Branch2 #show crypto ikev2 sa detailed
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.250/4500 37.211.149.33/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/988 sec
CE id: 2001, Session-id: 1
Status Description: Negotiation done
Local spi: 190A0556C327099D Remote spi: 83918630D3837E1A
Local id: 192.168.100.250
Remote id: 192.168.100.222
Local req msg id: 3 Remote req msg id: 0
Local next msg id: 3 Remote next msg id: 0
Local req queued: 3 Remote req queued: 0
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : Yes
Remote subnets:
192.168.250.1 255.255.255.255
192.168.101.0 255.255.255.0
192.168.203.0 255.255.255.0
192.168.42.0 255.255.255.0

IPv6 Crypto IKEv2 SA

Branch 1 aftre restratrd
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Branch 1# ping 192.168.250.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.250.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/12 ms

Branch 1 #show crypto ikev2 sa detailed
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.250/4500 37.211.149.33/4500 none/none READY
Encr: AES-CBC, keysize: 128, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/265 sec
CE id: 2001, Session-id: 1
Status Description: Negotiation done
Local spi: 71B4A7780B5A9464 Remote spi: ECAABF7773997035
Local id: 192.168.100.250
Remote id: 192.168.100.222
Local req msg id: 3 Remote req msg id: 0
Local next msg id: 3 Remote next msg id: 0
Local req queued: 3 Remote req queued: 0
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : Yes
Remote subnets:
192.168.250.1 255.255.255.255
192.168.101.0 255.255.255.0
192.168.203.0 255.255.255.0
192.168.42.0 255.255.255.0

IPv6 Crypto IKEv2 SA

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
HUB (HO-FLXVP)

HO-FLXVPN#show crypto ikev2 sa detailed
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.222/4500 37.210.165.251/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/187 sec
CE id: 1003, Session-id: 3
Status Description: Negotiation done
Local spi: ECAABF7773997035 Remote spi: 71B4A7780B5A9464
Local id: 192.168.100.222
Remote id: 192.168.100.250
Local req msg id: 0 Remote req msg id: 3
Local next msg id: 0 Remote next msg id: 3
Local req queued: 0 Remote req queued: 3
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : No
Remote subnets:
192.168.14.1 255.255.255.255
192.168.114.0 255.255.255.0

IPv6 Crypto IKEv2 SA

+++++++++++++++++++++++++++++++++++++++++++++++

HO-FLXVPN#show crypto ikev2 sa detailed
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.222/4500 37.210.165.251/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/187 sec
CE id: 1003, Session-id: 3
Status Description: Negotiation done
Local spi: ECAABF7773997035 Remote spi: 71B4A7780B5A9464
Local id: 192.168.100.222
Remote id: 192.168.100.250
Local req msg id: 0 Remote req msg id: 3
Local next msg id: 0 Remote next msg id: 3
Local req queued: 0 Remote req queued: 3
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : No
Remote subnets:
192.168.14.1 255.255.255.255
192.168.114.0 255.255.255.0

IPv6 Crypto IKEv2 SA

now both branch 1 and branch 2 are showing crypto ikev2 sa detailed but im unable to ping branch 2 to hub

[Rob Ingram]

The branch2 spoke router is not sending it's local routes to the hub, you can determine this in the output of "show crypto ikev2 sa detail" on the hub. Add the authorisation policy "FLEX-AUTH-POL-01" to branch2 spokes' IKEv2 profile. This should send the 192.168.121.0 network to the hub.

HTH

[katheer_4u]

Hi

Please see the below branch 2 debug preview

Branch 2 

+++++++++++++++++++

EZDANWAKARA#
*Jan 23 20:33:24.002: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 23 20:33:24.002: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 82BDD478CC9187E5 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 20:33:24.790: %FLEXVPN-6-FLEXVPN_CONNECTION_DOWN: FlexVPN(POC-FLEX-CLIENT-01) Client_public_addr = 192.168.100.250 Server_public_addr = 37.211.149.33
*Jan 23 20:33:34.790: Domain: query for xxxxxxxx.dyndns.org type 1 to 216.146.35.35Reply received ok
*Jan 23 20:33:34.950: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 20:33:34.950: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 20:33:34.950: IKEv2:SA is already in negotiation, hence not negotiating again
*Jan 23 20:33:55.198: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 23 20:33:55.198: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 82BDD478CC9187E5 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 20:34:04.950: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 20:34:04.950: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 20:34:04.950: IKEv2:SA is already in negotiation, hence not negotiating again
*Jan 23 20:34:34.949: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 20:34:34.949: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 20:34:34.949: IKEv2:SA is already in negotiation, hence not negotiating again
*Jan 23 20:34:57.293: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 23 20:34:57.293: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 82BDD478CC9187E5 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:500/To 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 82BDD478CC9187E5 - Responder SPI : 6CB68A02F68DAF0D Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):Verify SA init message
*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):Checking NAT discovery
*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):NAT INSIDE found
*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):NAT detected float to init port 4500, resp port 4500
*Jan 23 20:34:57.361: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH secret key, DH Group 19
*Jan 23 20:34:57.385: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH secret
*Jan 23 20:34:57.385: IKEv2:(SA ID = 1):[IKEv2 -> Crypto Engine] Calculate SKEYSEED and create rekeyed IKEv2 SA
*Jan 23 20:34:57.385: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] SKEYSEED calculation and creation of rekeyed IKEv2 SA PASSED
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Completed SA init exchange
*Jan 23 20:34:57.385: IKEv2:Config data to send:
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-request
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-dns, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-dns, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-nbns, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-nbns, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv6-dns, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv6-subnet, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 242, data: Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.5(1)T1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Mon 02-Mar-15 04:13 by prod_rel_team
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: split-dns, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: banner, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: config-url, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: backup-gateway, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: def-domain, length: 0
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Have config mode data to send
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Generate my authentication data
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 192.168.100.250, key len 14
*Jan 23 20:34:57.385: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
*Jan 23 20:34:57.385: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Get my authentication method
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):My authentication method is 'PSK'
*Jan 23 20:34:57.385: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 20:34:57.389: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_AUTH message
*Jan 23 20:34:57.389: IKEv2:(SESSION ID = 1,SA ID = 1):Constructing IDi payload: '192.168.100.250' of type 'IPv4 address'
*Jan 23 20:34:57.389: IKEv2:(SESSION ID = 1,SA ID = 1):ESP Proposal: 1, SPI size: 4 (IPSec negotiation),
Num. transforms: 3
AES-CBC MD596 Don't use ESN
*Jan 23 20:34:57.389: IKEv2:(SESSION ID = 1,SA ID = 1):Building packet for encryption.
Payload contents:
VID IDi AUTH CFG SA TSi TSr NOTIFY(INITIAL_CONTACT) NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

*Jan 23 20:34:57.389: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:4500/From 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 82BDD478CC9187E5 - Responder SPI : 6CB68A02F68DAF0D Message id: 1
IKEv2 IKE_AUTH Exchange REQUEST
Payload contents:
ENCR

*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:4500/To 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 82BDD478CC9187E5 - Responder SPI : 6CB68A02F68DAF0D Message id: 1
IKEv2 IKE_AUTH Exchange RESPONSE
Payload contents:
VID IDr AUTH CFG SA TSi TSr NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Process auth response notify
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Searching policy based on peer's identity '192.168.100.222' of type 'IPv4 address'
*Jan 23 20:34:57.465: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 20:34:57.465: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's policy
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's policy verified
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's authentication method
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's authentication method is 'PSK'
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's preshared key for 192.168.100.222
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's authentication data
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 192.168.100.222, key len 14
*Jan 23 20:34:57.465: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
*Jan 23 20:34:57.465: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
*Jan 23 20:34:57.465: IKEv2:(SESSION ID = 1,SA ID = 1):Verification of peer's authenctication data PASSED
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 20:34:57.469: IKEv2:Using mlist default and username FLEX-AUTH-POL-01 for group author request
*Jan 23 20:34:57.469: AAA/BIND(00000011): Bind i/f
*Jan 23 20:34:57.469: AAA/AUTHOR (0x11): Pick method list 'default'
*Jan 23 20:34:57.469: IKEv2:(SA ID = 1):[IKEv2 -> AAA] Authorisation request sent
*Jan 23 20:34:57.469: IKEv2:(SA ID = 1):[AAA -> IKEv2] Received AAA authorisation response
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Received valid config mode data
*Jan 23 20:34:57.469: IKEv2:Config data recieved:
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-reply
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.250.1 255.255.255.255
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.101.0 255.255.255.0
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.203.0 255.255.255.0
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.42.0 255.255.255.0
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 246, data: Cisco IOS Software, ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(2)S3, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 25-Mar-16 16:59 by mcpre
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Error in settig received config mode data
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Auth exchange failed
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):: Auth exchange failed
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Abort exchange
*Jan 23 20:34:57.469: IKEv2:(SESSION ID = 1,SA ID = 1):Deleting SA
*Jan 23 20:35:04.949: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 20:35:04.949: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 20:35:04.949: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH public key, DH Group 19
*Jan 23 20:35:04.949: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
*Jan 23 20:35:04.949: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH key
*Jan 23 20:35:04.949: IKEv2:IKEv2 initiator - no config data to send in IKE_SA_INIT exch
*Jan 23 20:35:04.949: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_SA_INIT message
*Jan 23 20:35:04.949: IKEv2:(SESSION ID = 1,SA ID = 1):IKE Proposal: 1, SPI size: 0 (initial negotiation),
Num. transforms: 4
AES-CBC SHA256 SHA256 DH_GROUP_256_ECP/Group 19

*Jan 23 20:35:04.949: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 2EC958A5979924EA - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 20:35:04.949: IKEv2:(SESSION ID = 1,SA ID = 1):Insert SA

*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:500/To 192.168.100.250:500/VRF i0:f0]
Initiator SPI : 2EC958A5979924EA - Responder SPI : FB7F569B2415B41F Message id: 0
IKEv2 IKE_SA_INIT Exchange RESPONSE
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):Verify SA init message
*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_SA_INIT message
*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):Checking NAT discovery
*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):NAT INSIDE found
*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):NAT detected float to init port 4500, resp port 4500
*Jan 23 20:35:05.009: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH secret key, DH Group 19
*Jan 23 20:35:05.033: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH secret
*Jan 23 20:35:05.033: IKEv2:(SA ID = 1):[IKEv2 -> Crypto Engine] Calculate SKEYSEED and create rekeyed IKEv2 SA
*Jan 23 20:35:05.033: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] SKEYSEED calculation and creation of rekeyed IKEv2 SA PASSED
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Completed SA init exchange
*Jan 23 20:35:05.033: IKEv2:Config data to send:
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-request
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-dns, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-dns, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-nbns, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-nbns, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv6-dns, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv6-subnet, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 242, data: Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.5(1)T1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Mon 02-Mar-15 04:13 by prod_rel_team
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: split-dns, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: banner, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: config-url, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: backup-gateway, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: def-domain, length: 0
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Have config mode data to send
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Generate my authentication data
*Jan 23 20:35:05.033: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 192.168.100.250, key len 14
*Jan 23 20:35:05.033: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
*Jan 23 20:35:05.033: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):Get my authentication method
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):My authentication method is 'PSK'
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_AUTH message
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):Constructing IDi payload: '192.168.100.250' of type 'IPv4 address'
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):ESP Proposal: 1, SPI size: 4 (IPSec negotiation),
Num. transforms: 3
AES-CBC MD596 Don't use ESN
*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):Building packet for encryption.
Payload contents:
VID IDi AUTH CFG SA TSi TSr NOTIFY(INITIAL_CONTACT) NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

*Jan 23 20:35:05.037: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:4500/From 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 2EC958A5979924EA - Responder SPI : FB7F569B2415B41F Message id: 1
IKEv2 IKE_AUTH Exchange REQUEST
Payload contents:
ENCR

*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:4500/To 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 2EC958A5979924EA - Responder SPI : FB7F569B2415B41F Message id: 1
IKEv2 IKE_AUTH Exchange RESPONSE
Payload contents:
VID IDr AUTH CFG SA TSi TSr NOTIFY(SET_WINDOW_SIZE) NOTIFY(ESP_TFC_NO_SUPPORT) NOTIFY(NON_FIRST_FRAGS)

*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Process auth response notify
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Searching policy based on peer's identity '192.168.100.222' of type 'IPv4 address'
*Jan 23 20:35:05.113: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.250
*Jan 23 20:35:05.113: IKEv2:Found Policy 'POC-POL-01'
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's policy
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's policy verified
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's authentication method
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Peer's authentication method is 'PSK'
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Get peer's preshared key for 192.168.100.222
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Verify peer's authentication data
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Use preshared key for id 192.168.100.222, key len 14
*Jan 23 20:35:05.113: IKEv2:[IKEv2 -> Crypto Engine] Generate IKEv2 authentication data
*Jan 23 20:35:05.113: IKEv2:[Crypto Engine -> IKEv2] IKEv2 authentication data generation PASSED
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Verification of peer's authenctication data PASSED
*Jan 23 20:35:05.113: IKEv2:(SESSION ID = 1,SA ID = 1):Check for EAP exchange
*Jan 23 20:35:05.113: IKEv2:Using mlist default and username FLEX-AUTH-POL-01 for group author request
*Jan 23 20:35:05.113: AAA/BIND(00000012): Bind i/f
*Jan 23 20:35:05.113: AAA/AUTHOR (0x12): Pick method list 'default'
*Jan 23 20:35:05.113: IKEv2:(SA ID = 1):[IKEv2 -> AAA] Authorisation request sent
*Jan 23 20:35:05.117: IKEv2:(SA ID = 1):[AAA -> IKEv2] Received AAA authorisation response
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Received valid config mode data
*Jan 23 20:35:05.117: IKEv2:Config data recieved:
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-reply
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.250.1 255.255.255.255
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.101.0 255.255.255.0
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.203.0 255.255.255.0
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.42.0 255.255.255.0
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 246, data: Cisco IOS Software, ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(2)S3, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 25-Mar-16 16:59 by mcpre
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Set received config mode data
*Jan 23 20:35:05.117: IKEv2:(SESSION ID = 1,SA ID = 1):Processing IKE_AUTH message
*Jan 23 20:35:05.117: IKEv2:IPSec policy validate request sent for profile POC-IKEV2-PROFILE-01 with psh index 1.

*Jan 23 20:35:05.117: IKEv2:(SA ID = 1):[IPsec -> IKEv2] Callback received for the validate proposal - PASSED.

*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):IKEV2 SA created; inserting SA into database. SA lifetime timer (86400 sec) started
*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):Session with IKE ID PAIR (192.168.100.222, 192.168.100.250) is UP
*Jan 23 20:35:05.121: IKEv2:IKEv2 MIB tunnel started, tunnel index 1
*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):Load IPSEC key material
*Jan 23 20:35:05.121: IKEv2:(SA ID = 1):[IKEv2 -> IPsec] Create IPsec SA into IPsec database
*Jan 23 20:35:05.121: IKEv2:(SA ID = 1):[IPsec -> IKEv2] Creation of IPsec SA into IPsec database PASSED
*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-set
*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.21.1 255.255.255.255
*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 8, data: 192.168.121.0 255.255.255.0
*Jan 23 20:35:05.121: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: app-version, length: 242, data: Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.5(1)T1, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Mon 02-Mar-15 04:13 by prod_rel_team
*Jan 23 20:35:05.121: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel23, changed state to up
*Jan 23 20:35:05.125: IKEv2:(SESSION ID = 1,SA ID = 1):Checking for duplicate IKEv2 SA
*Jan 23 20:35:05.125: IKEv2:(SESSION ID = 1,SA ID = 1):No duplicate IKEv2 SA found
*Jan 23 20:35:05.125: IKEv2:(SESSION ID = 1,SA ID = 1):Sending info exch config
*Jan 23 20:35:05.125: IKEv2:(SESSION ID = 1,SA ID = 1):Building packet for encryption.
Payload contents:
CFG
*Jan 23 20:35:05.125: IKEv2:(SESSION ID = 1,SA ID = 1):Checking if request will fit in peer window

*Jan 23 20:35:05.125: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:4500/From 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 2EC958A5979924EA - Responder SPI : FB7F569B2415B41F Message id: 2
IKEv2 INFORMATIONAL Exchange REQUEST
Payload contents:
ENCR

*Jan 23 20:35:05.125: %FLEXVPN-6-FLEXVPN_CONNECTION_UP: FlexVPN(POC-FLEX-CLIENT-01) Client_public_addr = 192.168.100.250 Server_public_addr = 37.211.149.33

*Jan 23 20:35:05.153: IKEv2:(SESSION ID = 1,SA ID = 1):Received Packet [From 37.211.149.33:4500/To 192.168.100.250:4500/VRF i0:f0]
Initiator SPI : 2EC958A5979924EA - Responder SPI : FB7F569B2415B41F Message id: 2
IKEv2 INFORMATIONAL Exchange RESPONSE
Payload contents:
CFG

*Jan 23 20:35:05.153: IKEv2:(SESSION ID = 1,SA ID = 1):Processing ACK to informational exchange
*Jan 23 20:35:05.153: IKEv2:Config data recieved:
*Jan 23 20:35:05.153: IKEv2:(SESSION ID = 1,SA ID = 1):Config-type: Config-ack
*Jan 23 20:35:05.153: IKEv2:(SESSION ID = 1,SA ID = 1):Attrib type: ipv4-subnet, length: 0
*Jan 23 20:35:05.153: IKEv2:(SESSION ID = 1,SA ID = 1):Set received config mode data

[Rob Ingram]

Ok, so does the hub now have the branch2 spoke routes? (I can see in the debugs the routes appear to be sent)
Can you now ping?

[katheer_4u]

hi

still im unable to ping the hub to branch 2 to hub and hub to branch 2

but i can see the brach 2 routing table have hub route 

but in the hub doesn't have the branch to route 

branch 2 #ping 192.168.250.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.250.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

HUB

HO-FLXVPN#show crypto ikev2 sa detail
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
2 192.168.100.222/4500 37.210.165.251/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/630 sec
CE id: 1006, Session-id: 6
Status Description: Negotiation done
Local spi: 4E6743C21E7F38E9 Remote spi: 35C7F3F6FD62FC3B
Local id: 192.168.100.222
Remote id: 192.168.100.250
Local req msg id: 0 Remote req msg id: 3
Local next msg id: 0 Remote next msg id: 3
Local req queued: 0 Remote req queued: 3
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : No
Remote subnets:
192.168.14.1 255.255.255.255
192.168.114.0 255.255.255.0

IPv6 Crypto IKEv2 SA

HO-FLXVPN#ping 192.168.21.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.21.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5

++++++++++++++++++++++++++++++++++++++++++++++++++++

Branch 2 #show crypto ikev2 sa detail
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.250/4500 37.211.149.33/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/1167 sec
CE id: 2003, Session-id: 2
Status Description: Negotiation done
Local spi: 2EC958A5979924EA Remote spi: FB7F569B2415B41F
Local id: 192.168.100.250
Remote id: 192.168.100.222
Local req msg id: 3 Remote req msg id: 0
Local next msg id: 3 Remote next msg id: 0
Local req queued: 3 Remote req queued: 0
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : Yes
Remote subnets:
192.168.250.1 255.255.255.255
192.168.101.0 255.255.255.0
192.168.203.0 255.255.255.0
192.168.42.0 255.255.255.0

IPv6 Crypto IKEv2 SA

[katheer_4u · Accepted solution]

hi

in the branch 2 already add the in the ikev2 authorization policy FLEX-AUTH-POL-01

ip access-list standard FLEX-ROUTE-ACL-01
permit 192.168.121.0 0.0.0.255

crypto ikev2 authorization policy FLEX-AUTH-POL-01
route set interface
route set access-list FLEX-ROUTE-ACL-01

crypto ikev2 profile POC-IKEV2-PROFILE-01
match identity remote address 0.0.0.0
identity local address 192.168.100.250
authentication remote pre-share key 6 cisco123
authentication local pre-share key 6 cisco123
aaa authorization group psk list default FLEX-AUTH-POL-01
virtual-template 23

hub 

+++++++++++++++++++++++

HO-FLXVPN#show crypto ikev2 sa detail
IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
2 192.168.100.222/4500 37.210.165.251/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/630 sec
CE id: 1006, Session-id: 6
Status Description: Negotiation done
Local spi: 4E6743C21E7F38E9 Remote spi: 35C7F3F6FD62FC3B
Local id: 192.168.100.222
Remote id: 192.168.100.250
Local req msg id: 0 Remote req msg id: 3
Local next msg id: 0 Remote next msg id: 3
Local req queued: 0 Remote req queued: 3
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : No
Remote subnets:
192.168.14.1 255.255.255.255
192.168.114.0 255.255.255.0

IPv6 Crypto IKEv2 SA

[Rob Ingram · Accepted solution]

The authorisation policy was not defined in the ikev2 profile in the initial configuration you provided.

Regardless, it looks like you have defined the same local identity under the ikev2 profile of both spokes. The need to be unique, that's going to confuse the hub.

crypto ikev2 profile POC-IKEV2-PROFILE-01
identity local address 192.168.100.250 << change this on spoke2

[katheer_4u]

now it's working
thanks you so much such for your valuable time
i'm really appreciate your help
thanks a lot

[katheer_4u]

Good day

Yesterday i was changed the local IP and the branch 2 VPN was UP and when i came and checked today 

branch 1 still up but branch 2 its down please see the debug 

*Jan 24 10:50:29.712: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.251
*Jan 24 10:50:29.712: IKEv2:Found Policy 'POC-POL-01'
*Jan 24 10:50:29.712: IKEv2:SA is already in negotiation, hence not negotiating again
*Jan 24 10:50:30.548: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 24 10:50:30.548: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.251:500/VRF i0:f0]
Initiator SPI : 5F5F0F97996045F0 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 24 10:50:35.536: IKEv2:(SESSION ID = 1,SA ID = 1):: Maximum number of retransmissions reached
*Jan 24 10:50:35.536: IKEv2:(SESSION ID = 1,SA ID = 1):Failed SA init exchange
*Jan 24 10:50:35.536: IKEv2:(SESSION ID = 1,SA ID = 1):Initial exchange failed: Initial exchange failed
*Jan 24 10:50:35.536: IKEv2:(SESSION ID = 1,SA ID = 1):Abort exchange
*Jan 24 10:50:35.536: IKEv2:(SESSION ID = 1,SA ID = 1):Deleting SA
*Jan 24 10:50:35.536: %FLEXVPN-6-FLEXVPN_CONNECTION_DOWN: FlexVPN(POC-FLEX-CLIENT-01) Client_public_addr = 192.168.100.251 Server_public_addr = 37.211.149.33
*Jan 24 10:50:45.535: Domain: query for xxxxxxxxxxx.dyndns.org type 1 to 216.146.36.36Reply received ok
*Jan 24 10:50:46.079: IKEv2:Searching Policy with fvrf 0, local address 192.168.100.251
*Jan 24 10:50:46.079: IKEv2:Found Policy 'POC-POL-01'
*Jan 24 10:50:46.083: IKEv2:(SESSION ID = 1,SA ID = 1):[IKEv2 -> Crypto Engine] Computing DH public key, DH Group 19
*Jan 24 10:50:46.083: IKEv2:(SA ID = 1):[Crypto Engine -> IKEv2] DH key Computation PASSED
*Jan 24 10:50:46.083: IKEv2:(SESSION ID = 1,SA ID = 1):Request queued for computation of DH key
*Jan 24 10:50:46.083: IKEv2:IKEv2 initiator - no config data to send in IKE_SA_INIT exch
*Jan 24 10:50:46.083: IKEv2:(SESSION ID = 1,SA ID = 1):Generating IKE_SA_INIT message
*Jan 24 10:50:46.083: IKEv2:(SESSION ID = 1,SA ID = 1):IKE Proposal: 1, SPI size: 0 (initial negotiation),
Num. transforms: 4
AES-CBC SHA256 SHA256 DH_GROUP_256_ECP/Group 19

*Jan 24 10:50:46.083: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.251:500/VRF i0:f0]
Initiator SPI : 92B5064D643128E6 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 24 10:50:46.083: IKEv2:(SESSION ID = 1,SA ID = 1):Insert SA
*Jan 24 10:50:48.075: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 24 10:50:48.075: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.251:500/VRF i0:f0]
Initiator SPI : 92B5064D643128E6 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 24 10:50:51.971: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 24 10:50:51.971: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.251:500/VRF i0:f0]
Initiator SPI : 92B5064D643128E6 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 24 10:50:59.591: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 24 10:50:59.591: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.251:500/VRF i0:f0]
Initiator SPI : 92B5064D643128E6 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)

*Jan 24 10:51:14.227: IKEv2:(SESSION ID = 1,SA ID = 1):Retransmitting packet

*Jan 24 10:51:14.227: IKEv2:(SESSION ID = 1,SA ID = 1):Sending Packet [To 37.211.149.33:500/From 192.168.100.251:500/VRF i0:f0]
Initiator SPI : 92B5064D643128E6 - Responder SPI : 0000000000000000 Message id: 0
IKEv2 IKE_SA_INIT Exchange REQUEST
Payload contents:
SA KE N VID VID NOTIFY(NAT_DETECTION_SOURCE_IP) NOTIFY(NAT_DETECTION_DESTINATION_IP)


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

IPv4 Crypto IKEv2 SA

Tunnel-id Local Remote fvrf/ivrf Status
1 192.168.100.251/500 37.211.149.33/500 none/none IN-NEG
Encr: Unknown - 0, PRF: Unknown - 0, Hash: None, DH Grp:0, Auth sign: Unknown - 0, Auth verify: Unknown - 0
Life/Active Time: 86400/0 sec
CE id: 0, Session-id: 0
Status Description: Initiator waiting for INIT response
Local spi: A7045B1350320EF9 Remote spi: 0000000000000000
Local id: 192.168.100.251
Remote id:
Local req msg id: 0 Remote req msg id: 0
Local next msg id: 1 Remote next msg id: 0
Local req queued: 0 Remote req queued: 0
Local window: 1 Remote window: 1
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is not detected
Cisco Trust Security SGT is disabled
Initiator of SA : Yes

IPv6 Crypto IKEv2 SA


++++++++++++++++++++++++++++++++++++++++++++++++++++++++

HO HUB there is no remote

Tunnel-id Local Remote fvrf/ivrf Status
2 192.168.100.222/4500 37.210.165.251/4500 none/none READY
Encr: AES-CBC, keysize: 128, PRF: SHA256, Hash: SHA256, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/8896 sec
CE id: 1013, Session-id: 11
Status Description: Negotiation done
Local spi: F99A87FC7A5E1858 Remote spi: 526757D69A6D6DBF
Local id: 192.168.100.222
Remote id: 192.168.100.250
Local req msg id: 2 Remote req msg id: 5
Local next msg id: 2 Remote next msg id: 5
Local req queued: 2 Remote req queued: 5
Local window: 5 Remote window: 5
DPD configured for 0 seconds, retry 0
Fragmentation not configured.
Extended Authentication not configured.
NAT-T is detected inside
Cisco Trust Security SGT is disabled
Initiator of SA : No
Remote subnets:
192.168.14.1 255.255.255.255
192.168.114.0 255.255.255.0

IPv6 Crypto IKEv2 SA

[Rob Ingram · Accepted solution]

I assume that debug is from branch2 spoke, can that branch2 router even communicate/ping the hub?

Can you run ikev2 debug on the hub when you attempt to establish the tunnel please, upload the output here.

Can you upload the current configuration of the hub and both spokes, please upload them as text file makes it easier to review.

[katheer_4u]

Hi
Please see the attached hub, branch1 spoke and branch2 spoke. yes even i cant ping from the branch2 spoke to hub or hub to branch2 spoke and im useing both branch1 spoke and branch2 spoke singel fqdn XXXXX.dyndns.org and hud not have it . and its haveing port forwarding iplocal ip form the broadband router