cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2858
Views
0
Helpful
23
Replies

IPSEC TUNNEL..HUB & SPOKE Problem

sandeepkhera
Level 1
Level 1

Hi All,

I am facing a problem in site to site VPN in HUB and spoke topology.

Scenario : HUB routet (2811) is making Ipsec tunnel with 100 Spokes (851). with a common pre shared key. Around 90 tunnels are up but 10 tunnels are not coming up. There is similar config an IOS in each spoke router.  DPD is enabled at HUB site.

HUB crypto config is same for all the sites and access-lists are mirrored in HUB and spoke..

During debug of one of spoke i got following msgs,,..

*Mar  2 09:37:59.847: ISAKMP:(0):found peer pre-shared key matching x.x.x.x

*Mar  2 09:37:59.847: ISAKMP:(0): constructed NAT-T vendor-rfc3947 ID

*Mar  2 09:37:59.847: ISAKMP:(0): constructed NAT-T vendor-07 ID

*Mar  2 09:37:59.847: ISAKMP:(0): constructed NAT-T vendor-03 ID

*Mar  2 09:37:59.847: ISAKMP:(0): constructed NAT-T vendor-02 ID

!

!

*Mar  2 09:37:59.887: ISAKMP:(0): sending packet to x.x.x.x my_port 500 peer_port 500 (I) MM_SA_SETUP

*Mar  2 09:37:59.887: ISAKMP:(0):Sending an IKE IPv4 Packet.

*Mar  2 09:37:59.887: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

*Mar  2 09:37:59.887: ISAKMP:(0):Old State = IKE_I_MM2  New State = IKE_I_MM3

!

*Mar  2 09:38:01.887: ISAKMP:(0): phase 1 packet is a duplicate of a previous packet.

*Mar  2 09:38:01.887: ISAKMP:(0): retransmitting due to retransmit phase 1

*Mar  2 09:38:04.203: ISAKMP:(0): retransmitting phase 1 MM_SA_SETUP...

*Mar  2 09:38:04.203: ISAKMP (0:0): incrementing error counter on sa, attempt 1 of 5: retransmit phase 1

!(5 attemps)

*Mar  2 09:38:04.887: ISAKMP:(0):peer does not do paranoid keepalives.

*Mar  2 09:38:04.887: ISAKMP:(0):deleting SA reason "Death by retransmission P1" state (I) MM_SA_SETUP (peer x.x.x.x)

*Mar  2 09:38:04.887: ISAKMP:(0):deleting SA reason "Death by retransmission P1" state (I) MM_SA_SETUP (peer x.x.x.x)

*Mar  2 09:38:04.887: ISAKMP: Unlocking peer struct 0x82182080 for isadb_mark_sa_deleted(), count 0

*Mar  2 09:38:04.887: ISAKMP: Deleting peer node by peer_reap for x.x.x.x: 82182080

*Mar  2 09:38:04.887: ISAKMP:(0):deleting node -653888495 error FALSE reason "IKE deleted"

*Mar  2 09:38:04.887: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL

*Mar  2 09:38:04.887: ISAKMP:(0):Old State = IKE_I_MM3  New State = IKE_DEST_SA

It is showng NAT-T in debug but there is no nating happening in between..??

Please share your thoughts and findings on that.

Regards

S.K

1 Accepted Solution

Accepted Solutions

Hi,

Yes this helped confirm you are limited to 100 IKE SA's. The DH count for the most part is an internal construct, so you don't have to worry too much about it. Hope this helps.

Thanks,

Wen

View solution in original post

23 Replies 23

Marcin Latosiewicz
Cisco Employee
Cisco Employee

NAT-T you see is just VID exchange.

Looking at debugs I see that router you're poiting out sends MM2 message and waits for MM3.

Are you sure there is on ACL on the way that would block UDP/500?

It would explain, us always getting MM1 message and resending indefinetly MM2.

You can confirm by sniffing traffic or creating ACLs to match inbound udp/500 traffic.

Marcin

Hi Marcin,

Thnx for your reply.

Yah i m sure that there is no ACL in between that block UDP port 500.

As I mentioned that 100 spokes are connected with HUB router and 90 tunnels are up with same config.

Facing this same issue in 10 tunnels they are not coming up.

SK

Do you have any hardware ipsec vpn card in that hub router ? check for error logs on the 2811 if any ?

Thanks

Manish

There is no extra hardware card...Its a simple 851 router

No ipsec related logs on 2811

SK

Seems like a very strange issue, I have few questions :-

1> are all the spokes running same code version ?

2> do you really need DPD ? This issues of paranoide keepalive generally surfaces with DPD ?

Thanks

Manish

ya all spokes on same key.

but all other spokes are up on same HUB config for them..

DPD I have configured for continuous polling of peers. coz I think its better to be DPD.

SK

can you please check the IOS version on the 851 routers , one with active tunnel and other with tunnel not coming up ? It is a case of DPD incompatability that is leading to sa being deleted as far my expirence goes. here's a helpful link on DPD :-

http://www.cisco.com/en/US/docs/ios/12_3t/12_3t7/feature/guide/gtdpmo.html#wp1049827

Try turning off DPD , since your 2811 has 100 ipsec tunnels using software encryption it consumes a lot of CPU plus addtion of DPD will cause it to behave in an unexpected way as DPD takes up some CPU usage as well.

Thanks

Manish

IOS ver is same on both "c850-advsecurityk9-mz.124-15.T7.bin".

Is turning off DPD causes some outage?? coz its a live network and generating revenue..Any changes @ HUB side may b risky..

But we have another network on other 2811 and is running > 200 tunnels. only difference is that router uses different keys for all spokes..

SK


Guys,

I'm not sure which path you're going but:

- Did you capture/classify ingress packets on both sides to see if you're receiving both ways communication?

- IKE is not handled by hardware encryption module .... at least not on lowerend devices.

I would be relally curious to check if the communication is both ways on port udp/500 it's very easy.

You just add an entry in inbound access-list of external intrerface with:


On hub side:

1 permit udp h SPOKE_IP eq 500 any

On spoke side:

1 permit udp h HUB_IP eq 500 any

(By default Cisco routers if not behind NAT are using source and destination of 500)

Rememebr to add a "permit ip any any" at the end of that access-list if you're not using one already!

Marcin

Dear Marcin,

There is an ACL which is mirrored on both sites and getting hits...

Also i have checked with individual accesslists for UDP and IP and it is found that traffic is both ways.

I m sharing a full spoke debug of the scenario to show the clear picture what is happening.

Here it is showing that there is some issue in key xchange..dont know why it is happening

Regards

SK

SK.

There is a point where the hub is initiating ... odd!

Anyway.

*Mar  2 01:46:12.107: ISAKMP (0:0): received packet from 10.11.7.121 dport 500 sport 500 Global (N) NEW SA

We receive MM3 and send MM4.... and start MM_KEY_EXCHANGE.

*Mar  2 01:46:12.927: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH
*Mar  2 01:46:12.927: ISAKMP:(0):Old State = IKE_R_MM2  New State = IKE_R_MM3

*Mar  2 01:46:12.927: ISAKMP:(0): processing KE payload. message ID = 0
*Mar  2 01:46:12.971: ISAKMP:(0): processing NONCE payload. message ID = 0
*Mar  2 01:46:12.971: ISAKMP:(0):found peer pre-shared key matching 10.11.7.121
*Mar  2 01:46:12.971: ISAKMP:(2027): processing vendor id payload
*Mar  2 01:46:12.971: ISAKMP:(2027): vendor ID is Unity
*Mar  2 01:46:12.971: ISAKMP:(2027): processing vendor id payload
*Mar  2 01:46:12.971: ISAKMP:(2027): vendor ID is DPD
*Mar  2 01:46:12.971: ISAKMP:(2027): processing vendor id payload
*Mar  2 01:46:12.971: ISAKMP:(2027): speaking to another IOS box!
*Mar  2 01:46:12.971: ISAKMP:received payload type 20
*Mar  2 01:46:12.971: ISAKMP:received payload type 20
*Mar  2 01:46:12.971: ISAKMP:(2027):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
*Mar  2 01:46:12.971: ISAKMP:(2027):Old State = IKE_R_MM3  New State = IKE_R_MM3

*Mar  2 01:46:12.975: ISAKMP:(2027): sending packet to 10.11.7.121 my_port 500 peer_port 500 (R) MM_KEY_EXCH
*Mar  2 01:46:12.975: ISAKMP:(2027):Sending an IKE IPv4 Packet.
*Mar  2 01:46:12.975: ISAKMP:(2027):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
*Mar  2 01:46:12.975: ISAKMP:(2027):Old State = IKE_R_MM3  New State = IKE_R_MM4

We are expecting MM5 now. MM5 if any peer is behind NAT will be done on port udp/4500

Did you by any chance check inbound acl for udp/4500 packets?

We seem not to recieve MM5 from initiator.

Since both peers are IOS maybe we can get debug from both?

On hub you could do "debug crypto cond peer ipv4 ...." and the IPv4 address being spoke's public IP....

Marcin

thanks guys for helping me.

Finally with a struck i found the reason that why tunnels are not establishing.

When i exactly counted the no of tunnels than i found that HUB router is supporting only 100 tunnels and not more than that.

I don't know how it is happening but it is.. I have 3660 with IOS "c3660-jk9o3s-mz.124-15.T5.bin". As per its Datasheet it can support 2000 encrypted tunnels but it is not accepting more than 100 tunnels. we are using AES-128 encryption with SHA hashing algorithm for tunnels.

Is there any tool that we can use to calculate number of tunnels that a router can accept with particualr transform set??

Regards:

SK

i think 100 tunnel is a small number for 3800 series but iw ill let others comment on this too if they have seen something similar

have you enabled logs, can you enable them as debug level send to syslog and see if you see something there

also please paste the output of

sh crypto call admission statistics

when you are trying to establish tunnel

Hi,

If you are running the software crypto engine, then indeed you are running into the 100 IKE SA limit. Currently these limits are the
following for software crypto on the ISR routers:

- 1000 IPSec SAs
- 100 IKE SAs
- 50 DHs

If you need to scale more than the above, then we recommend you use an AIM vpn module, and you can find the tunnel scaling numbers in the following lilnk:

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6635/ps7180/prod_brochure09186a00801f0a72_ns125_Networking_Solutions_Brochure.html

Thanks,

Wen

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: