Solved: Multiple site VPN with Metro-e

vsb2211 · ‎02-20-2019

Looking for some configuration help. I have 4 locations connected by Cox Metro-Ethernet via Cisco 2901 Routers. Metro-E is on Interface Gig0/1 of each router. Routing and connectivity is working between all sites. Now I need to encrypt any traffic for my internal subnets that go across this Metro-E. Sites B,C,D do not have direct internet access all traffic is routed down to SiteA via Metro-E. Metro-E has no internet access just site to site mesh. SiteA has a separate internet connection via a separate Firewall, So SiteA has a default route to this firewall for any internet traffic. I'm about to pull my hair out, I've tried all the examples I could find regarding multiple site-to-site VPN's and none have worked. I can usually only get SiteA to SiteB VPN working then trying to get SiteA to SiteC or D fails.

SiteA

------

SiteA#show run

Building configuration...

Current configuration : 917 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteA

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX152446J3-

license boot module c2900 technology-package securityk9

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 192.1.1.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.10 255.255.255.0

duplex auto

speed auto

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 192.1.1.253

ip route 192.1.2.0 255.255.255.0 10.10.10.20

ip route 192.1.3.0 255.255.255.0 10.10.10.30

ip route 172.24.4.0 255.255.255.0 10.10.10.40

!

ip flow-export version 9

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

-----

SiteB

SiteB#show run

Building configuration...

Current configuration : 871 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteB

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX1524V60N-

license boot module c2900 technology-package securityk9

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 192.1.2.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.20 255.255.255.0

duplex auto

speed auto

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.10.10

ip route 172.24.4.0 255.255.255.0 10.10.10.40

ip route 192.1.3.0 255.255.255.0 10.10.10.30

!

ip flow-export version 9

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

--------

SiteC

SiteC#show run

Building configuration...

Current configuration : 871 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteC

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX15246A9N-

license boot module c2900 technology-package securityk9

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 192.1.3.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.30 255.255.255.0

duplex auto

speed auto

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.10.10

ip route 172.24.4.0 255.255.255.0 10.10.10.40

ip route 192.1.2.0 255.255.255.0 10.10.10.20

!

ip flow-export version 9

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

-----

SiteD

SiteD#show run

Building configuration...

Current configuration : 871 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteD

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX1524K59R-

license boot module c2900 technology-package securityk9

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 172.24.4.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.40 255.255.255.0

duplex auto

speed auto

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.10.10

ip route 192.1.3.0 255.255.255.0 10.10.10.30

ip route 192.1.2.0 255.255.255.0 10.10.10.20

!

ip flow-export version 9

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

-----

Richard Burts · ‎02-28-2019

I am not convinced that in the live network you would necessarily need the static arp. In the live environment routers do not reboot often and once they get arp entries for the peer routers they will periodically refresh the arp entries and keep the mac address of the peer router in their arp table dynamically. But on the live router you certainly will have the option to configure static entries in the arp table if you want to do that.

HTH

Rick

HTH

Rick

View solution in original post

Richard Burts · ‎02-27-2019

The configs that you posted are basic configs that do implement the networking that you describe. 4 sites connected by Metro Ethernet. Site A provides connectivity to the Internet. Each site communicates directly with any other of the sites for traffic within the Enterprise and each other site uses site A for Internet access. You do not show us what you have tried to do for encrypting traffic so we do not know what worked and what did not work.

In preparing to implement encryption between sites there is a basic question that you need to consider. It is clear that you need to provide encryption when sites B, C, ad D communicate with site A. But what about when site B communicates with site C? If you follow the current model in which each site communicates directly with each other site then you need to implement something like DMVPN which supports direct communication between spoke sites. If you want to change from the current model then you could configure 3 site to site vpn on site A. And you could make the network into more of a hub and spoke topology in which site B communicating with site C would have B forward its traffic to A and A would forward to C. Either one could work. The choice is yours.

HTH

Rick

HTH

Rick

vsb2211 · ‎02-28-2019

Hi Rick,

Thanks for responding, I kind of had a mini-breakthrough last night and was going to post an update this morning. So I've managed to get all sites VPN working and encrypting. However, if the routers are rebooted I have to ping each router form every other router first....if not then I can only get one VPN up, the others will never connect. I have no idea what is going on with that. So basically

Step 1) when I start up Cisco Packet Tracer and all my routers load up I ping from Site A router to Site B,C,D routers. Then I ping from site C to D and C to B all from the routers. I don't specify an interface just basic ping to the LAN interface of each sites router.

Step 2) I ping from the workstations connected at each site to the other workstations at the other sites and all VPN tunnels get established or ping from the routers specifying the source address as the local LAN address.

If I skip Step 1 and just try Step 2 I can get one VPN tunnel going but then I can't get any other VPN tunnel to connect.

Below are my configs for each of the routers with the crypto info.

Site A

Building configuration...

Current configuration : 1792 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteA

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX152446J3-

license boot module c2900 technology-package securityk9

!

crypto isakmp policy 1

authentication pre-share

!

crypto isakmp key cisco123 address 10.10.10.20

crypto isakmp key cisco456 address 10.10.10.30

crypto isakmp key cisco789 address 10.10.10.40

!

crypto ipsec transform-set set1 esp-des esp-md5-hmac

crypto ipsec transform-set set2 esp-des esp-md5-hmac

crypto ipsec transform-set set3 esp-des esp-md5-hmac

!

crypto map cmap 1 ipsec-isakmp

set peer 10.10.10.20

set transform-set set1

match address 160

!

crypto map cmap 2 ipsec-isakmp

set peer 10.10.10.30

set transform-set set2

match address 170

!

crypto map cmap 3 ipsec-isakmp

set peer 10.10.10.40

set transform-set set3

match address 180

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 192.1.1.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.10 255.255.255.0

duplex auto

speed auto

crypto map cmap

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 192.1.1.253

ip route 192.1.2.0 255.255.255.0 10.10.10.20

ip route 192.1.3.0 255.255.255.0 10.10.10.30

ip route 172.24.4.0 255.255.255.0 10.10.10.40

!

ip flow-export version 9

!

access-list 160 permit ip 192.1.1.0 0.0.0.255 192.1.2.0 0.0.0.255

access-list 170 permit ip 192.1.1.0 0.0.0.255 192.1.3.0 0.0.0.255

access-list 180 permit ip 192.1.1.0 0.0.0.255 172.24.4.0 0.0.0.255

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

Site B

Building configuration...

Current configuration : 1792 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteB

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX1524V60N-

license boot module c2900 technology-package securityk9

!

crypto isakmp policy 1

authentication pre-share

!

crypto isakmp key cisco123 address 10.10.10.10

crypto isakmp key cisco456 address 10.10.10.30

crypto isakmp key cisco789 address 10.10.10.40

!

crypto ipsec transform-set set1 esp-des esp-md5-hmac

crypto ipsec transform-set set2 esp-des esp-md5-hmac

crypto ipsec transform-set set3 esp-des esp-md5-hmac

!

crypto map cmap 1 ipsec-isakmp

set peer 10.10.10.10

set transform-set set1

match address 160

!

crypto map cmap 2 ipsec-isakmp

set peer 10.10.10.30

set transform-set set2

match address 170

!

crypto map cmap 3 ipsec-isakmp

set peer 10.10.10.40

set transform-set set3

match address 180

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 192.1.2.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.20 255.255.255.0

duplex auto

speed auto

crypto map cmap

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.10.10

ip route 172.24.4.0 255.255.255.0 10.10.10.40

ip route 192.1.3.0 255.255.255.0 10.10.10.30

ip route 192.1.1.0 255.255.255.0 10.10.10.10

!

ip flow-export version 9

!

access-list 160 permit ip 192.1.2.0 0.0.0.255 192.1.1.0 0.0.0.255

access-list 170 permit ip 192.1.2.0 0.0.0.255 192.1.3.0 0.0.0.255

access-list 180 permit ip 192.1.2.0 0.0.0.255 172.24.4.0 0.0.0.255

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

Site C

Building configuration...

Current configuration : 1792 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteC

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX15246A9N-

license boot module c2900 technology-package securityk9

!

crypto isakmp policy 1

authentication pre-share

!

crypto isakmp key cisco456 address 10.10.10.10

crypto isakmp key cisco456 address 10.10.10.20

crypto isakmp key cisco789 address 10.10.10.40

!

crypto ipsec transform-set set1 esp-des esp-md5-hmac

crypto ipsec transform-set set2 esp-des esp-md5-hmac

crypto ipsec transform-set set3 esp-des esp-md5-hmac

!

crypto map cmap 1 ipsec-isakmp

set peer 10.10.10.10

set transform-set set1

match address 170

!

crypto map cmap 2 ipsec-isakmp

set peer 10.10.10.20

set transform-set set2

match address 160

!

crypto map cmap 3 ipsec-isakmp

set peer 10.10.10.40

set transform-set set3

match address 180

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 192.1.3.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.30 255.255.255.0

duplex auto

speed auto

crypto map cmap

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.10.10

ip route 172.24.4.0 255.255.255.0 10.10.10.40

ip route 192.1.2.0 255.255.255.0 10.10.10.20

ip route 192.1.1.0 255.255.255.0 10.10.10.10

!

ip flow-export version 9

!

access-list 170 permit ip 192.1.3.0 0.0.0.255 192.1.1.0 0.0.0.255

access-list 160 permit ip 192.1.3.0 0.0.0.255 192.1.2.0 0.0.0.255

access-list 180 permit ip 192.1.3.0 0.0.0.255 172.24.4.0 0.0.0.255

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

Site D

Building configuration...

Current configuration : 1794 bytes

!

version 15.1

no service timestamps log datetime msec

no service timestamps debug datetime msec

no service password-encryption

!

hostname SiteD

!

ip cef

no ipv6 cef

!

license udi pid CISCO2901/K9 sn FTX1524K59R-

license boot module c2900 technology-package securityk9

!

crypto isakmp policy 1

authentication pre-share

!

crypto isakmp key cisco789 address 10.10.10.10

crypto isakmp key cisco789 address 10.10.10.20

crypto isakmp key cisco789 address 10.10.10.30

!

crypto ipsec transform-set set1 esp-des esp-md5-hmac

crypto ipsec transform-set set2 esp-des esp-md5-hmac

crypto ipsec transform-set set3 esp-des esp-md5-hmac

!

crypto map cmap 1 ipsec-isakmp

set peer 10.10.10.10

set transform-set set1

match address 180

!

crypto map cmap 2 ipsec-isakmp

set peer 10.10.10.30

set transform-set set2

match address 170

!

crypto map cmap 3 ipsec-isakmp

set peer 10.10.10.20

set transform-set set3

match address 160

!

spanning-tree mode pvst

!

interface GigabitEthernet0/0

description Link to Switch

ip address 172.24.4.254 255.255.255.0

duplex auto

speed auto

!

interface GigabitEthernet0/1

description Metro-E

ip address 10.10.10.40 255.255.255.0

duplex auto

speed auto

crypto map cmap

!

interface Vlan1

no ip address

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.10.10

ip route 192.1.3.0 255.255.255.0 10.10.10.30

ip route 192.1.2.0 255.255.255.0 10.10.10.20

ip route 192.1.1.0 255.255.255.0 10.10.10.10

!

ip flow-export version 9

!

access-list 180 permit ip 172.24.4.0 0.0.0.255 192.1.1.0 0.0.0.255

access-list 170 permit ip 172.24.4.0 0.0.0.255 192.1.3.0 0.0.0.255

access-list 160 permit ip 172.24.4.0 0.0.0.255 192.1.2.0 0.0.0.255

!

line con 0

!

line aux 0

!

line vty 0 4

login

!

end

Richard Burts · ‎02-28-2019

Glad to know that you have made some progress. I have several comments about what you have posted.

I see that you are implementing a full mesh of vpn tunnels where each router has a vpn tunnel to each of the other routers. I started to go through each of the configs and to match parameters configured on a router with parameters configured on each of the other routers. Then I realized that if all of the vpn tunnels come up successfully and pass traffic successfully (which I think you say is the case) then the config parameters must match ok.

So on to the questions you are asking.

You tell us "if the routers are rebooted I have to ping each router form every other router first." That sort of makes sense. If a router reboots then its arp table is empty. ping from every other router is a way to get the appropriate entries into the arp table so that the routers will be able to communicate with each other. You can check and verify this by rebooting a router and then doing show arp.

There is sort of similar logic about needing step 1 and step 2. You need step 1 to populate the arp table of the router with the mac address of each of its peers. And you need step 2 because a vpn tunnel comes up when there is interesting traffic that must be encrypted and sent through the tunnel.

You also tell us "If I skip Step 1 and just try Step 2 I can get one VPN tunnel going but then I can't get any other VPN tunnel to connect" I am puzzled at that. Do you always start from site A? Would the behavior change if you started from some other site? Do you always begin by starting the vpn to the next site (A to B, B to C, etc). Would the behavior change if you changed the order in which you attempt to start them? I could perhaps understand it better if you said that without step 1 that step 2 did not work. But if step 2 is able to start a single vpn then I am quite puzzled why it can not start a second vpn. I wonder if there is some aspect of Packet Tracer that causes this.

HTH

Rick

HTH

Rick

vsb2211 · ‎02-28-2019

I was suspecting the ARP tables last night and you are correct when the routers reload or reboot the ARP table is cleared. There might be some limitations with Packet Tracer....I can't find how to add a static ARP entry in packet tracer. It's not even an option in the global config, however if I check my production 2901 it does have the ARP command. I bet if I set static ARP entries on the routers when I roll this out to production, it probably will work just fine, I just wish I could test it in packet tracer first.

Richard Burts · ‎02-28-2019

I am not convinced that in the live network you would necessarily need the static arp. In the live environment routers do not reboot often and once they get arp entries for the peer routers they will periodically refresh the arp entries and keep the mac address of the peer router in their arp table dynamically. But on the live router you certainly will have the option to configure static entries in the arp table if you want to do that.

HTH

Rick

HTH

Rick

vsb2211 · ‎02-28-2019

Thanks Rick! I'll post back once I go into production.

Richard Burts · ‎02-28-2019

I am glad that my explanations have been helpful. Thank you for marking this question as solved. This will make it easier for other participants in the community to identify discussions which have helpful information. Please do let us know as you get this into the live network.

HTH

Rick

HTH

Rick