converting from MPLS to point to point via site-to-site tunnels

[Lee Dress]

We are in the process of converting our remote offices from MPLS to Site-to-Site via ASA Tunnels.

I have no control over the routing of the MPLS, and it takes 30 days to get it turned off.

This site in question is circled in Blue.

my current configuration is like this

My plan for the site is to remove it from the MPLS and move it to a tunnel like this:

I can build the tunnel successfully between both ASA Devices, but I can't reroute traffic because the MPLS Router in my site thinks it knows the way to get to the 50 site.  I've told the core switch to route the 50 traffic via the ASA, but it goes through the router first, and I guess the router ignores my route.

ip default-gateway 172.16.100.2 (This is the ASA Address)

ip classless

ip route 0.0.0.0 0.0.0.0 172.16.100.2

ip route 172.16.100.0 255.255.255.0 192.168.3.254

ip route 192.168.50.0 255.255.255.0 172.16.100.2

My question is this.

Can I add a true internal address to an interface on my ASA, Attach it to the Core switch, and route the 50 traffic through that?

Like this?

Eventually the MPLS Network is going to go away anyway, so I'm thinking eventually this will need to happen anyway, or I'll need to add another router to make a new DMZ.

[Lee Dress]

first off, thank you again for all of your help here..

right now the route on the core switch points to the AT&T router.

HQ core switch -> AT&T Router (3.254)  -> MPLS ->Remote (50.254)

Remote -> MPLS -> AT&T Router -> HQ core

at the remote site, when I disconnect the MPLS router  (50.254) and replace it with the remote ASA, I will use the same 50.254  so the remote site will still THINK it's the same route. 

everything in that site points to 50.254 as the default gateway.

at that point I will switch my core's routing to go through 3.253, so now the route will look like this:

HQ -> NEW Router (3.253) -> ASA -> VPN Tunnel-> Remote (50.254)

Remote -> VPN Tunnel -> ASA -> NEW Router(3.253) -> HQ

I  changed the route on my core switch to my NY Site (192.168.0.x) last night to make sure  that the routes from a working VPN Tunnel will route properly, and that  one is working fine.

Tracing route to [192.168.0.4]

over a maximum of 30 hops:

  1    <1 ms    11 ms    <1 ms  coresw1 [192.168.3.251]

  2    <1 ms    <1 ms    <1 ms  nj_router [192.168.3.253]

  3    16 ms    23 ms    16 ms  nydc [192.168.0.4]

[Jon Marshall]

Lee

HQ -> NEW Router (3.253) -> ASA -> VPN Tunnel-> Remote (50.254)

Remote -> VPN Tunnel -> ASA -> NEW Router(3.253) -> HQ

Unless i am missing something it won't.  You need to reread my last post.

The return path will go via the AT&T MPLS router. Have a look at your HQ ASA route table at the moment. Pick a subnet from the core switch and see what route is used by the HQ ASA to send traffic for that subnet. It will be via the AT&T router at HQ.

Adding a route to the core switch for site 50s subnet only affects outbound traffic not return traffic.  The return traffic will get to the HQ ASA firewall and then be sent via the AT&T MPLS router to your core switch.

From my last post, assume 192.168.5.1 is on your HQ core switch then trace it through the network hop by hop to site 50 and then back from site 50.

The issue is not at site 50 it is once traffic gets back to the ASA at HQ. Like i say, it might not be an issue but only you can say. The VPN tunnel will still work because it is only after the traffic has left the firewall at HQ to get back to the core switch that the path is different.

Please have a reread of my last post and make sure you fully understand it. I just want to make sure any solution you implement you fully understand how it is working which will make it much easier later on when you start making other changes.

Edit - unless you are actually proposing to have all  HQ internet traffic (ie VPN and non VPN) go via the new router and you have changed the routes on the ASA for all the internal subnets on the core switch and pointed them to the new router, in which case please just ignore all of the above.

Jon

[Lee Dress]

I tried switching the site over to the tunnel, and it did not work.

the MPLS router is disconnected at the remote site, but traffic stll refuses to return.

it could be maybe an arp cache on the remote site that makes it think that 50.254 is a different hardware address.

I might have to try this over a weekend and see if it clears up.

I'll re-read all of your posts tonight.

Thanks again..

[Lee Dress]

you are right about the problem here, but it's going to take a weekend to fix for a permanent solution

my ASA tells every inside network to go through the AT&T MPLS Router to get back in.

that's confusing the 50 route.

[Jon Marshall]

Lee

It shouldn't stop the VPN from working though, it just means a different return path. It can be fixed but it's really up to you whether you want/need to do it.

I don't think this is what stopped your test from working.

Jon

[Lee Dress]

I was just informed that the 50 site will be disconnected on January 3rd, so when that's done, the VPN should work right away.

since I have no control over the MPLS Routing, I'm at their mercy.

that's why I'm trying to work "around" their circuit.

I'm just not 100% sure how I should work around it, and if I'll break my other sites.

I was thinking of pointing these inside addresses to the new router's DMZ address, and repointing my core switch to the lan side of the new router.

the only problem is I don't know how the MPLS DMZ routes, so I might just make a total mess.

[Jon Marshall]

Lee

I was thinking of pointing these inside addresses to the new router's DMZ address, and repointing my core switch to the lan side of the new router.

the only problem is I don't know how the MPLS DMZ routes, so I might just make a total mess.

Outbound from the core switch I suspect it simply routes to the MPLS cloud for remote sites and if it doesn't have a route it sends it to the ASA HQ for internet traffic.

Return traffic from the internet it merely routes it to the core switch.

I wouldn't point all the subnets via the new router because then you are going to be in a bit of a mess. What happens for MPLS traffic from your core site if you bypass the MPLS router ?

I don't think the routing at HQ stopped the tunnel coming up. I think it was more likely, as you say, an issue at site 50 with arp caches etc.

I would leave your core switch pointing to the AT&T router for now. If you want return traffic from site 50s VPN to go via the new router we can NAT outbound on the router but be aware that would mean updating the VPN config on both ASAs because the address range would be different.

I think the new router should only be used for site 50 at the moment otherwise you could break all connectivity.

I would say again though, i do not think anything at HQ caused the tunnel to not work.  That said if you want site 50s return traffic to go via the new router we can setup NAT and update the VPN config on the ASAs.

Jon

[Lee Dress]

Found my problem. 

The NAT rule on my ASA at HQ was wrong. 

everything else was

it was set to allow traffic between site 50 and HQ on the outside interface as opposed to the inside interface.

once I changed the NAT rule for the 50 site, and rerouted my internal traffic to the new router, the whole thing lit up.

I'm writing up the whole procedure so I don't make the same mistake again.

I have 3 more sites I need to switch over when they each get better internet connections installed.

the good thing is that I have a new router installed so I don't have to wait for a site to be disconnected from MPLS to switch them over.

Jon,

thanks again for all of your help. 

[Jon Marshall]

Lee

No problem, glad you got it all working.

Jon