MPLS failover to Comcast issue(s)

redneckgeek · ‎03-21-2016

Long time forum reader here, first time poster...
Looking for some assistance in trying to get this dialed in.

We're a car dealership and a few months ago, we received a 'audit' from the manufacturer.
In a nutshell, we needed to install a 2nd ISP should the primary MPLS go down, and we opted for Comcast Business.

Unfortunately, the MPLS provider has 100% control over the router, and anytime we need to make changes, we have to call into their helpdesk. Since Comcast is not 'them' the response was 'Sorry, Charlie.' (Personally, I've got no problems taking over the data side, but since they're also giving us SIP and a pseudo T1 for ShoreTel, I've been extremely gun-shy in demanding we get access to the router that we own.)
In the meantime, I've deployed a 2821 ISR to the site. I have, after hours, verified that the crypto map is setup and establishes an IPSec tunnel to our main-site and transmits data.
Tweaks were required (denying the explicit /24s to the OUTGOING ACL and adding the permits to the CRYPTO_LIST ACL at the main-site).

In order for the remote site to connect via the tunnel, I believe I will need to setup HSRP (and since we have zero access to the MPLS router, might cause an issue) or, the most likely scenario, utilize IP SLA, but that will be saved for a different topic.

Please forgive me because I don't know how to ask the question properly, but herein lies my problem / question:

Since I'm going from my core to the edge (at remote-site), how can I properly setup the main-site access-list to know if it's MPLS or IPSec tunnel?
I can manually add the remote site's IP's to the ACL and traffic will flow, but I'm quite unsure how to get it to know when traffic is coming in over MPLS or the IPSec tunnel.

Since the main-site-edge is the main router, I can't readily tweak the ACL's without breaking the existing site connectivity.

I would certianly like to get this resolved as we would like to be able to have backup internet installed at each site.
I'm beginning to think that the easiest thing would be to take control over the router and setup everything internally in there, but I'm starting to have doubts about that, too.

Of course, we have until the end of the month to get everything setup, and Comcast only got off their keisters last week to finish the buildout.

Any assistance would be greatly appreciated.

Thanks,

J

main-site-edge:

crypto dynamic-map IPSEC_IN 10
 set security-association lifetime seconds 86400
 set transform-set ESP_AES256_SHA1
 match address CRYPTO_LIST
!
ip access-list extended CRYPTO_LIST
 permit ip 10.0.0.0 0.255.255.255 192.168.16.0 0.0.3.255
 permit ip 192.168.239.0 0.0.0.3 192.168.30.2 0.0.0.1
!

remote-site-edge:

crypto map IPSEC 10 ipsec-isakmp
 set peer 192.0.2.45
 set security-association lifetime seconds 86400
 set transform-set ESP_AES256_SHA1
 match address CRYPTO_TO_MAIN
 crypto map IPSEC
!
ip access-list extended CRYPTO_TO_MAIN
 permit ip 10.x.166.0 0.0.0.255 10.0.0.0 0.255.255.255
 permit ip 10.x.40.0 0.0.0.255 10.0.0.0 0.255.255.255
 permit ip 10.x.115.0 0.0.0.255 10.0.0.0 0.255.255.255
!

Philip D'Ath · ‎03-21-2016

If it was me I would take over control of everything, but then I am a network engineer.

You have a 3750G at the remote site - a layer 3 switch. Presumably you have a layer 3 switch at the main site as well.

I would tell the MPLS provider you want them to dynamically advertise to you (probably via something simple like RIPv2) the remote site subnets, at each site. The layer 3 switches will learn this. Make sure the layer 3 switches are the default gateway at each site.

Over the second ISP I would build simple encrypted GRE tunnels. Run RIP over this as well, but with a worse metric than the MPLS network.

Then your layer 3 switches should learn two paths to the other. A better once via MPLS. If the MPLS goes down, after about 90s they'll forget that path leaving only the path via the Internet tunnel.

redneckgeek · ‎03-22-2016

I have the tunnel up and running, and at the remote site, change over the route. I think my issue lies at the main site. (I included a quick layout of the site in the initial post.)

The tunnels aren't an issue, but the main site is. The router connected to the MPLS is managed by them but we have a 2921 on the other side, from there, we doll out public IP's via NAT. It also is the head-end from my home (not included in the chart) so I have a constant connection w/o having to fire up a VPN client.

The issue I'm running into is at the main site. At the main site, the L3 is the default route for all subnets, connected to our router via a /31 on a routed interface to our router, then out to the interwebs.

The router at the main site has ACLs in place for 'normal' operation.

ip nat inside source list OUTGOING interface GigabitEthernet0/1 overload

and the OUTGOING ACL:

ip access-list extended OUTGOING
 deny ip 10.0.0.0 0.255.255.255 192.168.16.0 0.0.3.255
 deny ip any 10.0.0.0 0.255.255.255
 deny ip any 192.168.0.0 0.0.255.255
 deny ip host 10.16.239.193 any
 deny ip host 10.16.239.194 any
 deny tcp host 10.16.239.200 eq 443 any
 deny ip 192.168.200.0 0.0.0.255 any
 permit tcp host 10.16.239.183 any eq smtp
 permit tcp host 10.16.239.179 any eq smtp
 permit tcp host 10.229.239.10 any eq smtp
 permit tcp host 10.16.239.200 any eq smtp
 permit tcp host 10.228.239.100 any eq smtp
 deny tcp any any eq smtp log
 <SNIP>

The only way I can get the tunnel to pass data is if, when the link goes down, to manually add a deny statement for each subnet above the permits in order to pass data over the IPSec.

On top of that, once the tunnel is up, the main-edge router knows that the remote site is now over the IPSec tunnel and will continue to transmit data over it even once the MPLS returns. Only until I get on the remote site and until I issue a 'clear crypto peer <IP>' or 'clear crypto sa peer <IP' the tunnel stays UP-ACTIVE and continues to pass data.

But, I need to work on one issue at a time, and right now, that is: how can I properly setup the ACLs to route the traffic?

Or do I, as you said, change all of that and instead create a GRE tunnel? And with that, will I have to create a GRE tunnel for each site as we implement Comcast as backup inernet?