Route question, remote office, MPLS and local Internet, EIGRP, BGP and Default Gateway?

[usimplyc]

Have a weird (to me) routing issue and was hoping to get some insight on the "why".   I setup a remote office the other day for a client that included a router, switch and ASA.  Office traffic goes out the Ethernet MPLS and Internet goes out local Broadband service.  Client uses BGP for the MPLS and EIGRP for local routing.  I configured the equipment as per their info and setup EIGRP on all three devices as well as the BGP on the MPLS.  Dropped it all in and I could access all remote networks from the router console.  So far so good.  Also, all three devices had full EIGRP tables showing all remote networks.

Default routes at this time:  ROUTER->MPLS Network, SWITCH-> ROUTER, ASA->Broadband Network

• I setup a DHCP scope with the default route set to the ASA (has local route for Internet and EIGRP for Offices).  Internet worked fine but could not connect to remote networks.
• I changed the DHCP default route to point to the Router - ALL traffic went out the MPLS.  Good except they wanted local Internet access.

I changed the default route statements on the equipment.  MPLS-> ASA, SWITCH-> ASA  (ASA stayed the same)

• Changed the DHCP scope default route back to the MPLS
• Office traffic worked correctly, Internet went out local Internet

I understand that the Router, using EIGRP and BGP, allowed Office traffic to traverse the MPLS network while its default route to the ASA allowed for the local Internet.  The question is why didn't traffic flow correctly when the ASA was set as the DHCP scope default gateway when it also had a proper EIGRP route list?  Would traffic also fail to flow if the Switch was set as the DHCP scope default gateway?

I'm working on getting a copy of the EIGRP tables, but, could it possibly be that the Router redistributes   the WAN gateway as the next hop to the local EIGRP devices (Switch/ASA) so they were trying to pass L3 traffic to a non-reachable IP?

 Not sure, any insight would be welcomed, thanks!

[Jon Marshall]

The first thing that springs to mind is that the ASA will not redirect traffic back out of the same interface it receives it on without addditional configuration. 

So if the clients send traffic to the ASA for remote office subnets if the ASA sees the routes to those subnets back out of the same interface then it not forward it unless you have configured it to do so. 

Jon

[Richard Burts]

I am a little uncertain about the topology of the remote office network. Is it the case that the switch connects to the router, that the router connects on one interface to MPLS and on another interface to the ASA, and that the ASA connects to Broadband. Or is it the case that the switch connects to the router on one interface and connects to the ASA on another interface? It is significant to know whether traffic from the client transits the router in getting to the ASA. Can you clarify this?

Can you also clarify how the switch is operating. You identify the switch as a L3 switch. Is routing enabled on the switch? Or is the switch operating as L2 and doing just layer 2 forwarding of traffic?

HTH

Rick

[usimplyc]

Hi Rick,

Can do.   Router connects to the Switch and the Switch then connects to the ASA.  The switch is operating as a L3 device and running EIGRP (Client template for all remote offices).  

MPLS <-> ROUTER <-> SWITCH <-> ASA <-> BROADBAND

Thank you.

[Richard Burts]

Thank you for the clarification. It is helpful to know that users connect to the switch which is doing the layer 3 routing for user traffic. The switch will forward traffic toward the router for MPLS and toward the ASA for Internet. In this case the default route on the switch becomes critical. Is the switch default route a configured static default or is the switch default route learned from a routing protocol?

I have a secondary question about the MPLS. Does the MPLS advertise just the subnets of the various offices, or does it also advertise a default route?

HTH

Rick

[Jon Marshall]

Rick 

The switch was not routing for user traffic as far as I can tell. 

The two tests run were a DHCP scope with the default router set to the ASA and the same DHCP scope with the default router set to the MPLS router. 

There is no mention of setting the L3 switch as the default router. 

I suspect when it was the ASA it was not configured to redirect traffic back out of the same interface and when it was the MPLS router it was either receiving a default route via BGP or the ASA was not configured to redistribute it's default route into EIGRP. 

Jon

[Richard Burts]

Jon

Here is part of what the original poster has said

The switch is operating as a L3 device and running EIGRP (Client template for all remote offices).  

I think that the switch is routing user traffic and therefore the switch default route is a critical component of the issue.

HTH

Rick

 [edit] also notice the updated diagram about connectivity

MPLS <-> ROUTER <-> SWITCH <-> ASA <-> BROADBAND

If user traffic comes into the switch and goes to the left to get to the router/MPLS or goes to the right to get to the ASA/Internet then pretty clearly the switch must be making the routing decision.

[Jon Marshall]

Rick 

Re the last point, not necessarily because the users, the ASA and the MPLS router could all be in the same vlan. 

In fact if the DHCP scope was handing out the ASA and then the MPLS router as the default router then they must all be in the same vlan/IP subnet. 

Jon

[usimplyc]

Correct, this remote office has only the one VLAN although other remote offices have multiple.  I have not worked on those sites but I would think that the Switch's IP would need to be setup as the client's default gateway, yes?

Thanks!

[Jon Marshall]

As far as I can tell yes if you had multiple user vlans then you would route them on the L3 switch and either have a static default route pointing to the ASA or on the ASA redistribute the default route into EIGRP. 

There is no reason why you couldn't do this in this site even if you only have one user vlan but it is up to you really as I don't know what your standard configurations look like. 

Jon

[Richard Burts]

I agree with Jon that if the normal network implementation is to route user traffic on the switch with a default route on the switch pointing to the ASA and Internet and with specific networks learned from MPLS that you could certainly use that approach for this site which has only a single user subnet.

HTH

Rick

[Jon Marshall]

Rick 

I am not saying you are wrong but one thing is clear. 

You cannot set the default gateway to be the ASA or the MPLS router if the L3 switch is doing the routing for the client vlan and the original post quite clearly states that is what was done. 

Quite what the L3 switch is meant to be doing is a mystery but nowhere is it stated it was routing the client traffic. 

Jon

[Richard Burts]

Jon

I see your point. When the original poster said that the switch was operating as L3 switch I assumed that meant that the switch was routing user traffic. And I realize that I need to remember the lesson about assumptions. You are right that we do not know what is really going on here and need clarification from the original poster.

HTH

Rick

[usimplyc]

Rick,

The switch has a static (manually configured) default route that is currently set to the ASA.  The MPLS route advertisements do not include a default gateway, just the subnets.  

The switch is setup as a L3 router (standard client template) as some remote offices include multiple subnets although this jobsite has only the one.

Thanks!