Solved: Routing to multiple Default Gateways

davidjknapp · ‎02-17-2010

This may be a simple process, but I can not find a difinity answer. Hope someone can help. I am designing a template that allows for multiple sites to connect to an MPLS network as well as support a separate isolated network at each site. the isolated network will need to route to some of the subnets so I need to point the gateway to the core (3750). The MPLS connected network will also be pointed to its respective vlan ip on the core switch for the site. The issue I have is I need the isolated network to utilize a local internet connection while the MPLS side of the house will be connected up to the primary hub via the MPLS.

In the example below VLAN 100 and 120 will be routed to the MPLS via the switches default gateway; VLAN 200 and 220 need to be routed to the local gateway (10.2.4.1). So if my default route is as configured below, how do I redirect LANS 200 and 220?

interface GigabitEthernet1/0/1
description Network 1
switchport access vlan 100
!
interface GigabitEthernet1/0/2
description Network 2
switchport access vlan 120
!
interface GigabitEthernet1/0/3
description Network 3
switchport access vlan 200
!
interface GigabitEthernet1/0/4
description Network 4
switchport access vlan 220
!
interface Vlan100
description LAN1
ip address 10.100.100.1 255.255.255.0
!
interface Vlan120
description LAN2
ip address 10.4.123.254 255.255.255
!
interface Vlan200
description LAN3
ip address 10.100.200.1 255.255.255.0
!
interface Vlan220
description LAN4
ip address 10.2.4.254 255.255.255.0
!
Route 0.0.0.0 0.0.0.0 10.4.123.1

Jon Marshall · ‎02-17-2010

davidjknapp wrote:

This may be a simple process, but I can not find a difinity answer. Hope someone can help. I am designing a template that allows for multiple sites to connect to an MPLS network as well as support a separate isolated network at each site. the isolated network will need to route to some of the subnets so I need to point the gateway to the core (3750). The MPLS connected network will also be pointed to its respective vlan ip on the core switch for the site. The issue I have is I need the isolated network to utilize a local internet connection while the MPLS side of the house will be connected up to the primary hub via the MPLS.

In the example below VLAN 100 and 120 will be routed to the MPLS via the switches default gateway; VLAN 200 and 220 need to be routed to the local gateway (10.2.4.1). So if my default route is as configured below, how do I redirect LANS 200 and 220?

David

There are generally 2 approaches to this -

1) PBR (Policy Based routing) where you can route traffic based on the source IP address so you could tell your switch that for vlan 200 and 220 it should route traffic to 10.2.4.1

For PBR support you need IPServices on your 3750 and you need to enable the routing template ie. "sdm prefer routing"

2) VRF-lite. If your networks are islolated totally then you could use vrf-lite where you end up with literally 2 virtual routers on the same physical switch. vlan 100/120 would be in one virtual router and vlan 200/220 would be in the other. Each virtual router has completely separate routing tables so they could each have their own default route.

It depends on whether vlans 200/220 need to communicate with vlans 100/120 and vice-versa. If they do then i would go the PBR route. If they don't i would look at VRF-lite.

Jon

View solution in original post

Jon Marshall · ‎02-22-2010

davidjknapp wrote:

http://www.cisco.com/en/US/tech/tk364/technologies_configuration_example09186a00802135d3.shtml

David

Did you enable the routing template ie. "sdm prefer routing"

Jon

View solution in original post

Jon Marshall · ‎02-23-2010

David

Glad you got it working.

As for directing traffic the key thing to be aware of is that only traffic matched in the acl will be sent to the next-hop specified in your route-map. And so it's all a question of configuring your acl properly so lets you wanted a host (192.168.5.1) to go across the MPLS network to the antivirus server but out the other link for http traffic

access-list 101 deny ip host 192.168.5.1 host

access-list 101 permit tcp host 192.168.5.1 any eq http

route-map PBR permit 10

match ip address 101

set ip next-hop x.x.x.x

because the first line is a deny it will not be sent the next-hop of x.x.x.x but routed via the normal routing table. The second line for http traffic is a permit and so will be sent to the next-hop ip of x.x.x.x.

So you just need to be specific in your acls that you use with your route-maps.

Jon

View solution in original post

Jon Marshall · ‎02-17-2010

davidjknapp wrote:

This may be a simple process, but I can not find a difinity answer. Hope someone can help. I am designing a template that allows for multiple sites to connect to an MPLS network as well as support a separate isolated network at each site. the isolated network will need to route to some of the subnets so I need to point the gateway to the core (3750). The MPLS connected network will also be pointed to its respective vlan ip on the core switch for the site. The issue I have is I need the isolated network to utilize a local internet connection while the MPLS side of the house will be connected up to the primary hub via the MPLS.

In the example below VLAN 100 and 120 will be routed to the MPLS via the switches default gateway; VLAN 200 and 220 need to be routed to the local gateway (10.2.4.1). So if my default route is as configured below, how do I redirect LANS 200 and 220?

David

There are generally 2 approaches to this -

1) PBR (Policy Based routing) where you can route traffic based on the source IP address so you could tell your switch that for vlan 200 and 220 it should route traffic to 10.2.4.1

For PBR support you need IPServices on your 3750 and you need to enable the routing template ie. "sdm prefer routing"

2) VRF-lite. If your networks are islolated totally then you could use vrf-lite where you end up with literally 2 virtual routers on the same physical switch. vlan 100/120 would be in one virtual router and vlan 200/220 would be in the other. Each virtual router has completely separate routing tables so they could each have their own default route.

It depends on whether vlans 200/220 need to communicate with vlans 100/120 and vice-versa. If they do then i would go the PBR route. If they don't i would look at VRF-lite.

Jon

davidjknapp · ‎02-17-2010

Thanks for the fast follow-up. This is exactly what I need. Do to the fact that there will be a third vlan (Call it VLAN300) that will be able to communicated to either network, VRF will not work. But great to know for other circumstatnces. I will investigate Policy Based Routing and attempt to impliment in a lab enviorment.

Jon Marshall · ‎02-17-2010

davidjknapp wrote:

Thanks for the fast follow-up. This is exactly what I need. Do to the fact that there will be a third vlan (Call it VLAN300) that will be able to communicated to either network, VRF will not work. But great to know for other circumstatnces. I will investigate Policy Based Routing and attempt to impliment in a lab enviorment.

David

No problem. glad to have helped.

Just for info you can actually "leak" routes between vrfs but i agree in your case PBR looks a better option.

Jon

davidjknapp · ‎02-22-2010

I have Attempted to setup a lab using two 3750's and IPs from 10.1.1.0/24 - 10.1.5.0/24 I have a client using 10.1.1.5 (port 1) and another at 10.1.2.5 (port 47). I want to route the 10.1.1.0 to the default gateway of 10.1.3.254 and the network 10.1.2.0 to the gateway 10.1.4.254. Both of those ip's are the second switch witch also has the netowork 10.1.5.0.

When I traceroute from either client I see the default gateway of the switch (10.1.3.254). Not sure where I am going wrong... I know that I have tried to enter the code "IP Policy Route-Map " at the vlan interfaces the traffic is comming from, and it does not show in the config. Relavent code is below. Any help would be awsome!! Thanks

!
interface GigabitEthernet1/0/1
description Network 1
switchport access vlan 100
switchport mode access
!
!
interface GigabitEthernet1/0/3
switchport access vlan 200

!
interface GigabitEthernet1/0/16
description Nework 3
switchport access vlan 300
switchport mode access
!
interface GigabitEthernet1/0/33
description Network 3
switchport access vlan 400
!
!
interface GigabitEthernet1/0/47
description Network 2
switchport access vlan 200
switchport mode access
!
!
interface Vlan100
description LAN1
ip address 10.1.1.1 255.255.255.0
!
interface Vlan200
description LAN3
ip address 10.1.2.1 255.255.255.0
!
interface Vlan300
description LAN3
ip address 10.1.3.1 255.255.255.0
!
interface Vlan400
description LAN4
ip address 10.1.4.1 255.255.255.0
!
!
ip route 0.0.0.0 0.0.0.0 10.1.3.254
!
!
access-list 10 permit 10.1.1.5
access-list 20 permit 10.1.2.5
!
route-map LAB permit 10
match ip address 10
set ip default next-hop 10.1.3.254
!
route-map LAB permit 20
match ip address 20
set ip default next-hop 10.1.4.254
!

Jon Marshall · ‎02-22-2010

davidjknapp wrote:

!
!
interface Vlan100
description LAN1
ip address 10.1.1.1 255.255.255.0
!
interface Vlan200
description LAN3
ip address 10.1.2.1 255.255.255.0
!
interface Vlan300
description LAN3
ip address 10.1.3.1 255.255.255.0
!
interface Vlan400
description LAN4
ip address 10.1.4.1 255.255.255.0
!
!
ip route 0.0.0.0 0.0.0.0 10.1.3.254
!
!
access-list 10 permit 10.1.1.5
access-list 20 permit 10.1.2.5
!
route-map LAB permit 10
match ip address 10
set ip default next-hop 10.1.3.254
!
route-map LAB permit 20
match ip address 20
set ip default next-hop 10.1.4.254
!

David

Couple of things -

1) "set ip default next-hop" means use the routing table first and if there is no specific match then use PBR. I suspect you want to use PBR before the routing table so change this to "set ip next-hop

2) you need to apply the policy maps to the L3 vlan interfaces so -

int vlan 100

ip policy route-map LAB

int vlan 200

ip policy route-map LAB

Jon

davidjknapp · ‎02-22-2010

I did see that the 'set ip defaul next-hop" runs as a post routing table solution - which is how I want it. If there is an internal route - I want it to use it, however it there is not internal route (Internet bound traffic) then go to this route. For the lab I did not using routing protocals to prevent the routes from getting transmitted...I just stuck to static routes.

I did entered the exact commands, and they did not appear to have any effect - including to the config.

int vlan 100

ip policy route-map LAB

int vlan 200

ip policy route-map LAB

I was using SSH so I may have missed an error if the switch did not accept the command. I found a Cisco article discussing the command :

SDM Prefer Extended-Match

Which I implemented but have not tested after the reload. I discovered this after I realized that "Policy Based Routing" was listed as "Disabled" when displaying "Show ip int Vlan 100"

Will be testing after re-entering the command in the Vlans Tomorrow.

Thanks again!!! I will post my results for others to see!

http://www.cisco.com/en/US/tech/tk364/technologies_configuration_example09186a00802135d3.shtml

Jon Marshall · ‎02-22-2010

davidjknapp wrote:

http://www.cisco.com/en/US/tech/tk364/technologies_configuration_example09186a00802135d3.shtml

David

Did you enable the routing template ie. "sdm prefer routing"

Jon

davidjknapp · ‎02-23-2010

I have got the lab working. What was needed was to enter "SDM prefer routing" then reload. After which I had to remove the "set ip default next-hop 10.1.4.254" and exchange it for "set ip next-hop 10.1.4.254" (the 3750 does not support default next-hop).

http://www.cisco.com/en/US/docs/switches/lan/catalyst3750e_3560e/software/release/12.2_37_se/configuration/guide/swuncli.html

I do have a question related to this. Our setup will consist of one 3750g-48ts, and then 2960G-24ts's as the access switches. We will have 5 vlans at each site. There will be two gateway routers: 1 going direct to the internet (linksys SOHO), and 2 going to datacenter via MPLS (1800 series). the datacenter has different internet feed. Lets call the network that uses local internet 100 and the network that goes through the MPLS 200.

With the configuration as it, the next-hop command will push all traffic to the local internet router from network 100. How do I route traffic I want to keep in the network (going to say antivirus server at datacenter). Wouldn't the next-hop command force all traffic to the internet gateway bypassing the routing table? If this is the case - What other options to I have? Route the traffic to a router and then to default next-hop there?

Jon Marshall · ‎02-23-2010

David

Glad you got it working.

As for directing traffic the key thing to be aware of is that only traffic matched in the acl will be sent to the next-hop specified in your route-map. And so it's all a question of configuring your acl properly so lets you wanted a host (192.168.5.1) to go across the MPLS network to the antivirus server but out the other link for http traffic

access-list 101 deny ip host 192.168.5.1 host

access-list 101 permit tcp host 192.168.5.1 any eq http

route-map PBR permit 10

match ip address 101

set ip next-hop x.x.x.x

because the first line is a deny it will not be sent the next-hop of x.x.x.x but routed via the normal routing table. The second line for http traffic is a permit and so will be sent to the next-hop ip of x.x.x.x.

So you just need to be specific in your acls that you use with your route-maps.

Jon

davidjknapp · ‎02-24-2010

Once again, Jon straitens me out!! - Cisco does not allow "Deny" ACL's on the 3750 Platform, however, I can do it the other way and tell the ACL to ignore anything on local lan by using : access-list 120 permit ip 10.1.2.0 0.0.0.255 10.0.0.0 0.255.255.255 - concidering our entire network sits in the 10.0.0.0 range and no internet traffic should be there.

Thanks Again!!

Jon Marshall · ‎02-24-2010

davidjknapp wrote:

Once again, Jon straitens me out!! - Cisco does not allow "Deny" ACL's on the 3750 Platform, however, I can do it the other way and tell the ACL to ignore anything on local lan by using : access-list 120 permit ip 10.1.2.0 0.0.0.255 10.0.0.0 0.255.255.255 - concidering our entire network sits in the 10.0.0.0 range and no internet traffic should be there.

Thanks Again!!

David

No problem, glad to be of help. It's often surprising which bits of the full PBR set switches don't support !

Jon

Gregory Flett · ‎11-18-2010

I have a similar question but a very different scenario.

We have a brand new site that uses full redundency on all connections and a connected by a port channel running at 20Gbps which is overkill

All outbound private address traffic travels via an MPLS router. The core switches are 4506s and the router is a 2800 series I believe. Right now the fa0/0 connection from the router is connected to one of the switches and we have an unconfigured fa0/1 interface that is supposed to connect to the other core switch to provide redundency and as a bonus load balancing.

The easiest way to accomplish this would be to load a routing protocol that supports load-balancing (ospf or eigrp) on to the router and use a couple of /30 networks to connect them to the switches - however the MPLS routers are managed by a team above mine and they refuse to load ANY routing protocols on the MPLS router.

Is it possible to set up a redundent connection, if even just for a failover, by any other means? PBR was close to the solution but I don't want to direct the traffic based on a policy, I just want it to have two routes to use.

Thanks!

Jon Marshall · ‎11-18-2010

Greg

Just to clarify.

You have a pair of 4500 switches that connect to a 2800 router that connects to MPLS. Only one of the links is active. ?

If so -

1) how are you telling that switchto send traffic to 2800 router ? - is it just a static route on the 4500 ?

2) what devices are connected to the 4500 switches ie. other switches or end devices.

3) Are the 4500 switches exchanging routes between themselves ie. if a device on the non 2800 connected switch wants to get to the MPLS clould how does the 4500 switch know it has to send the traffic to the other 4500 switch ?

4) How does the MPLS router know what networks are on the 4500 switch - do they have static routes.

5) If 4) is true then are the static routes only pointing to the connected 4500 switch ? If so would the MPLS group be happy to add statics for same networks pointing to other 4500 as well ?

I need answers to all of the above to understand how current routing works.

Note, as you mention, running a dynamic routing protocol such as OSPF/EIGRP between the 4500 switches and the 2800 router is by far the best solution and anything else is really just a bit of a bandaid. Did the MPLS router admin people actually give a good reason why they do not want to enable a routing protocol ?

Jon

Gregory Flett · ‎11-18-2010

Hi Jon - thanks for the reply..

Yes the pair of 4500s are connected to the 2800 and only one of the links is active.

The switches are set up in HSRP and MDF01 is the active the virtual IP is 10.68.32.1, the 4500s are physically .2 and .3, they have matching routing tables as shown below:

     10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
S       10.0.0.0/8 [1/0] via 10.68.32.4
C       10.68.46.0/24 is directly connected, Vlan219
C       10.68.45.0/24 is directly connected, Vlan119
C       10.68.36.0/24 is directly connected, Vlan117
C       10.68.34.0/24 is directly connected, Vlan115
C       10.68.32.0/24 is directly connected, Vlan110
C       10.68.33.0/24 is directly connected, Vlan111
S*   0.0.0.0/0 [1/0] via 10.68.32.6
CZX-MDF-4506-01-L3#

The first static link is to the MPLS router and the gateway of last resort is to a pair of ASAs running in active/standby and connected to the "external" internet link. The active ASA is connected to MDF01, the standby to MDF02.

As you can see from the routing table there are multiple VLANs configured on each 4500. Each switch has a single gigabit connection to each of the 6 IDFs in the facility and the IDFs are on VLAN 110 (infrastructure) with IP addresses matching their name (e.g. IDF4 is 10.68.32.14, and IDF 5 is 10.68.32.15). There is also a Cisco 4400 wireless controller hanging off of MDF01. VTP is configured on the MDFs and all the connections to the IDFs are trunks. The 4500s work at L3, the 3750s run at L2.

The following is the output of the MPLS router, I don't have access to anything other than the running config:

router bgp 65***
no synchronization
bgp log-neighbor-changes
network 10.68.32.0 mask 255.255.240.0
network 172.21.193.36 mask 255.255.255.252
network 199.***.***.*** mask 255.255.255.255
neighbor 172.21.193.37 remote-as 1803
neighbor 172.21.193.37 soft-reconfiguration inbound
no auto-summary
!
ip forward-protocol nd
ip route 10.68.32.0 255.255.240.0 10.68.32.1

I think that answers all of your questions but just in case let me do it inline anyway...

1) how are you telling that switchto send traffic to 2800 router ? - is it just a static route on the 4500 ?

Yes. All 10.x traffic is routed via the MPLS router at 10.68.32.4

2) what devices are connected to the 4500 switches ie. other switches or end devices.

There are a total of 8 3750s in the network. Both IDF0 and IDF1 have two stacked 3750s, the remaining IDFs have a single 3750. There are a few end devices plugged in to the MDFs, a WeatherGoose and a WLAN controller and a few other little things that shouldn't affect routing.

3) Are the 4500 switches exchanging routes between themselves ie. if a device on the non 2800 connected switch wants to get to the MPLS clould how does the 4500 switch know it has to send the traffic to the other 4500 switch ?

Each MDF switch was configured separately with matching static routes, no dynamic exchange is currently taking place but HSRP is solving that. We have been waiting until the correct configuration is put on the router before addressing a dynamic routing protocol on the 4500s.

4) How does the MPLS router know what networks are on the 4500 switch - do they have static routes.

The router only has a single static route to the 10.68.32.0/20 network with next hop being the virtual HSRP interface. Spittling the static route into two seperate routes matching half of the VLANs wouldn't really be a "solution".

5) If 4) is true then are the static routes only pointing to the connected 4500 switch ? If so would the MPLS group be happy to add statics for same networks pointing to other 4500 as well ?

Techincally I don't think 4 is valid as completely "true" but I'm hoping the team would be willing to put additional static routes on the MPLS router if necessary. Would two static routes on different networks on the 2800 and a dynamic protocol between the switches work? I'm trying to think about how that would work without creating a loop.

Finally...

Did the MPLS router admin people actually give a good reason why they do not want to enable a routing protocol ?

The response was simply:

11/18/10 15:24:55 Greenwich/Universal (steven.********):

sorry greg no OSPF on the MPLS routers.

How's that for a good reason? HA!

Load-balancing is just an extra, there really isn't much need for it and I'm not quite sure how that would work with HSRP anyway. The main goal is just redundency. The only technician on site is reasonably unfamiliar with cisco management and if either MDF goes down I want to ensure that they will not lose connectivity to the private network (although we could set up a VPN on their ASAs but being 16 hours time difference from Changzhou we'd prefer the switchover to be dynamic). I'm not going to ask you to paint the sistine chapel here, if the only solution is just ridiculous I understand, and if so I'll go try to put some sense in to the MPLS admins group.

Thanks for your advice!