Failover routing design help Needed

support systemsGo · ‎01-16-2014

Hello.

We are looking to have a setup like this:

User PCs

|

3750x

(stack - ip base)

/ \

Servers---------------3750x ---------------- 3750x---------------- Servers

(stack - (stack -

ip services) ip services)

| |

Router Router

| |

ISP1 ISP2

We would like to have routing (and vLans) done on the switches, and have internet failover from ISP1 to ISP2 if ISP1 fails, and go back to ISP1 when it comes back up. Trunks between all switches. We also would like to have all devices on the same vLAN if possible.

What is the best approach to do this?

(Note that left and right sides [in brown and green font] are in separate site locations, and that user end [in red font] switches only have ip base, which limits eigrp functionality.)

We tried following this, but doesn't fit our site exactly:

http://www.geekmungus.co.uk/cisco-and-networking/failoverinternetconnectionusingipslatrackingandeigrproutingforinter-sitelinks

(Also ran into issue where switch in the middle would have two routes to internet - so possible issue with priority routes)

Thanks in advance

Jon Marshall · ‎01-16-2014

It's not clear how the user end connects. Is it in a separate site again with connections to both sites or is it that you have users in each site ?

As a general answer, asssuming you are not receiving a defaullt route from the ISP you could run IP SLA on the 3750 switches (with IP Services) to monitor the ISP links. You would have two default routes on each switch but make the preferred route via ISP1 and then track that route and remove it if the link went down.

Alternatively you could use IP SLA on the routers and then redistribute a static route into EIGRP and have the switches get that but i'm not sure what this actually adds because you still have to run IP SLA anyway.

Could you answer the following questions -

So i'm assuming at the moment the 3750 stacks with IP services are L2 connected to each other ie. they don't route ?

Why do you want one vlan ?

Do the servers need L2 adjacency or not ?

How may users do you have ?

The main issue with having only one vlan (apart from broadcasts etc) is that any single device could affect all other devices including servers. But there may be a reason you want to do that and it may be worth trying to setup failover first before looking into creating vlans.

Finally i'm assuming that the connections from the routers to the 3750 IP services switches are in a different IP subnet (vlan). Is this the case ?

Jon

support systemsGo · ‎01-16-2014

Hi

The user end is a separate site again, with connections to both sites.

There will be about 35 users.

The 3750 stacks with IP services are connected by 2 fiber connections for redundancy.

Both stacks have EIGRP on them.

The servers need to be on the same vlan for failover purposes.

How it is setup now:

All switches have EIGRP on them.

Stack1 has a default route to ISP1 and Stack2 has default route to ISP2.

Both stacks have IP SLA tracking and route maps configured like in the example above.

(Except that currently all network devices, including routers, are all on same subnet.)

Problem is that user end stack switches with IP base, get two routes to internet with same priority.

So EIGRP on this stack has additional configuration to only accept one route for same destination.

When ISP1 fails, user end stack properly fails over route to ISP2.

But does not fail back over to ISP1 when it comes back.

Jon Marshall · ‎01-17-2014

All switches have EIGRP on them.

So the IP Base switches are running EIGRP stub are they ?

Problem is that user end stack switches with IP base, get two routes to internet with same priority.

So EIGRP on this stack has additional configuration to only accept one route for same destination

So what do you actually want to happen ie. all users on the switch stack use ISP1 unless it fails and then switch over to ISP2 ?

Where are the default routes coming from ie. are you receiving them from the ISP (i assume not as you are running IP SLA) ?

So are the routers configured with static default routes and you are redistributing into EIGRP ?

If you can answer all the questions above we can look at a possible solution.

Jon

support systemsGo · ‎01-17-2014

So the IP Base switches are running EIGRP stub are they ?

Yes, they are on IP Base.

So what do you actually want to happen ie. all users on the switch stack use ISP1 unless it fails and then switch over to ISP2 ?

Yes. This is what we would like to happen.

Where are the default routes coming from ie. are you receiving them from the ISP (i assume not as you are running IP SLA) ? So are the routers configured with static default routes and you are redistributing into EIGRP ?

Yes this is correct. Default routes are configured on the switches. There is a default 0.0.0.0 route to ISP1 on one switch stack, and another to ISP 2 on the other switch stack. IP Base switch stack are getting both routes. Routers only have route to ISP gateway.

Jon Marshall · ‎01-18-2014

Message was edited by: Jon Marshall

Jon Marshall · ‎01-18-2014

Apologies for removing last post, my brain isn't working very well today.

3750_1 = IP Services stack connected to ISP1

3750_2 = IP Services stack connected to ISP2

3750_3 = IP Base stack

I think the easiest solution is to apply delay but it needs to be carefully worked out because otherwise in a failover scenario traffic from 3750_3 could end up going via 3750_1.

What i am still not clear on is you say everything is one vlan which must mean that the uplinks from 3750_3 to both 3750_1 and 3750_2 must be L2. Your diagram is also showing a direct connection between 3750_1 and 3750_2 which again presumably is L2.

So one of these links must be blocking due to STP.

Can you clarify exactly how everything is interconnected because it's very diffcult to pick the right solution without knowing that. So -

1) when you say one vlan do you mean for the clients and servers and routers, the clients and servers only, or just clients

2) is there a direct connection between 3750_1 and 3750_2 and is it L2 or L3

3) if it is L2 and you have the same vlan for both clients and servers then the uplinks from 3750_3 must be L2 as well so STP must be blocking one of these links ?

4) are 3750_1 and 3750_2 peering with each other via EIGRP as well as with 3750_3

5) if the IP Services connections to there routers are in a diferent IP subnet where the routing done for the client (and possibly server) vlan.

I basically need to know what is L2 and what is L3 in terms of interconnects.

Jon

Jon Marshall · ‎01-19-2014

One other question as well -

if each 3750IPS is generating a default route into EIGRP how are you doing this ?

Jon

support systemsGo · ‎01-21-2014

Below are the configs for each site:

3750_1

SiteA#sh run

Building configuration...

Current configuration : 5071 bytes

!

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

no service password-encryption

!

hostname SiteA

!

boot-start-marker

boot-end-marker

!

no aaa new-model

switch 1 provision ws-c3750v2-48ps

system mtu routing 1500

!

track 1 ip sla 1 reachability

ip routing

!

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface FastEthernet1/0/1

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface FastEthernet1/0/2

switchport access vlan 10

switchport mode access

!

interface FastEthernet1/0/3

switchport access vlan 10

no keepalive

!

interface FastEthernet1/0/4

!

interface FastEthernet1/0/5

!

interface FastEthernet1/0/47

switchport trunk encapsulation dot1q

switchport mode trunk

channel-group 1 mode active

!

interface FastEthernet1/0/48

switchport trunk encapsulation dot1q

switchport mode trunk

channel-group 1 mode active

!

interface GigabitEthernet1/0/1

!

interface GigabitEthernet1/0/2

!

interface GigabitEthernet1/0/3

!

interface GigabitEthernet1/0/4

!

interface Vlan1

no ip address

!

interface Vlan10

ip address 10.10.20.1 255.255.255.0

!

interface Vlan50

description *** Site A - Site B ***

ip address 192.168.50.1 255.255.255.252

!

interface Vlan57

description *** Site A - User Site ***

ip address 192.168.50.41 255.255.255.248

!

router eigrp 1

network 10.0.0.0

network 192.168.50.0 0.0.0.3

network 192.168.50.40 0.0.0.7

redistribute static

!

ip local policy route-map IP_SLA_SiteA

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.20.254 track 1

ip http server

ip http secure-server

!

ip sla 1

icmp-echo 4.2.2.2 source-ip 10.10.20.1

timeout 2000

threshold 300

frequency 15

ip sla schedule 1 life forever start-time now

ip sla enable reaction-alerts

access-list 101 permit icmp host 10.10.20.1 host 4.2.2.2

route-map IP_SLA_SiteA permit 10

match ip address 101

set ip next-hop 10.10.20.254

!

SiteA#

3750_B

SiteB#sh run

Building configuration...

Current configuration : 5083 bytes

!

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

no service password-encryption

!

hostname SiteB

!

boot-start-marker

boot-end-marker

!

no aaa new-model

switch 1 provision ws-c3750v2-48ps

system mtu routing 1500

!

track 1 ip sla 1 reachability

ip routing

!

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface FastEthernet1/0/1

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface FastEthernet1/0/2

switchport access vlan 10

switchport mode access

!

interface FastEthernet1/0/3

!

interface FastEthernet1/0/4

!

interface FastEthernet1/0/47

switchport trunk encapsulation dot1q

switchport mode trunk

channel-group 1 mode active

!

interface FastEthernet1/0/48

switchport trunk encapsulation dot1q

switchport mode trunk

channel-group 1 mode active

!

interface GigabitEthernet1/0/1

!

interface GigabitEthernet1/0/2

!

interface GigabitEthernet1/0/3

!

interface GigabitEthernet1/0/4

!

interface Vlan1

no ip address

!

interface Vlan10

ip address 10.10.20.2 255.255.255.0

!

interface Vlan50

description *** Site B - Site A ***

no ip address

!

interface Vlan55

description *** Site B - User Site ***

ip address 192.168.50.33 255.255.255.248

delay 1000

!

router eigrp 1

network 10.0.0.0

network 192.168.50.0 0.0.0.3

network 192.168.50.32 0.0.0.7

redistribute static

distance eigrp 190 200

!

ip local policy route-map IP_SLA_SiteB

ip classless

ip route 0.0.0.0 0.0.0.0 10.10.20.253 track 1

ip http server

ip http secure-server

!

ip sla 1

icmp-echo 8.8.8.8 source-ip 10.10.20.2

timeout 2000

threshold 300

frequency 15

ip sla schedule 1 life forever start-time now

ip sla enable reaction-alerts

access-list 101 permit icmp host 10.10.20.2 host 8.8.8.8

route-map IP_SLA_SiteB permit 10

match ip address 101

set ip next-hop 10.10.20.253

SiteB#

3750_3

UserSite#sh run

Building configuration...

Current configuration : 4635 bytes

!

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

no service password-encryption

!

hostname UserSite

!

boot-start-marker

boot-end-marker

!

no aaa new-model

switch 1 provision ws-c3750v2-48ps

system mtu routing 1500

ip routing

!

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface FastEthernet1/0/1

switchport access vlan 10

switchport mode access

!

interface FastEthernet1/0/47

switchport trunk encapsulation dot1q

switchport mode trunk

channel-group 1 mode active

!

interface FastEthernet1/0/48

switchport trunk encapsulation dot1q

switchport mode trunk

delay 350

channel-group 1 mode active

!

interface GigabitEthernet1/0/1

!

interface GigabitEthernet1/0/2

!

interface GigabitEthernet1/0/3

!

interface GigabitEthernet1/0/4

!

interface Vlan1

no ip address

!

interface Vlan10

ip address 10.10.20.3 255.255.255.0

!

interface Vlan55

description *** UserSite - Site B ***

no ip address

!

interface Vlan57

description *** UserSite - Site A ***

no ip address

!

router eigrp 1

maximum-paths 1

network 10.0.0.0

network 192.168.50.32 0.0.0.7

network 192.168.50.40 0.0.0.7

redistribute static

offset-list 99 out 300 FastEthernet1/0/48

!

ip classless

ip http server

ip http secure-server

!

ip sla enable reaction-alerts

access-list 99 permit 10.10.20.0 0.0.0.255

!

UserSite#

Jon Marshall · ‎01-22-2014

3750_3 is the IP Base switch ?

If so you only have one port channel interface but how does that work ie. you have two separate switch stacks (3750_1 and 3750_2).

I know you can spread an etherchannel across member of the same stack but i didn't think you could spread an etherchannel across different stacks ?

i would have expected to see two port channels on the 3750_3.

Can you clarify ?

In addition are the IP Services switch stacks interconnected via single trunk link ie. not an etherchannel ?

Sorry for all the questions but it's not clear how this is setup.

Jon

support systemsGo · ‎01-22-2014

This is how the previous engineer (who left suddenly) set it up .

Ports 47 and 48 of 3750_1 are physically connected to Ports 47 and 48 of 3750_2 respectively.

(I believe they are trunk links)

Port 1 of 3750_1 is connected to Port 47 of 3750_3. Port 1 of 3750_2 is connected to Port 48 of 3750_3.

Not sure why etherchannel is configured in this way.

If you think it should be removed or reconfigured in another way, please let me know what you think.

Jon Marshall · ‎01-22-2014

Not sure it needs changing, but the confusion is that on the 3750_3 interfaces fa1/0/47 and fa1/0/48 are configured in an etterchannel but from your description they should be separate links ie. one link each goes to a different IP Services stack.

Can you post "sh etherchannel summary" from 3750_3 because i don't think it is actually acting as an etherchannel.

Jon

support systemsGo · ‎01-22-2014

Ok. Please wait a few hours.

I will need to go onsite to do this command.

In the mean time, can you see anything else wrong with the eigrp routes?

Jon Marshall · ‎01-22-2014

I'm just going through them.

The ideal setup would be to have 3750_1 advertise a default route and only have 3750_2 advertise a default route if the ISP1 link went down. You can do this by -

1) 3750_1 redistributes it's static route into EIGRP and 3750_2 and 3750_3 receive it as AD 170

2) on 3750_2 you configure the default route with an AD > 170. No need to track the route. Because the AD is > 170 3750_2 uses the route learnt from 3750_1 and so does not advertise it's own route ie. it's own default route is not installed in the routing table and so it is not redistributed into EIGRP.

3) if the ISP1 link goes down then 3750_1 removes the default route (because of IP SLA) and so no longer redistributes it into EIGRP

4) 3750_2 now installs it's own default route and advertises this to 3750_1 and 3750_3 so all traffic goes via 3750_2

5) if the link to ISP1 comes back up 3750_1 reinstalls it's route and we are back to step 1)

The problem is your interconnect etherchannel between 3750_1 and 3750_2. If that failed then with the above 3750_2 no longer gets the default route via EIGRP. It could go via 3750_1 -> 3750_3 to 3750_2 and then it would but if 3750_3 is an EIGRP stub it won't pass the route on to 3750_2.

So 3750_2 loses the route even though ISP1 is still up and advertises the default route. That is the only failure scenario where the above solution wouldn't work.

So i was thinking of using delays on interfaces to try and account for all failure scenarios. But i need to understand the setup and the configuration as it is.

That etherchannel connection is very confusing at the moment. In addition if you are going on site can you make sure 3750_3 is actually running IP Base because there is no mention of the EIGRP stub feature in any of the configurations on all of the 3750 switches.

Jon

Jon Marshall · ‎01-22-2014

The more i look at the configurations the less i understand

1) on 3750_1 you have -

interface Vlan10

ip address 10.10.20.1 255.255.255.0

!

interface Vlan50

description *** Site A - Site B ***

ip address 192.168.50.1 255.255.255.252

!

interface Vlan57

description *** Site A - User Site ***

ip address 192.168.50.41 255.255.255.248

but on 3750_2(B) you only have -

interface Vlan50

description *** Site B - Site A ***

no ip address

!

interface Vlan55

description *** Site B - User Site ***

ip address 192.168.50.33 255.255.255.248

delay 1000

ie. vlan 50 does not have an IP address

and on 3750_3 you have -

interface Vlan55

description *** UserSite - Site B ***

no ip address

!

interface Vlan57

description *** UserSite - Site A ***

no ip address

ie. no ip addresses configured at all

2) there are delays and offsets everywhere on 3750_3 and obviously none of them are working properly

3) you say there is only one vlan for clients and servers, presumably 10.10.20.0/24 - is that right ?

4) IP SLA is using local PBR and this would only be needed if there was no "permanent" option when adding a static route

5) each switch has a vlan interface for vlan 10 but if that is the same subnet used for clients and servers on all sites then 3750_3 certainly doesn't need an interface for it because it will never be routed.

etc.. Basically there is, as far as i can see far too much configuration for what you need. So can you answer these questions -

1) is 10.10.20.0/24 the subnet used for clients and servers ?

2) if it is is there any reason why you want that one subnet across all sites. You can if you want i just need to know the reasoning eg there may be servers at each 3750_1/2 site that need L2 adjacency to work properly

3) how is that etherchannel configured on 3750_3 ie. if the physical links are going to two separate switch stacks then it cannot be an etherchannel.

4) what feature set is on 3750_3

5) if you try this command "sh ip route 0.0.0.0 0.0.0.0 ?" is there a "permanent" option keyword

6) what are the default gateways set to on the clients/servers. Is it the same or does it differ per site. I suspect if vlan 10 is the client/server vlan then each site uses a different default gateway.

I don't like to crticise any setup because you never know the full picture and the previous engineer may have had good reason to set it up like this but it makes no sense to me whatsoever. Unless there is something else happening that you have not covered this could do with a whole redesign.

If possible can you have a full read of all i have written and answer the specific questions because otherwise it's going to be very difficult to help.

Jon