GLBP and STP

[ALIAOF_]

Can some one tell me what would be the best practice when implementing GLBP and Spanning Tree.  I have two switches in GLBP cofiguration, so can I make one switch the root for all the VLANs and the secondary switch which will be the forwarder secondary ? 

Or do I need to have the same priority on both switches ?

I'm posting the related Spanning Tree configuration

SWITCH 1:

interface   vlan 200

ip address 10.1.200.249 255.255.252.0

glbp 200 ip 10.1.200.1

glbp 200 priority 110

glbp 200 preempt

glbp 200 weighting 5

glbp 200 load-balancing weighted

glbp 200 authentication md5 key-string XXXXXXXXX

glbp 200 weighting track 200 decrement 20

!

spanning-tree vlan 200 priority 0

SWITCH 2:

interface Vlan200

ip   address 10.1.200.250 255.255.252.0

ip   helper-address 192.168.0.2

glbp 200 ip 10.1.200.1

glbp 200 preempt

glbp 200 weighting 6

glbp 200 load-balancing weighted

glbp 200 authentication md5 key-string XXXXXXXXXXX

glbp 200 weighting track 200 decrement 20

!

spanning-tree vlan   1,5,9,48,200,250 priority 4096

[Peter Paluch]

Hello Mohammad,

What is actually your network topology with regard to distribution and access layer switches? The current best practice is to physically connect each access switch to both distribution switches, and have the distribution switches interconnect via a direct Layer3 (i.e. no switchport) link with an appropriate routing protocol over it. Physically a triangle, this topology has no Layer2 loops because the distribution switches are connected by a L3 link, so there is no STP blocking port here. The placement of the root switch becomes less important in this topology - it may be any of the distribution layer switches. GLBP in this topology runs optimally as the frames towards a particular GLBP AVF are carried by the direct link from the access switch to the appropriate distribution layer switches.

Best regards,

Peter

[ALIAOF_]

Thank you for the reply Peter. Currently unfortunately I do not have any routing protocol running that is the phase 2 of this project as many changes are required just to get this going for now I didn't want to work on enabling a routing protocol just yet. 

I am attaching the diagram that I have.  Basically the two switches that will be configured with glbp will be the main core switches.  So in the mean time will the following steps be ok?

1- Configure SWITCH 1 as the root (Shown above)

2- Configure SWITCH 2 as the secondary root (Shown above)

3- Lower the priority or path cost on the link that is connecting Switch 1 and Switch 2

For now I will have only one Access Layer switch 10.1.200.102 connected to both of the switches, Eventually I will be connecting 10.1.200.101 to both of the switches as well via port channels.  There will be one more switch that I will add later that will also connect to both of the switches.

[Jon Marshall]

Mohammed

You don't say whether the interconnect between your 2 core/distro switches ie. the 4948s in your diagram is a L2 trunk or a L3 link. Assuming it is a L2 trunk the point Peter was making is that if you have an access-layer switch connected via L2 trunks to 2 distribution switches and the distribution switches are interconnected via a L2 trunk then GLBP doesn't really work the way you want it to for the following reason -

access-layer switch = as1

dist switch 1 = ds1

dist switch 2 = ds2

lets say you have vlan 10 on as1 and the L3 SVIs for vlan 10 are on ds1 and ds2.

Because ds1 and ds2 are connected via a L2 trunk there is a L2 loop ie. as1 -> ds1 -> ds2 -> as1. So STP blocks for example as1 -> ds2. Now the only path from as1 to the dist switches is as1 -> ds1. With GLBP either dist switch could be used as the forwarder but if ds2 is used as1 cannot take a direct path, it has to to go as1 -> ds1 -> ds2. So the interconnect between ds1 and ds2 gets used a lot more with GLBP in this design than it would with say HSRP where you only get one forwarder ie. one dist switch doing the forwarding.

So if ds1 and ds2 are interconnected via a L2 trunk be aware that using GLBP could mean either ds1 or ds2 are chosen for a particular client to forward traffic. If it's ds2 then traffic still has to go via as1 -> ds1 to get to ds2.

If you use a L3 interconnect between ds1 and ds2 then as Peter says, there is no L2 loop so both links from as1 will be forwarding. However there is a caveat with this in that you cannot use this design if you have the same vlan on multiple access-layer switches ie. in our example vlan 10 could only be on as1. If you had an as2 switch it could not use vlan 10, it would have to have it's own vlan. If you want the same vlan on multiple access-layer switches you should not use a L3 link between your distribution switches.

So it's important to understand that GLBP will only run over the network after STP has done it's job of blocking any redundant links and what that means in terms of traffic flow from your access-layer switch to the distribution switches. If you want to fully utilise GLBP then as discussed you would be better using a L3 interconnect between your distro switches but if you decided to do this then you need to be aware of your existing setup and how doing that would affect what you already have.

If the existing interconnect is L3 then it should work fine.

Jon

[ALIAOF_]

Thank you Jon for the reply, yes initially I am planning on having the interconnect between the two switches as a L2 link.  I was thinking about utilizing the "spanning-tree cost xxxx" command to reduce the cost on the L2 link between the two switches so that link does not get utilized.  Here is the related document:

http://www.cisco.com/en/US/docs/solutions/Enterprise/Campus/HA_campus_DG/hacampusdg.html#wp1108489

Later on I am planning on implementing a dynamic routing protocol and connecting via L3 link.  However in that case you mention if I have the same VLAN on multiple switches I can not use that.  Can you educate me a bit on that I don't understand completely.  Here is how the VLAN's are setup (I will use your example)

DS1 & DS2 = VLAN1, VLAN4, VLAN5, VLAN9, VLAN150, VLAN200, VLAN250 (SVI's)

                     VLAN7, VLAN8, VLAN128, VLAN132, VLAN136 (In VLAN database)

AS1 = VLAN 200 (SVI) rest of the VLAN's are in the VLAN database only

So since I have VLAN 200 on DS1, DS2 & AS1 setup as an interface with an IP address, L3 interconnect between DS1 & DS2 can not be used?

[Jon Marshall]

Mohammed

Okay, so you are going to block the interconnect link so that bith uplinks are forwarding from the access-layer switch which sounds perfectly reasonable.

As for the L3 interconnect i meant vlans specific to the access-layer switch and not between the access-layer switch and the distribution switches. So you would be fine with for example vlan 10 on ds1/ds2 and as1 and vlan 11 on ds1/ds2 and as2 but not vlan 10 on both access-layer switches ie. vlan 10 on ds1/ds2/as1/as2 and the same for vlan 11.

However i'm not sure that it would be a problem. I need to sit down and draw up a few diagrams because without going into all the details i was assuming it wouldn't work due to the distribution switches not failing over properly but i'm not sure i'm right about that. 

Apologies for what i think is misleading information.

Jon

[ALIAOF_]

Thank you for the reply again Jon.  Ok so with the L2 it all makes sense.  With the L3 please accept my apologies but I am getting there slowly and I am very thankful for your input and sharing your knowledge.  So with L3

DS1 & DS2 = VLAN10 and VLAN11 (SVI's)  - VLAN12  & VLAN13 (Just in the database not actual interfaces)

DS1 VLAN10 IP: 10.1.0.10 (VIP: 10.1.0.1)

DS2 VLAN10 IP: 10.1.0.20 (VIP: 10.1.0.1)

example:

interface vlan 10 (DS1)

ip address 10.1.0.10 255.255.25.0

!

*******************************************

interface vlan 10 (DS2)

ip address 10.1.0.20 255.255.255.0

!

interface vlan 11 (DS1)

ip address 10.1.1.10 255.255.25.0

!

*******************************************

interface vlan 11 (DS2)

ip address 10.1.1.20 255.255.255.0

!

vlan 12 (DS1 & DS2)

!

vlan 13 (DS1 & DS2)

AS1 & AS2 = VLAN10 (SVI using VLAN 10 as the management) - VLAN11, VLAN12 & VLAN13 (Just in the database)

AS1 VLAN10 IP: 10.1.0.101 (Default Gateway: 10.1.0.1)

AS2 VLAN10 IP: 10.1.0.102 (Default Gateway: 10.1.0.1)

interface vlan 10 (AS1)

ip address 10.1.0.101 255.255.255.0

!

************************************************

interface vlan 10 (AS2)

ip address 10.1.0.102 255.255.255.0

!

vlan 11 (AS1 & AS2)

!

vlan 12 (AS1 & AS2)

!

vlan 13 (AS1 & AS2)

[Jon Marshall]

Mohammed

The above is not a L3 config.  I think there is also a typo ie. on ds2 you have configured vlan 11 with an IP from vlan 10 subnet - did you mean int vlan 10 rather than int vlan 11.

Basically with L2 setup you would have -

1) as1 and as2 are fine with their config as long as vlan 10 is the management subnet for the switches

2) on ds1 and ds2 you would have L3 SVIs for vlan 10 and vlan 11 with GLBP config under the SVI config

3) you would also have a L2 trunk (probably an etherchannel trunk) between ds1 and ds2

with L3 you would have -

1) as1 and as2 as above

2) same as above ie. L3 SVI for vlan 10 and vlan 11 with GLBP config

3) instead of a L2 etherchannel ? trunk you would configure a L3 etherchannel between the 2 switches

Jon

[ALIAOF_]

Oops my bad sorry about the typo I fixed it yes you are correct it was supposed to be VLAN11.  Ok so with the L3 config above setup with L3 ether channel works.  And the fact that I have VLAN 11, 12 & 13 on AS1 and AS2 shouldn't be an issue with the L3 interconnect .... Am I on the right track with that statement?

[Jon Marshall]

Mohammed

You have made me think a bit about this

Figure 41 in the link you gave doesn't seem right to me ie. there seems to be a L2 loop formed between the access-layer switches. And this i think is what is nagging me about the design. I am going to post a question of my own linking to that doc and get some opinions. I suspect i may be missing somethiing but if somebody else could point it out it might jog my memory as to why i thought vlans had to be specific to each access-layer switch.

Jon

[ALIAOF_]

With figure 41 if the cost between the two interconnects is lowered and that is blocked then shouldn't the two switches forward out of both links "F2" in the picture and loop will be stopped the blocked port on one end of the L2 link between the two Distribution switches?

[Jon Marshall · Accepted solution]

Mohammed

Okay i posted a quick question and Giuseppe answered confirming my doubts. Have a look at this thread for details but basically figure 41 in the design doc is wrong -

https://supportforums.cisco.com/thread/2173126?tstart=0

another of the links would need to be blocked. So lets say it is (using the diagram) Access B to dist B. This means for Access B to get to dist B it has to go - Access B -> dist A -> Access A -> dist B  which is not an optimal path at all. Because remember dist B could still be a forwarder for clients connected to Access B even though the path is via the A switches.

Note also that as discussed in the other thread if each access switch had it's own vlan then you wouldn't face this problem because one access-layer switch could  not use the other as a transit switch. And each access-layer switch could be forwarding on both it's links as you have blocked the L2 interconnect.

A similiar issue occurs with a L3 interconnect. You would still need to block one of the access-layer switches uplinks to avoid a L2 loop between the access-layer switches.

This is the problem with having the same vlan on multiple access-layer switches using GLBP. It's not to say traffic wouldn't flow but some of  it would flow via other access-layer switches and this is not a good design. Allowing GLBP messages via the access-layer switches is fine but you don't want data traffic from one access-layer switch to have to go via another access-layer switch, it's just not going to work properly.

This is why it is recommended if you are using a L3 interconnect you should separate vlans on each access-layer switch. And also with the figure 41 design from the doc.

If you must have the same vlans on multiple access-layer switches then -

1) do not block the dist switch interconnect. By blocking it (as in fig 41) you are forcing some data traffic to go via the other access-layer switch. If you unblock the interconnect then at least the traffic can take the shorter path to get to the other dist switch. *** Edit - also the interconnect should be L2 and not L3.

2) if you use GLBP accept the interconnect will be used a lot more than HSRP because both dist switches could be forwarding

3) You could consider simply using HSRP instead which would mean the interconnect is used less to get to the client gateway but all forwarding is done by one dist switch, although this can be done on a per vlan basis.

Hope the above makes sense.

Jon

[ALIAOF_]

Jon thanks so  much again for a great answer.  That makes sense and I understand that blocking the interconnect can start sending some traffic via the access switch which will degrade performance a whole lot more. 

Considering diagram 41 and as you mentioned blocking two links, what if..

1- I reduce the spanning-tree cost on the L2 interconnect

2- Reduce the spanning-tree cost on the link from AccessB to DistributionA

3- Reduce the spanning-tree cost on the link from AccessA to DistributionA

4- Reason = since Distribution A will be the AVG and Distribution will be the AVF

5- If something happens to the link between AccessB to DistributionB or AccessA to DistributionB path cost on those links should decrease and the other alternate links should start forwarding.

Note: Kind of like Figure 40 but block an additional link i.e between the two distribution switches. 

**EDIT: I just realized if I do that then both links won't be active at the same time to the distribution, but it will atleast provide redundancy until I can separate out the VLAN's on the switches**

And the reason for all this is because in my design I will have the VLAN's setup the same way as in Figure 41 i.e; VLAN10, 11, 12, 13 on all the access layer switches.  So I'll need to block (reduce path cost) on multiple links....

[Jon Marshall · Accepted solution]

Mohammed

Firstly if you are using the same vlans on all the access-layer switches then i would suggest not using a L3 link between your distribution switches.

Secondly the basic principle is to keep it simple and STP generally works without all this manipulation of path costs. In fact the only link you want to ensure does not block is the ds1 -> ds2 interconnect otherwise you do get data traversing access-laye switches. If you want to use GLBP then i would suggest making sure your L2 interconnect between ds1 and ds2 is an etherchannel trunk and if it is and the uplinks from your access-layer switches are not etherchannel then STP will automatically block some of the access-layer uplinks and not the ds1 -> ds2 interconnect.

What you want is to set ds1 and ds2 as STP root/secondary. Ensure the L2 interconnect ds1 -> ds1 is not blocked and then let STP decide which uplinks from access-layer switches it blocks. No need to overcomplicate it by manipulating all the port costs on individual links.

If you use GLBP then you will use both ds1 and ds2 to forward data from clients (with the proviso described earlier that the interconnect will be quite heavily utlised). Or you could use HSRP and manually setup up load-balancing from the access-layer switches using ds1 root for odd vlans and ds2 root for even vlans which may be the better choice because you can load-balance in a more deterministic way.

Is there a specific reason you need the same vlans on all access-layer switches ? As you can see from this discussion that is the main limiting factor.

Jon

[ALIAOF_]

Thank you Jon, I would absolutely LOOOOOVE to separate the VLAN's out as it is more clean and organized however I acquired this network when I started this job and I'm slowly working on moving things piece by piece to best practice and optimized topology.  Since I have to make all the changes and I only have very limited allowed down time I wanted to at least get the GLBP working in the first phase and update the spanning tree configuration on all the switches DC and Corp as that is different all over the place too, VTP settings update as well on all the switches. 

And currently the way servers are connected they are spread all over the switches so separating VLAN's would require even more down time.  I wanted to get on that as phase 2 and then Phase 3 to introduce EIGRP and connect the links via L3 interconnect

But like you mentioned I definitely want to keep it simple and not over complicate it, so what you suggested absolutely makes sense.  Thank you again my phase 1 changes are going in on Friday night I'll come back and update here with the results , wish me luck.