Layer 3 design, VRF-lite, ospf and mstp.

[runeaagesen]

Dear Community.

We have a network basically configured with Catalyst 3560G/E/X, full feature, that is vrf-lite, ospf.

The "problem" occured when I had the first network-ring configured, and one port was blocked by spanning-tree in this ring-structure.

All 3560s are configured in the same mstp-instance 0, and my thoughts was that the redundancy/ring-structure was handled by ospf and not stp.

There is no vlan throughout the hole network, since I have "routing-vlan" between each 3560, please take a look at the attched example.

In this ring-excample, one port will be blocked by stp. I was hopeing that this was handled by ospf instead.

My question is:

Does anyone have a best practice how to configure my network? What is there I have done wrong!

If each 3560 is configured in different instances, would this solve what I trying to achieve?

If I add "spanning-tree bpduguard/bpdufilter enable" on each port between each 3560 solve this, all configured with instance 0.

Best regards

Rune Aagesen

[Giuseppe Larosa]

Hello Rune,

>>

If I add "spanning-tree bpduguard/bpdufilter enable" on each port between each 3560 solve this, all configured with instance 0.

please don't do this or you will create a terrible bridging loop ! This is not an option at all.

According to the network diagram the L2 trunks between the different C3560 carry different subset of vlans. So one option is to take advantage of MST by introducing additional MST instances. Each additional MST instance will handle a set of vlans and will have a different root bridge ( = one of the endpoint switch of a specific trunk link).

All switches have to agree on MST region name, MST revision number and Vlans to MST instances mapping in order to avoid partitioning of the MST region.

Until inter switch links are L2 trunks they will be handled by MST and not by OSPF. OSPF would be in use only if you would change the links in routed L3 links in all other cases STP plays a role.

Hope to help

Giuseppe

[runeaagesen]

Hi Giuseppe.

Thanks a lot for your time.

First, you don't suggest to use bpdufilter/bpduguard between the 3560s, because this would create bridging loop.

My thought was since there are different vlans between each 3560, there will not be a loop, since this is not a L2 topology,

or am I missing something now?

This design was suggested by a CCIE-guy some years ago, and I was told my network is a routed L3 network.

Therefor I don't understand what you write:

"Until inter switch links are L2 trunks they will be handled by MST and  not by OSPF. OSPF would be in use only if you would change the links in  routed L3 links in all other cases STP plays a role."

Are my trunks between my 3560s not routed links? Any suggestion how this should have been configured?

Because he told me that now OSPF will take care of the bridging loop, since I now have a L3 network!

Best regards

Rune

[Giuseppe Larosa]

Hello Rune,

your network is not a pure L3 network and the confirmation of this fact is given by MST activity that blocked one link because all Vlans are associated to MST instance 0 => single topology => one link is blocked.

The issue with using bpdu filter is that if in the future someone makes a configuration error allowing other vlans on the inter switch links the loop can form. I would not use it.

Another option for your design would be moving to spanning-tree mode PVST+ that has one instance for each Vlan. this should avoid the link in blocking state as set of permitted vlans are different on each inter-switch link.

Rapid PVST would be better for convergence.

Or you can keep MST but you need to change vlans mappings introducing additional MST instances as described in my previous post.

Finally if the vlan sets permitted on each interswitch link are not used by clients downstream ( I guess they are) you can convert each link to a routed port with a single IP subnet on it. This would be a true L3 topology.

Hope to help

Giuseppe

[runeaagesen]

Once again, thanks a lot for your time.

I have to say I'm a bit disappointed of the CCIE-guy suggesting this "L3" design!

First, the reason for choosing MSTP, is that we still have som old L2-switches in our network, running stp/rstp.

Excample, in site C, we have two L2-switches, creating a local loop. If I run PVST I'm not able to do this.

The reason for using VRF-lite, was that we wanted to have several different networks isolated, so to start with, vrf y cannot comunicate with anything in vrf z. The original routing process, is our management network. To allow interaction between vrfs, we are using a asa-5510, as a local acl.

As we was explained, regarding this design, each vrf was a fully functioning network (virtual though).

........ you can convert each link to a routed port with a single IP subnet on it. This would be a true L3 topology.

This is interesting, because, as a mentioned, we believed we had a true L3 topology, regarding CCIE-guy.

Do you have any suggestion in how to solve this, if we use the attached excample?

Best regards

Rune

[Giuseppe Larosa]

Hello Rune,

you cannot use routed links or you would need one routed link for each VRF between each pair of switches. This is the reason for using L2 trunks my guess is that you have 3 VRFs defined. (VRF lite).

I'm sorry I have given a misleading suggestion I had forgot it is a VRF lite scenario.

At Layer3 each VRF has its own topology, but now at L2 only one topology exists.

At this point I would suggest to tune MST in order to build different topologies as you have just said that you need MST for interoperability.

However, the current network scenario works with the only difference that it is MST that manages the network redundancy and not OSPF. I mean the network connectivity is good someone could also end with  "do not touch it" suggestion.

The suggested MST changes would provide an improvement.on how links are used.

Hope to help

Giuseppe

[runeaagesen]

Hi Giuseppe, I really appreciate taking time to give me a helping hand.

So, what you are saying, is (I'll try my very best to understand):

Each VRF has it's own routing instance and topology, at layer 3. So far I follow.

So in my excample I have the originally routing instance/process and two configured VRFs, y and z.

Each VRF need a routing-vlan between each switch.

..... but now at L2 only one topology exists.

I'm having a bit difficult to understand this, since I have a L3 routed network.

If I in my excample also configured, let say, vlan 100 between each switch, then I understand I would have a L2 topology.

But, could the reason be that since I have MSTP instance 0 configured on all five switches, MSTP see this as a L2 network regardless of the VLANs mapped to the instance? I was reading something in an article from cisco.com now:

http://www.cisco.com/en/US/tech/tk389/tk621/technologies_white_paper09186a0080094cfc.shtml

To put it in another way, this is just how MSTP works, if I have understood this correctly.

All topology in the same MSTP instance, is in real a L2 topology, regardless of vlan or routed network!

So, if I understand what you suggest, to have OSPF to manage the network redundancy (not MSTP), I need to (in my excample) configure each switch, A, B, C, D and E in each MSTP instance 0-4.

On switchA all VLANs are mapped to instance 0, in both directions.

On switchB all VLANs are mapped to instance 1, in both directions, and so on.

As far as I have read, I can configure up to 64 MSTP instances. What if (or when) my network consists of more than 64 L3 switches, can I start all over with instance 1, but "far away" from my first instance 1!

I'm sorry to bother you with this, but I've asked around some time, but have not got any solution, until now, if I've understood this correctly so far.

This was interesting, because it seem to me that you know your things.

I've tried to ask the CCIE-guy I mentioned, but could not give me any explanation.

Best regards

Rune

[Giuseppe Larosa]

Hello Rune,

yes you have a single L2 topology because all Vlans are mapped to MST instance 0. From the point of view of MST it sees a ring so it reacts by blocking one link. It doesnn't look at the list of vlans permitted on each trunk link.

My suggestion is that given the Vlan space 1-4094 currently mapped all to MST instance 0 you should divide the space in at least 5 subsets in the SAME WAY on all switches or you will get MST region partitioned

>>

On switchA all VLANs are mapped to instance 0, in both directions.

On switchB all VLANs are mapped to instance 1, in both directions, and so on.

this is not correct as I have explained above it would lead to each switch considering itself in a separate MST region and different MST regions interact as they were old single STP instance switches.

Example:

vlans 10.20,30  associated to instance 1

vlans 11,21,31 associated to instance 2

vlans 12,22,32 associated to instance 3

vlans 13,23,33 associated to instance 4

on all switches because MST BPDU messages  contains an hash of the vlans to instances mapping so they can easily detect mismatches on mappings.

I would suggest to read carefully the document you have found to get the necessary knowledge of MST

For the moment your network is working the way we have discussed up to now.

Hope to help

Giuseppe

[runeaagesen]

Hi again Giuseppe

Once again, thanks a lot.

I agree, I need to do some reading to fully understand this "problem". I'm not there yet, but hopefully I will be.

I do agree that my network is working, but so far, MSTP will manage the redundancy.

But so far, regarding what you have written, my network is not configured as it should be.

You see, we was recommended to "divide" the hole network into four "L2-groups" (I believe to fully understand, you need to see the hole network). This is done by bpdufiler/guard on links between these L2-groups. Never fully understood why we was recommended this, but I'll try to look into this.

Thanks a lot so far.

Best regards

Rune