Spanning tree help, using firewall software switch for connections

[PHarrisonCWSI]

Hi guys,

I'm new to a lot of these concepts so maybe I'm missing something obvious, but some input would be really appreciated!

Basically I'm setting up a Fortigate firewall HA pair (active/passive) and four 2960S switches in our datacentre.  There will be one Fortigate firewall and two switches per cabinet, with the cabinets linked by a ethernet connections.  The idea being that in say cabinet A a host could have two NICs with one connected to each switch, so if a switch fails in a cabinet it's no problem, and if a firewall fails in either cabinet it's no problem.

I have it all in-place and it's working fine but I have a bit of a concern with how spanning-tree has set the links.  I had envisaged lots of connections between all the switches and traffic could take the shortest route, but of course this routing not switching which became obvious when I started looking into STP .

I've attached a diagram of how STP has enabled/disabled links.  The Fortigate is configured with a four port software switch that the links from the four 2960s connect to.  The problem in the attached diagram is that if servers on different switches want to talk to each other the traffic will be sent down the link to the Fortigate then back down another link to the relevant switch.  It seems like a waste to take this route when there are (currently blocked) links between the switches themselves.  The main thing I'm concerned about is the load it's going to put on the Fortgate if it has to software-switch all possible traffic in the network.  Now of course the good thing here is if a switch fails STP should bring up another link to the Fortigate, and in reality most of the traffic on the network will be from servers to the Fortigate (not too much inter-server traffic) but that could change.

Is there a way to fix this or should I not be concerned in the first place?  It seems like if I could stop the Fortigate acting as a switch then STP woudl be enable all the inter-switch uplinks without creating loops and hence have more efficient paths between switches.

Thanks for any help!

[Aninda Chatterjee]

Hi Philip,

I have never worked with a Fortigate firewall before so pardon my lack of knowledge in that regard. It's interesting how you state that the firewall is configured with a four port 'software switch'. The way your topology has converged, this does seem to imply that the firewall is actively taking part in spanning-tree and more importantly, it seems to have become the root bridge for your layer 2 switched network.

If you were to go to your 2960s and do a show spanning-tree vlan , what do you see as the spanning-tree state for the interface that goes to your firewall? Is it listed as a root port? If you do the same command on the active firewall, does it state that it is the root bridge anywhere?

Coming to your servers.. Is it a requirement that all your server to server traffic pass through the firewall? I would recommend that one of the 2960s be configured as your root bridge. You could do this by lowering the priority of the switch by using the spanning-tree vlan 1-4094 priority to something lower than what the firewall has configured - remember, in spanning-tree, lower is better. This should get your switch to switch links up and forwarding from a spanning-tree perspective since the cost to the root bridge would be best via the direct link rather than going through the firewall and then to the root bridge.

The above change is, of course, considering that you are unable to stop the firewall from taking part in spanning-tree.

Regards,

Aninda

Message was edited by: Aninda Chatterjee

[PHarrisonCWSI]

Thanks for the reply Aninda,

Here's an excerpt from the Fortigate admin guide that might help -

　

"The FortiGate unit does not participate in the Spanning Tree Protocol (STP). STP is an IEEE 802.1 protocol that ensures there are no layer-2 loops on the network. Loops are created when there is more than one route for traffic to take and that traffic is broadcast back to the original switch. This loop floods the network with traffic, reducing available bandwidth to nothing.

If you use your FortiGate unit in a network topology that relies on STP for network loop protection, you need to make changes to your FortiGate configuration. Otherwise, STP recognizes your FortiGate unit as a blocked link and forwards the data to another path. By default, your FortiGate unit blocks STP as well as other non-IP protocol traffic."

I've also attached a screenshot which I "think" is telling us that Switch3 is acting as the route bridge?  I looked at the four switches and all (except Switch3) are showing their Root Ports to be the physical interface that is connected to the Fortigate, but I suspect that is because the Fortigate switch is blindly forwarding on STP traffic so for instance Switch1 thinks it is talking to Switch3 through interface 2 but in reality it's going through the Fortigate on te way!

Again maybe I'm wrong to be worried about this, but it seems like if the firewall wasn't to allow communication between the four links then at least three of the inter-switch links could be enabled to create paths between them to avoid the firewall...  It's a great way to learn about STP if nothing else .

[Aninda Chatterjee]

Interesting. When you're using PVST+ (or Rapid_PVST+), a majority of  your BPDUs are sent to a Cisco proprietary mac address which is usually  not understood by third party devices (unless they are running in some  sort of compatibility mode). This results in the 3rd party device simply  flooding this out.

So for your topology, at the end of the day, I suppose  the firewall is behaving like a hub from a spanning-tree perspective.

I would like some more outputs to confirm what we are seeing. Any chance you have access to the CLI of these switches? If so, could you post the show spanning-tree vlan output for a VLAN from all 4 switches please?

Regards,

Aninda

[PHarrisonCWSI]

Yeah that certainly seems to be how it is behaving, again maybe this is normal but it doesn't seem optimal to me .

SW1-S1RAC4#show spanning-tree vlan 50

VLAN0050
  Spanning tree enabled protocol ieee
  Root ID    Priority    32818
             Address     2401.c72a.7c80
             Cost        4
             Port        2 (GigabitEthernet1/0/2)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32818  (priority 32768 sys-id-ext 50)
             Address     ccd5.3932.e580
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/0/1             Desg FWD 4         128.1    P2p
Gi1/0/2             Root FWD 4         128.2    P2p
Gi1/0/3             Altn BLK 4         128.3    P2p
Gi1/0/4             Altn BLK 4         128.4    P2p

SW2-S1RAC4#show spanning-tree vlan 50

VLAN0050
  Spanning tree enabled protocol ieee
  Root ID    Priority    32818
             Address     2401.c72a.7c80
             Cost        4
             Port        2 (GigabitEthernet1/0/2)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32818  (priority 32768 sys-id-ext 50)
             Address     5897.1ee1.2700
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/0/1             Desg FWD 4         128.1    P2p
Gi1/0/2             Root FWD 4         128.2    P2p
Gi1/0/3             Desg FWD 4         128.3    P2p
Gi1/0/4             Desg FWD 4         128.4    P2p

SW3-S2RBC3#show spanning-tree vlan 50

VLAN0050
  Spanning tree enabled protocol ieee
  Root ID    Priority    32818
             Address     2401.c72a.7c80
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32818  (priority 32768 sys-id-ext 50)
             Address     2401.c72a.7c80
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/0/1             Desg FWD 4         128.1    P2p
Gi1/0/2             Desg FWD 4         128.2    P2p
Gi1/0/3             Desg FWD 4         128.3    P2p
Gi1/0/4             Desg FWD 4         128.4    P2p

SW4-S2RBC3#show spanning-tree vlan 50

VLAN0050
  Spanning tree enabled protocol ieee
  Root ID    Priority    32818
             Address     2401.c72a.7c80
             Cost        4
             Port        1 (GigabitEthernet1/0/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32818  (priority 32768 sys-id-ext 50)
             Address     ccd5.3937.9a80
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/0/1             Root FWD 4         128.1    P2p
Gi1/0/2             Desg FWD 4         128.2    P2p
Gi1/0/3             Altn BLK 4         128.3    P2p
Gi1/0/4             Altn BLK 4         128.4    P2p

[Aninda Chatterjee]

Fantastic. Thank you very much for the outputs, Philip.

SW3 is clearly the root bridge:

SW3-S2RBC3#show spanning-tree vlan 50

VLAN0050
  Spanning tree enabled protocol ieee
  Root ID    Priority    32818
             Address     2401.c72a.7c80
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32818  (priority 32768 sys-id-ext 50)
             Address     2401.c72a.7c80
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi1/0/1             Desg FWD 4         128.1    P2p
Gi1/0/2             Desg FWD 4         128.2    P2p
Gi1/0/3             Desg FWD 4         128.3    P2p
Gi1/0/4             Desg FWD 4         128.4    P2p

All other switches see SW3 as the root. This can be confirmed from the 'Root ID' from the outputs. The 'Address' part of this lists out SW3 address - 2401.c72a.7c80.

I have one small question here - which port of SW3 is connected to the firewall? Would it be Gi1/0/1?

As an example, let's take SW1. This switch would be getting BPDUs from several places - most important of these would be the direct link between SW3 and itself, and then the link via the firewall. Any other path would would be of a much greater path cost.

Now, the path via the firewall and the direct link both have a path cost of 4. The tie breaker in this situation would be the sender port ID. You can do several things here:

1. Lower the port priority of the interface of SW3 that goes to SW1.

2. Lower the cost of the direct link between SW1 and SW3 to 3.

You can apply the same process for SW4.

Coming to SW2. From an algorithmic perspective, SW2 really has the best cost to reach the root bridge (SW3) through the firewall - it is a cost of 4. If it goes through SW1 or SW4, the cost jumps up to 8. So you'd really have to tweak your path costs if you would like SW2 to block its connection to the firewall.

Regards,

Aninda

[PHarrisonCWSI]

Yeah SW3 port 1 is connected to the firewall.

Thanks for that, starting to get a much clearer picture here .

The thing is I don't really want the switch-to-firewall links blocked by STP, ideally I want them AND the inter-switch links forwarding so that traffic destined for the Internet or other VLANs can go via the firewall but traffic between servers on different switches but in the same VLAN can just go across the switch-switch link.  From a loop point of view I think this should be doable?  I just need to figure out if we can stop the firewall acting as a switch and passing traffic from one link to the other right?