Spanning-tree basic: root bridge selects the wrong path

[Simonoulx27068]

Got a problem with integration between Ruckus and Cisco 9300 but after investigating the problem seems general and more related with the basic of STP behavior that I can't understand.

First here the schematic of the simplified network in cause:

Everything is working with spanning-tree PVST+. The Cisco 9300 is the root bridge and the Ruckus got the port 1/1/1 in the blocked state. We are using native vlan (1) for management. We know it's not the best practice we plan to change this.

All traffic for all VLANs works as it should. We can reach the Cisco management IP 172.16.1.39 without issue.

BUT we loose the connection intermittently to the Ruckus management IP address 172.16.1.109.

After investigating the log from both sides the issue seems from Cisco 9300.

The problem happens when the control data (CDP, LLDP, BDPU) flows on the block port 1/1/1 as it should. But at that time the Cisco 9300 (root bridge) updates is mac address table for the Ruckus device but on the wrong port 1/0/23. After that all user data like HTTP or ICMP takes the wrong designed port 1/0/23 to the Ruckus device. Since on the other side the port is blocked the communication is lost.

With Cisco device we don't have this problem. So maybe there is some kind of mecanism to prevent this (CDP?).

So how the root bridge knows wich designated port to use if control data can be receive from two ports coming from the same foreign device?

[Georg Pauwen]

Hello,

on the 9300, try and set interface 2/1/6 as the port with the lower port priority:

9300#conf t

9300(config)#interface GigabitEthernet 2/1/6

9300(config-if)# spanning-tree port-priority 0

[Simonoulx27068]

Hi Georg,

Thanks for helping but maybe I don't explain correctly (sorry for my english!).

Spanning-tree works as it should for 99,9% of the traffic. The problem concerns ONLY the management IP address of the Ruckus device and I need basic explanation of how works STP on the root bridge when there is two designated port to the same foreign device. How the root bridge decide wich port to use since both are in forwarding state? 

The root bridge seems to decide by learning from wich port the foreign mac address was seen last time. That's work because all downstream packets come from the right port. BUT control data (BDPU, LLDP, CDP) coming from the Ruckus take both links and the Cisco 9300 learns the mac adress table of the Ruckus device intermittenlty on the wrong port. When this happen we can't reach the Ruckus device. 

Here what is happening. After control data was received from the port #2 (blocked on the Ruckus side), the root bridge try to reach the Ruckus device, and ONLY the Ruckus device using the wrong port #2. This issue doesn't happen when Cisco devices are connecting together in the same fashion so I am thinking something prevent this, maybe CDP.

[Georg Pauwen]

Hello,

the idea of setting the port priority is so the 9300 never uses the other port. In theory, there should never be two designated ports in forwarding state connected to the same device...

Sorry if I misunderstand what you are asking. What happens if you actually do set the priority as suggested ?

[paul driver]

---

@Georg Pauwen wrote:

Hello,

the idea of setting the port priority is so the 9300 never uses the other port. In theory, there should never be two designated ports in forwarding state connected to the same device...

---

Aoplogies @Georg Pauwen  but this is incorrect all ports on the stp root will be designated ports and as the c9k is the stp root so they would all be in a forwading state

[Georg Pauwen]

Two (designated) ports connected to the same neighboring switch are both forwarding ? What is the purpose of STP then ?

[Georg Pauwen]

Here I have Switch 1 connected to Switch 2 through ports GigabitEthernet 0/1 and 0/2. Switch 1 is the root. How can I get both ports to be in FWD state ? Or is that something specific to the 9300 ?

Switch1#show spanning-tree vlan 1
*Jan 20 00:33:31.155: %PLATFORM-5-SIGNATURE_VERIFIED: Image 'flash0:/vios_l2-adventerprisek9-m' passed code signing verification

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 24577
Address 0cad.24e6.d900
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 24577 (priority 24576 sys-id-ext 1)
Address 0cad.24e6.d900
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/0 Desg FWD 4 128.1 Shr
Gi0/1 Desg BLK 4 128.2 Shr

[Simonoulx27068]

Hi Georg,

Yes that is the way STP works. The root bridge gets all ports in forwarding state. This is the switch below the root bridge that is blocking one port to prevent loop.

https://www.cisco.com/c/en/us/support/docs/lan-switching/spanning-tree-protocol/5234-5.html

STP Rule 1—All ports of the root switch must be in forwarding mode.

[paul driver]

Hello

Is the ruckus using mst if so then you would have a interconnection between a stp pvst and non pvst domains, The ruckus will only understand vlan 1 bpdus all other bpdus for the other vlans it wont understand,

So you need to make vlan 1 on the cisco pvst domain have a better stp value (lower) than the ruckus vlan 1 but a higher value for all of its other vlans

C9K 
vlan 1 priority 4096
vlan 2−4094 priority 0

Ruckus 
Mst 0 priority 8192

[Simonoulx27068]

Thanks,

All VLANs, including VLAN 1 are on the same STP domain. All traffic flows as it should for all VLANs.

The problem is ONLY for reaching the management IP address of the Ruckus (172.16.1.109) that is on the VLAN 1. But we can reach normally the Cisco 2960 connected on the Ruckus side with is address on the same VLAN 1 (172.16.1.39). So there is no problem with STP for "normal" traffic. It's only for managing the Ruckus switch...

[Simonoulx27068]

When we clear the mac address-table on the root bridge the problem is fix for few seconds.
It's why I'm thinking the root bridge uses her mac address table to know wich forwarding port to use.

[paul driver]

Hello

Can you confirm the native(untagged) vlan between the cisco 9k and the ruckus switchports?
Post he config of those two ruckus ports please that are interconnecting the cisco devices.

  -

show spanning tree interface 1/1/1 ot 1/4/1 ( if applicable)
show spanning vlan xx ( native vlan)

[Simonoulx27068]

I had drill down the problem and I confirm that the problem comes from the mac address table (CAM) on the root bridge that gets updated by two control data protocol: LLDP and FDP. FDP is an equivalent of CDP for discovering neighbors devices in a Ruckus world.
I have disabled the LLDP and FDP transmission on the blocked port 1/1/1 with these commands:
no lldp enable ports ethernet 1/1/1
interface ethernet 1/1/1
no fdp run
And this fix the issue. But that's doesn't look normal to do this.

Again my question still did not been answered. What's is preventing the root bridge to use the blocked port when there is flow control data coming from the blocked port? I was thinking the mac address table (CAM) on the root bridge shouldn't be updated with these protocols. Can someone can confirm this statement? Maybe CDP doesn't cause rewrite of the CAM and it's why we don't have this problem with Cisco devices. But LLDP is a standard and pretty common so why LLDP updates the CAM on the root bridge? Is that the normal behavior or a bug?

[paul driver]

Hello

---

@Simonoulx27068 wrote:

What's is preventing the root bridge to use the blocked port when there is flow control data coming from the blocked port?

---

As we are on about spanning-tree pvst, then the root switch (C9K) shouldnt be recieving any stp BPDU's and the only time this would happen is if the ruckus sends a notification of a change towards the root switch about a link change via its root port which in your toplogy is ruckus port 1/4/1.
.

---

@Simonoulx27068 wrote:

I was thinking the mac address table (CAM) on the root bridge shouldn't be updated with these protocols. Can someone can confirm this statement? ?

---

You are correct the switches cam table is updated to assoicate mac address(s) of hosts reachable through the active port it shouldnt as my understaind do this on any blocked port and as I said no stp bpdu would be sent through a blocked port of a non stp root switch even though an stp bpdu is recieved from the root switch on the ruckus blocked port.

[Simonoulx27068]

Hi Paul,

You're right there is no BPDU frame received on the root bridge coming from the blocked port on the Ruckus side. BPDU frames are going downward and the Ruckus receives BPDU frames on both ports (blocked and forwarding state).

BUT the root bridge still continues to receive flow control even if the downward bridge is blocked (CDP, LLDP, PAgP, LACP, VTP). To verify that if we do "show cdp neighbors" on the root bridge we can see foreign Cisco devices even if the downward port is blocked. So CDP is going upward to the root bridge from the blocked port.