cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3101
Views
0
Helpful
7
Replies

Nexus 5000 system-mac command does not work as expected ?

dumlutimuralp
Level 1
Level 1

Hi,

I am using a Nexus 5000 vPC environment with NX-OS 5.0.3.N1.1c.

Just one edge switch (C3569) is connected to N5Ks as a vPC using LACP as channel protocol. I have used system-mac 0001.0010.0010 command under vpc on N5Ks.

Weird thing is , C3560 sees vPC system mac as expected but spanning-tree output shows the local mac of the primary N5K in the vPC. So when I reboot the primary N5k an STP convergence occurs.

I know that on N7Ks, this situation is overcome by using peer-switch command so that both N7Ks generates STP BPDUs with the system-mac.

Any comments appreciated.

Dumlu

7 Replies 7

Hi dumlu

Can you post your N5K relevant config and the output of your MAC table on the 3560.

Hi Ruhann,

It was a testbed and we had to poweroff those equipment in order to ship to their final production envirionment. So I do not have any output.

I noticed that I forgot to mention, Nexus ports which are connected to 3560s are MSTP P2P Boundary ports. Cause 3560s run PVST+ , however the root bridge is set to N5K-1.

If you say that you do not see this behaviour in a similar environment than it is ok. You seem to be a well known Nexus expert.

Dumlu

lol, thanks but not really, just been spending lots of playing with the Nexus boxes.

With regard to your question, the function of the VPC MAC should operate the same on both a pair of N7K, and N5K.

Manually specifying the VPC system MAC should have the same behavior as using the default, but I have seen one or two funnies with that.

For the most part it should work as you expected

hth

Yeah,

Cause as you know when vPC peers use different MAC addresses an STP convergence is inevitable when I reboot the vPC Operational Primary device.

Spanning tree and LACP generally use a switch's system MAC address for bridge ID field in the spanning tree BPDU and as part of LACP LAGID, respectively.  In a single chassis, the systemwide MAC address is used for this purpose.

But with VPCs, using the systemwide MAC address would not work because the vPC peers needs to appear as a single entity, hence the purpose of the VPC MAC.

Taking that a step further, the operational primary VPC device is responsible for answering and creating BPDUs. If the operational primary fails the secondary takes over the primary role as answers and sends the BPDU's.

With VPCs the operational primary would be sending traffic sourced with the VPC MAC address, but only on VPC member ports. On non VPC ports it both switches will use their system MAC as source.

Have a read through the following guides. Its explains this further

http://www.cisco.com/en/US/products/ps9670/products_implementation_design_guides_list.html

hth

Hi Ruhan,

Ive read these notes before.  Thanks for mentioning.  The thing is 3560 was connected to the pair of N5Ks through a vPC. That is why I expect to see the same Bridge ID on 3560.

However when I reboot the Operational Primary vPC peer (N5K-1) (system wide MAC address was set as 0001.1010.1010 on both N5Ks), I observed that the root bridge id changed to the MAC address of the N5K-2. 

However with LACP I did not observe the same behaviour. On LACP output I always saw the same system-priority and MAC whether if the N5K-1 is vPC primary or N5K-2 is vPC primary.

That is why I thought may be the behaviour was implemented on STP as well.

I know they enhanced the STP behaviour of N7Ks by developing peer-switch feature which used to make both N7Ks produce BPDUs with same system MAC.

Appreciate it.

Dumlu

Dumlu,

You are right that the peer-switch feature has changed the way that the two vPC peers send BPDUs out of their local vPC member ports. With the peer-switch feature, both vPC peers (operationally primary and operationally secondary) send BPDUs out of their vPC member ports toward Layer 2 access switches connected to the vPC system with vPCs. The BPDUs that they send contain the same bridge ID, which means that they are using the same bridge priority and the same system MAC. This system MAC should be automatically generated and contain the vPC domain ID in its least significant bits. If you don't want the vPC system to use this automatically generated and shared between the two vPC peers MAC address, you can use the system-mac command on both vPC peers to manually assign a shared MAC address that would be used in BPDUs sent out of local vPC member ports by both vPC peers.

Before the peer-switch feature was introduced, only the operationally primary vPC peer would send BPDUs out of its local vPC member ports. These BPDUs contained the bridge ID of the STP root configured on one of the vPC peers. However, if the STP root is currently operationally secondary vPC peer, then the operationally primary vPC peer sends BPDUs out of its local vPC member ports, but it uses the other vPC peer's (the STP root but operationally secondary vPC Peer) system MAC in the BPDUs. The operationally secondary vPC peer (even if it is configured as the STP root) would forward BPDUs received on its vPC member ports from Layer 2 switches onto the operationally primary vPC peer across the vPC peer link.

The same system-mac command has a similar effect in LACP, where LAGID  consists of the system ID and the channel-group number. The system ID  consists of the system priority and the system MAC. That system MAC is  created automatically on both vPC peers with the least significant bits  of the last octet populated with the vPC domain ID. Just like with the  system MAC address used in STP BPDUs, you can override this  automatically generated MAC address, using the same system-mac command  on both vPC peers, except that if you do this, LACP will always use the  system MAC that you manually configured. Again, STP will only use this  system MAC (or automatically generated system MAC for that matter) if  you enable the peer-switch feature.

The behavior you are describing seems to be consistent with the pre-peer-switch feature behavior. With that *legacy* behavior, if the operationally primary vPC peer was the same switch as the STP root, it would use its own system MAC in the BPDUs sent out of its local vPC member ports. This MAC address would not be shared between the two vPC peers. When the operationally primary vPC peer, which is also the STP root, would go down, the operationally secondary vPC peer would start sending BPDUs containing its own MAC address out of its local vPC member ports. Therefore, a Layer 2 switch connected to the vPC system with a vPC would see a new root (with a different MAC address). In this case, the system-mac command has no effect on which MAC address is used in BPDUs sent by only one vPC peer at a time. The behavior gets even more strange in the Active/Active (split-brain) scenario, where both vPC peers start forwarding BPDUs out of their local vPC member ports, using their own MAC addresses. The downstream Layer 2 switches would then see two different STP roots on of the same PortChannel interface because the BPDUs from both vPC peers in a split-brain condition will arrive on different links that are members of the same PortChannel on the Layer 2 switch. Therefore, I would much rather use the peer-switch feature now that it has become available.

Now, I have a question for you, How do you look up the MAC address used in LACP? I don't seem to be able to find a command for this.

Thanks!