Implementing VSS on Catalyst 6880-X

[bilal-saleem]

Dear All,

I am installing two new Cisco 6880-X in our core and these devices will act as PE in the network. Both Catalyst will be connected to core routers using Point to Point Links, whereas all VRFs & SVI will be created on these two Catalysts. All downlinks will be configured as an L-2 using LACP.

We want to setup VSS on our 2x 6880-x with following modules in each 6880 chassis:

1x 6880-X-16P10G 16x SFP-10G-LR

We have three 10G links, 2x10G links will be configured as VSL and 1x10G link will be configured for Dual Active Detection. Need your support to address following queries;

• In normal conditions, will the links between both Catalysts configured as VSL be utilized? If yes, what would be the impact on traffic flow in terms of delay/latency? As we have uplinks connecting to the core from both Catalyst switches.
• VRFs will be configured on Catalyst 6880, is there any mechanism of making one Catalyst active for some VRFs and second catalyst active for remaining VRFs?
• In case of split-brain situation when the VSL link breaks and there is no connectivity between both Primary & Standby 6880.
  • How both Catalysts-6880 will act in this situation?
  • What would be the impact on routing?
• Known bug scripts of implementing VSS on Cisco Catalyst-6880-X.

Thanks in advance.

Regards,

Bilal Saleem

[Sigurbjartur Helgason]

Hi Bilal Saleem,

1. If the network design is symmetrical and the paths from the VSS nodes to the destination network are equal-cost, or the uplinks are multi-chassis ether-channels, a VSS node will always attempt to send traffic out of local links instead of having it traverse the VSL.

2. No, it's not possible to have more than one control plane running in a VSS. However, most traffic forwarding functions, such as FIB/LFIB lookups and label imposition/disposition are performed by the PFCs which are active in both nodes. The active node takes care of maintaining the BGP/IGP adjacencies and tables and then programs the hardware ASICs in both boxes appropriately. Traffic that is punted out of the fast path is sent to the "active" node via the VSL link.

3. If the DAD link goes out along with the VSL links you have a problem as you will most likely have two identical routers operating in your network. I can only imagine the havoc it will play. Sufficing to say, this must not happen. However, if the DAD link is still up, one of the routers will reload and come back up in RPR-recovery mode, shutting down all interfaces except for the VSL interfaces.

4. The best way to find information about bugs is reading the release notes for the IOS version you plan to use and looking it up in the Bug Search Tool on CCO.

As a side note; since Dual-active is a very bad thing to have. If your downstream equipment supports ePAgP you might consider using that to signal your ether-channels instead of LACP since ePAgP includes dual-active detection. That way you're covered from all sides in case the VSL goes down.

Regards,
   Sigurbjartur

[bilal-saleem]

Hi Sigurbjartur,

Good Day!

Many thanks for the detailed reply. please find bellow my response and it would be great to have your feedback.

If the network design is symmetrical and the paths from the VSS nodes to the destination network are equal-cost, or the uplinks are multi-chassis ether-channels, a VSS node will always attempt to send traffic out of local links instead of having it traverse the VSL.

• Both the Catalysts Switches will be connected to the Core Routers with using Point-to-Point 10 Gig link (uplinks are not multi-chassis but two routers connecting with each Catalyst of VSS), an IGP will be configured on these links with the core. So, as you said depending on equal-cost, one of VSS node will send traffic to uplink or it will use VSL.

No, it's not possible to have more than one control plane running in a VSS. However, most traffic forwarding functions, such as FIB/LFIB lookups and label imposition/disposition are performed by the PFCs which are active in both nodes. The active node takes care of maintaining the BGP/IGP adjacencies and tables and then programs the hardware ASICs in both boxes appropriately. Traffic that is punted out of the fast path is sent to the "active" node via the VSL link.

• So, only one control plane, and we will be configuring VRFs on active VSS member.

If the DAD link goes out along with the VSL links you have a problem as you will most likely have two identical routers operating in your network. I can only imagine the havoc it will play. Sufficing to say, this must not happen. However, if the DAD link is still up, one of the routers will reload and come back up in RPR-recovery mode, shutting down all interfaces except for the VSL interfaces.

• As I have read some forums regarding this, I am not sure if it is right or not. If the DAD link goes out along with the VSL links, the active control plane transitions to standby mode and disable all the interfaces as in operational shut down state.

One more question that I have and it would be great if you address this as well. All the aggregation layer devices will be configured in VSS as well and uplinks from Aggregation layer to core layer will be configured with LACP on mode. Same will apply on access layer .i.e. from access to aggregation uplinks will be configured with LACP on mode. Now the question is, will both the links part of an LACP will be utilized simultaneously? Or only one link will be utilized? In other words, will there be load balancing among both the links configured as LACP from access à Aggregation and Aggregation à Core

Regards,
Bilal Saleem

[Sigurbjartur Helgason]

Hi Bilal Saleem,

Both the Catalysts Switches will be connected to the Core Routers with using Point-to-Point 10 Gig link (uplinks are not multi-chassis but two routers connecting with each Catalyst of VSS), an IGP will be configured on these links with the core. So, as you said depending on equal-cost, one of VSS node will send traffic to uplink or it will use VSL

Actually, since you're connecting both VSS cluster members to both core routers and all access switches to both VSS nodes, no traffic should ever cross the VSL during normal operation.

As I understand it, should one uplink fail, it is possible that traffic may pass the VSL if there are any non-ECMP paths in the FIB. For that reason, it's very important that the Core/Aggregation design be fully meshed, causing all paths to be equal cost.

So, only one control plane, and we will be configuring VRFs on active VSS member.

Correct again. You should think of it as one box. You will be "configuring" it on the active one because that's the one with the active CLI. The important thing is that the configuration will be implemented on both chassis.

As I have read some forums regarding this, I am not sure if it is right or not. If the DAD link goes out along with the VSL links, the active control plane transitions to standby mode and disable all the interfaces as in operational shut down state.

I must admit that I'm not certain if that is true or not, but I don't recall seing that stated anywhere.

If I may offer an opinion; I find it very doubtful, since if the VSL and DAD links go down, it's more likely that the peer node has failed rather than all the links failing at exactly the same time. And, in the absence of the links, it's impossible for the members to distinguish between a dual-active condition or if the peer node has failed. Therefore, if the active member would transition to recovery mode as you suggest, it could mean that both switches go out, severing the connection to a part of your network.

That been said; what you describe is exactly what happens when the VSL links go down and the DAD link is up except it's the standby node that transitions to recovery mode.

All the aggregation layer devices will be configured in VSS as well and uplinks from Aggregation layer to core layer will be configured with LACP on mode. Same will apply on access layer .i.e. from access to aggregation uplinks will be configured with LACP on mode.

It is unclear to me what you mean when you say: "LACP on mode." Normally when setting a Cisco ether-channel member port to "on" means that you're forcing the channel up without negotiation and you are in fact not using any aggregation protocol such as LACP or PAgP. To enable LACP on a port in Cisco boxes you should use modes "Active" or "Passive"

Now the question is, will both the links part of an LACP will be utilized simultaneously? Or only one link will be utilized? In other words, will there be load balancing among both the links configured as LACP from access à Aggregation and Aggregation à Core

This may be vendor specific but on Cisco equipment, regardless of whether you use LACP, PAgP or no protocol at all, traffic flows will always be load-balanced across the member links of an ether-channel. By default the Catalyst 6880-X looks at the source and destination IP addresses of IP packets to compute the hash, but this is configurable.

You might like to take a look at the following chapter of the Campus 3.0 Virtual Switching System Design Guide.

Regards,
   Sigurbjartur