Design Help: MEC across Domains in Stackwise Virtual

[musa-ktk]

Hello,

I am working on 4 devices of C9500s and need to clarify a design to create 2 Domains (each having 2 members)

Let's say we have 2 buildings, and each will have 2 of C9500s.

In Building A I have a domain 10.
In Building B I have a domain 20.

Now I also need to connect both buildings with 40G fibers. 2 links from each member switch, so 4 Links in total from Building 1 to Building 2.

How do I connect these 4 links across the buildings? My thoughts are to connect them across:

Domain 10: Switch1 connects to Domain20: Switch1 and Domain 20 Switch2
Domain 10: Switch2 connects to Domain 20: Switch1 and Domain 20 Switch2

So that means I am configuring Multi-chassis EtherChannel (MEC) across the stack members which is mentioned in this guide:High Availability Configuration Guide, Cisco IOS XE Dublin 17.10.x (Catalyst 9500 Switches) - Configuring Cisco StackWise Virtual [Cisco Catalyst 9500 Series Switches] - Cisco in Figure 1. Typical Network Design using Cisco StackWise Virtual. also attaching image here.


Also to keep in mind my Servers are all inside Building A (Domain 10). And all other edge switches (Access switches) connect to these C9500s.

The confusion I have is due to this paragraph right under the image in cisco articel i shared above:

Etherchannel in StackWise Virtual is capable of implementing Multi-chassis EtherChannel (MEC) across the stack members. When access layer and aggregation layer are collapsed into a single StackWise Virtual system, MEC across the different access layer domain members and across distribution and access layer switches will not be supported. MEC is designed to forward the traffic over the local link irrespective of the hash result.

I do not understand what it means what is not supported? and does it apply in my design? so a help on this is appreciated.

Is there anything I need to look out for? any gotchas? or design problems? 

Thank you

[MHM Cisco World · Accepted solution]

What we can get from normal PO?

Redundacy and load balance

In MEC we only get redundacy' the SW always try forward frame by it local link not send it via SVL to neighbor and neighbor forward to access SW (or host/server) via it link of MEC.

MHM

[Joseph W. Doherty]

I do not understand what it means what is not supported?

What it means, egress from one domain switch member, to its peer, is never used unless there no directly connected physical egress path from the switch member wanting to transmit traffic.

For example, domain 10 sw1 has data to send to the bottom left access switch.  Logically, there's a port channel between domain 10 and that access switch, so logically either port channel member could be used, but that's not what happens.  Domain 10 sw1 will only use the physical link directly connected between it and the bottom left access switch.  It would only send traffic to the bottom left access switch, via domain 10 sw2 if the domain 10 sw1 link to the bottom left access switch is down.  Conversely, the bottom left access switch would send its egress traffic to the logical domain 10 switch using either port channel link, following normal port channel distribution rules.

and does it apply in my design?

It does.  You want to always avoid forcing the inter domain path.

For example, if you, by design, only connected the bottom left access switch (or a sever) to domain 10 sw2.

Is there anything I need to look out for? any gotchas? or design problems? 

Yes, always have a physical connection from each domain switch member to devices the member pair connect to.

Design wise, consider what happens if a line card fails.  Assuming ever other device has a connection to both domain switch members, logically there should be no lost of connectivity, but all that failed line card's egress traffic now has to use the domain switch interconnection.  Does that interconnection allows for a whole line card's worth of bandwidth?  Possibly not, and that's without considering what other single homed connectivity might be using.