two or more chassis with clustered fabric interconnect

[minghui.qi]

Hi experts

Just wanna confirm the proper way to connect two or more UCS 5108 chassis with a pair of clustered 6120xp fabric interconnect.

I though they should connect like this: the first fabric extender of all chassis should connect to fabric interconnect A, and second fabric extender of all chassis connects to fabric interconnect B. like in the diagram here:  https://supportforums.cisco.com/docs/DOC-5949

however I have seen someone did this: chassis 1 connects as I though above, but for chassis 2, fabric extender 1 connecting to fabric interconnect B, and fabric extender 2 connecting fabric interconnect A - crossed over.

can someone tell me which is appropiate/best practice? any reason why crossing over on the second chassis??

thanks

ming

[stechamb · Accepted solution]

ming, you need to fix that implementation.  it's like ghostbusters: don't cross the streams!

Always consistently cable chassis fex to the same fabric interconnect.  Most common setup:

Fabric Interconnect A (Primary) is connected to all Left Hand (looking from back of chassis) FEX

Fabric Interconnect B (Subordinate) is connected to all Right Hand FEX.

Why?  You will get errors (can't be bothered looking them up!) and if you move a Service Profile from one chassis to another chassis that is cabled differently, then it won't work (CNAs cabled to different FIs where VIFs are active / inactive = bad).

PS - to change this, if I was you, I would power down the whole system - fabric interconnects, chassis, everything.  then re-cable.  then power up the fabric interconnects and chassis and check the configurations are ok. 

[minghui.qi]

Hi Steve,

thanks for your reply, I will get it fixed:

could you plealse explain a bit more on "it won't work (CNAs cabled to different FIs where VIFs are active / inactive = bad).", just wanna be clear on the real impact.. or please point me to a link or doc, I can't find such confiugration on standard Cisco Nexus 1000 v docs.

thank you

ming

[Nethaji Vinobaji]

Hi Ming,

Here is the link for Best Practices guide for Deploying Cisco Nexus 1000V Series Switches on
Cisco UCS Systems.

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white_paper_c11-558242.pdf

Hope this helps

-Nethaji V

[minghui.qi]

Thanks, Nethaji.. I have already got that doc. but I can't find it mentioning about two more more chassis connectivity, maybe I missed it?

[stechamb]

http://www.cisco.com/en/US/docs/unified_computing/ucs/hw/chassis/install/install.html#wp1294477

Scroll down a bit and you'll see the strict instructions.

[stechamb]

http://www.cisco.com/en/US/docs/unified_computing/ucs/hw/chassis/install/install.html#wp1294477

Scroll down a bit and you'll see the strict instructions.