How to configure a spine for breakout ports

[blebarro]

Hi, I have a new pod config we are trying to get working.  We have a 400g spine and would like to breakout a port from 400g to 4x100g.  I see how to do this in the apic for a leaf, but not for a spine.  My spine is a 9332D-GX2B, my codes are at 5.2.(7g).   Any suggestions would be greatly appreciated.

[Robert Burns]

I assume you're trying to discover a new Spine connected to a Leaf which the connection between Leaf & Spines is a 4x breakout (400G interface on GX2 > 4x100G Leaf connections).  For this you'll the switch uses Auto Breakout (rather than one that would have to be configured by APIC policy), no specific configuration is required as long as the breakout cable is supported on the platform.  Additionally GX2 switches default to Leaf role, which will requiring a couple things:

1. You'll need to use one of the Default Fabric ports on the 9332D-GX2, which are ports 25-32.  Ref

2. When you discover the 9332D-GX2 switch, you'll need to set it as "spine" role, which will cause it to reboot (be patient).  Once reboot, all interfaces will be configured as Fabric ports and can be used to connect to any other Leafs via breakout or otherwise.

Robert

[blebarro]

We tried to use the breakout cable from the spine to the leaf but it wouldn't discover the spine.    So we put a 100g qsfp in both the leaf (C93180YC-FX) and the spine just to get the spine discovered.  Now that the apic can see the spine, we want to use the 400g qsfp with the breakout cable for the connection to the leaf (C93180YC-FX) but there is no way to do breakout on a spine that I can see.  After reading your reply, I moved the qsfp to port 25 but I still don't get a link light and it says it is not connected.  I thought there was a dynamic qsfp and a non-dynamic, we have the 400g-DR4-S.  Thoughts?

[Robert Burns · Accepted solution]

I don't believe you can use that optic (QDD-400G-DR4-S) for breakout on Fabric interfaces.  They work only for Access ports (Leaf host connections).  If you want to breakout the Spine fabric interfaces then your options are listed in the APIC Layer2 Networking guide. 

Keep in mind, when you see optics listed for the platform on TMGmatrix showing switch compatability (HW) you have to also check the Software support (ACI) as GX/GX2 switches can operate as Leaf or Spine roles.  

Robert

[blebarro]

Thank you so much for all your help Robert. This was the fear. I have one more issue. I tried to connect the spine to a 1g leaf. ( N9K-C9348GC-FXP) with a 100g-DR4-S transceiver on both ends. (since we aren't able to use the breakout ports) I am not getting any link lights on either end. When I did this on the other leaf to spine, I had to enable the port on the leaf to be fabric vs access. Not sure what to do on the spine to get the 1G discovered by the apic. Any thoughts?

[Robert Burns]

Is the Leaf and Spine running 15.1 or later? 
Robert

[blebarro]

It is now and the leaf connected with the 100g. Thank you so much for all your help.

[RedNectar]

If your question has been answered, it is a great idea to mark the question as being answered.  This helps:

1. others with a similar problem find the correct answer
2. people who look for "unanswered" questions to answer finding this
3. prevent your question from becoming a "dead thread"