Connecting SG500X-24 to 3132Q-32 using 4-to-1 breakout cable, etc.

plu · ‎03-23-2021

I'm trying to connect a SG500X-24 GigE switch to a 3132Q-32 40Gbps switch using a 4-to-1 (40Gpbs-to-4x10Gbps) breakout cable. It seems I need to do some configuration on the 3132Q-32 to get the 40Gbps port (port #2) up and running to connect over the breakout cable. I am able to get to the 3132Q-32 over the serial console to access the NXOS CLI, but am overwhelmed by the reams of commands in the command reference, which seem to be only searchable alphabetically.

Also, the 3132Q-32 is supposed to allow 40Gbps port #1 to be multiplexed with the four 10Gbps ports, but I think some software configuration needs to happen with this as well, based on the LED indicators.

I am able to connect 10Gbps controllers to the 10Gbps port(s) on the SG500X-24 and achieve IP connectivity, so I know the connection is reasonable on the SG500X-24 side.

Would greatly appreciate any help on this.

Thanks in advance.

Reza Sharifi · ‎03-23-2021

Also, the 3132Q-32 is supposed to allow 40Gbps port #1 to be multiplexed with the four 10Gbps ports

According to the datasheet, the switches do support breakout cables.

https://www.cisco.com/c/en/us/products/switches/nexus-3132q-v-switch/index.html

As for the command to breakout the cable try this:

interface breakout module module number port port rangemap 10g-4x

config guide:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/interfaces/7_x/b_Cisco_Nexus_3000_Series_NX-OS_Interfaces_Configuration_Guide_7x/b_Cisco_Nexus_3000_Series_NX-OS_Interfaces_Configuration_Guide_7x_chapter_01.html

HTH

plu · ‎03-23-2021

The command offered seems to work, so I have ports #1 and #2 set with:

interface breakout module 1 port 1 map 10g-4x

interface breakout module 1 port 2 map 10g-4x

with the intent of using port #1 for four 10g links to 10g controllers, over the muxed four individual SFP+ ports

and using port #2 for the 4-to-1 breakout cable going to the SG500X-24 switch.

The connection on port #2 seems to be better/operational as its amber LEDs are now lit (whereas they were dark before the interface breakout command).

However, the LEDs on port #1 still have the QSFP+ port part lit (amber) and not the four SFP+ port parts. This seems to indicated I need to issue some other command to mux-select the connection from the QSFP+ port to the SFP+ ports.

Can you help me with this? I don't see any obvious commands in the config guide.

Thank you very much.

Reza Sharifi · ‎03-23-2021

Can you assign a speed command to each 10Gig interface?

speed 10000

and test again?

HTH

plu · ‎03-24-2021

Hi,

I tried the "speed 10000" but it wasn't accepted. In Web searching more solutions, I saw that all my ports were set for breakout to 10Gbps, so I changed ports 3-32 to 40Gbps. Now I see:

switch(config)# show int brief

--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
mgmt0 -- up 192.168.1.231 1000 1500

--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Por
t
Interface Ch
#
--------------------------------------------------------------------------------
Eth1/1/1 1 eth access down XCVR not inserted 10G(D) --
Eth1/1/2 1 eth access down XCVR not inserted 10G(D) --
Eth1/1/3 1 eth access down XCVR not inserted 10G(D) --
Eth1/1/4 1 eth access down XCVR not inserted 10G(D) --
Eth1/2/1 1 eth access down Administratively down 10G(D) --
Eth1/2/2 1 eth access down Administratively down 10G(D) --
Eth1/2/3 1 eth access down Administratively down 10G(D) --
Eth1/2/4 1 eth access down Administratively down 10G(D) --
Eth1/3 1 eth access down Administratively down 40G(D) --
Eth1/4 1 eth access down Administratively down 40G(D) --
Eth1/5 1 eth access down XCVR not inserted 40G(D) --
Eth1/6 1 eth access down XCVR not inserted 40G(D) --
Eth1/7 1 eth access down XCVR not inserted 40G(D) --
Eth1/8 1 eth access down XCVR not inserted 40G(D) --
Eth1/9 1 eth access down XCVR not inserted 40G(D) --
Eth1/10 1 eth access down XCVR not inserted 40G(D) --
Eth1/11 1 eth access down XCVR not inserted 40G(D) --
Eth1/12 1 eth access down XCVR not inserted 40G(D) --
Eth1/13 1 eth access down XCVR not inserted 40G(D) --
Eth1/14 1 eth access down XCVR not inserted 40G(D) --
Eth1/15 1 eth access down XCVR not inserted 40G(D) --
Eth1/16 1 eth access down XCVR not inserted 40G(D) --
Eth1/17 1 eth access down XCVR not inserted 40G(D) --
Eth1/18 1 eth access down XCVR not inserted 40G(D) --
Eth1/19 1 eth access down XCVR not inserted 40G(D) --
Eth1/20 1 eth access down XCVR not inserted 40G(D) --
Eth1/21 1 eth access down XCVR not inserted 40G(D) --
Eth1/22 1 eth access down XCVR not inserted 40G(D) --
Eth1/23 1 eth access down XCVR not inserted 40G(D) --
Eth1/24 1 eth access down XCVR not inserted 40G(D) --
Eth1/25 1 eth access down XCVR not inserted 40G(D) --
Eth1/26 1 eth access down Link not connected 40G(D) --"
Eth1/27 1 eth access down XCVR not inserted 40G(D) --
Eth1/28 1 eth access down XCVR not inserted 40G(D) --
Eth1/29 1 eth access down XCVR not inserted 40G(D) --
Eth1/30 1 eth access down XCVR not inserted 40G(D) --
Eth1/31 1 eth access down XCVR not inserted 40G(D) --
Eth1/32 1 eth access down XCVR not inserted 40G(D) --
switch(config)#

So I need to get turn off "Administratively down" and "Link not connected." I looked for ways to do this, but it seems various versions of Nexus OS use different commands and so have not been able to do so so far with the Nexus OS 7.3 I'm using.

For the Eth1/1/* breakout ports, I still need to (know how to) mux the signals from the QSFP+ port to the four SFP+ ports. The status says "XCVR" not inserted, but I'm sure two of the SFP+ ports have "live/good" 10Gbps cables inserted, connected to 10Gbps controllers on the other end (the ones that connect fine to the SFP+ ports on the SG500X-24 switch). The current set-up is for using a 4-to-1 breakout cable on the QSFP+ port, like what I have going on Eth1/2/*.

Thank you very much for help.

plu · ‎03-24-2021

I came across this interface configuration guide for Nexus 3000 (release 6.x, close enough to the 7.3 I'm using)...

and it provided the information I needed (at least for the most part).

https://manualzz.com/doc/23050114/cisco-nexus-3000-series-nx-os-interfaces-configuration-guide

In addition, this article:

https://community.cisco.com/t5/other-service-provider-subjects/interface-administratively-down/td-p/1656446

So...

conf t

int Eth1/2/1-4

no shut

int Eth1/3-4

no shut

exit

Got rid of the "Administratively down" mode.

And

cont t

hardware profile front portmode sfp-plus

exit

Muxed port #1 signals to the four SFP+ ports from the one QSFP+ port.

So, now I'm getting much closer to what I want and expect...

switch(config-if)# show int brief

--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
mgmt0 -- up 192.168.1.231 1000 1500

--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Por
t
Interface Ch
#
--------------------------------------------------------------------------------
Eth1/1/1 1 eth access up none 10G(D) --
Eth1/1/2 1 eth access down XCVR not inserted 10G(D) --
Eth1/1/3 1 eth access down XCVR not inserted 10G(D) --
Eth1/1/4 1 eth access down XCVR not inserted 10G(D) --
Eth1/2/1 1 eth access up none 10G(D) --
Eth1/2/2 1 eth access up none 10G(D) --
Eth1/2/3 1 eth access up none 10G(D) --
Eth1/2/4 1 eth access up none 10G(D) --
Eth1/3 1 eth access up none 40G(D) --
Eth1/4 1 eth access up none 40G(D) --
Eth1/5 1 eth access down XCVR not inserted 40G(D) --
Eth1/6 1 eth access down XCVR not inserted 40G(D) --
Eth1/7 1 eth access down XCVR not inserted 40G(D) --
Eth1/8 1 eth access down XCVR not inserted 40G(D) --
Eth1/9 1 eth access down XCVR not inserted 40G(D) --
Eth1/10 1 eth access down XCVR not inserted 40G(D) --
Eth1/11 1 eth access down XCVR not inserted 40G(D) --
Eth1/12 1 eth access down XCVR not inserted 40G(D) --
Eth1/13 1 eth access down XCVR not inserted 40G(D) --
Eth1/14 1 eth access down XCVR not inserted 40G(D) --
Eth1/15 1 eth access down XCVR not inserted 40G(D) --
Eth1/16 1 eth access down XCVR not inserted 40G(D) --
Eth1/17 1 eth access down XCVR not inserted 40G(D) --
Eth1/18 1 eth access down XCVR not inserted 40G(D) --
Eth1/19 1 eth access down XCVR not inserted 40G(D) --
Eth1/20 1 eth access down XCVR not inserted 40G(D) --
Eth1/21 1 eth access down XCVR not inserted 40G(D) --
Eth1/22 1 eth access down XCVR not inserted 40G(D) --
Eth1/23 1 eth access down XCVR not inserted 40G(D) --
Eth1/24 1 eth access down XCVR not inserted 40G(D) --
Eth1/25 1 eth access down XCVR not inserted 40G(D) --
Eth1/26 1 eth access down Link not connected 40G(D) --
Eth1/27 1 eth access down XCVR not inserted 40G(D) --
Eth1/28 1 eth access down XCVR not inserted 40G(D) --
Eth1/29 1 eth access down XCVR not inserted 40G(D) --
Eth1/30 1 eth access down XCVR not inserted 40G(D) --
Eth1/31 1 eth access down XCVR not inserted 40G(D) --
Eth1/32 1 eth access down XCVR not inserted 40G(D) --
switch(config-if)#

Eth1/1/1 does have a link to a 10Gbps controller at the other end, over the SFP+ port, while

Eth1/1/2-4 ports are empty.

Eth1/2/1-4 has the 1-to-4 (or 4-to-1) breakout cable going to the SG500X-24 switch.

There seems to be something wrong with Eth1/3 and Eth1/4, as I don't (think I) have any

transceivers inserted on them, but that's a story to be followed up. Perhaps they are defective

and hence set to "Administratively down." Don't know if "Administratively down" is something

only the user can set via "[no] shut" or something the system could also set by itself based on

some initialization conditions.

I still need to figure out the Eth1/26 "Link not connected" issue. But, I'm getting there.

Thanks.