cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
702
Views
3
Helpful
8
Replies

Can't Quote the Nexus 9332C

ex-engineer
Level 1
Level 1

Hi, folks. For some reason I am not able to find this switch in the Cisco CCW tool. I am using the right SKU right out of the ordering guide (N9K-C9332C), but all I get is 1 "remanufactured" option. Strange.

Can someone help? I need a 100G spine switch for a VxLAN/EVPN (NXOS) fabric. This one is perfect, but it won't come up in cisco commerce tool. Am I doing something wrong? Is this switch EoL? Is there another 100G spine switch for NXOS that has replaced this one? 

40/100G switch matrix where the Nexus 9332C appears

https://www.cisco.com/c/dam/en/us/products/switches/nexus-9000-series-switches/nexus-9300-40GE-switches-comparison.html

N9K-C9332C Datasheet

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/datasheet-c78-739886.html

2 Accepted Solutions

Accepted Solutions

Hello!

The N9K-C93600CD-GX can provide native 100G (as well as native 40G, if needed) ports without breakout. The previously-cited matrix is indicating the absolute maximum number of 100G links the switch can bring up. The data sheet for the Nexus 9300 GX series of switches confirms that the N9K-C93600CD-GX has 28 100/40Gbps QSFP28 ports acting as downlinks, with 8 400/100Gbps QSFP-DD ports acting as uplinks. If all 8 uplinks were broken out into 4 sets of 100Gbps links, that'd result in a total of 60 100Gbps ports.

With that being said, because the uplinks are 400/100Gbps QSFP-DD ports, they do accept native 100Gbps QSFP28 transceivers that link up at 100Gbps as you desire. We can confirm this by referencing the Cisco Optics-to-Device Compatibility Matrix filtered for this product; as you can see, there are numerous 100G transceiver options for this switch, both for the downlinks and the uplinks.

You've mentioned 100Gbps capabilities thus far, but just in case you have needs for 10/25/40Gbps, I recommend reviewing the "Overview" chapter of the Cisco Nexus 93600CD-GX NX-OS Mode Switch Hardware Installation Guide, which has important information about how the "port quad" mechanism works when 10/25Gbps links speeds are needed for this particular switch (either when working with breakout or through a CVR/QSA adapter).

I hope this helps - thank you!

-Christopher

View solution in original post

Hello!

Yes, you can convert the 400Gbps ports to 100Gbps ports simply by inserting a 100Gbps QSFP28 transceiver/optic; no breakout cable is needed. To remove any doubt, we have a N9K-C93600CD-GX in our lab environment running NX-OS 10.5(1):

N9K-1# show module 
Mod Ports Module-Type Model Status --- ----- ------------------------------------------------ --------------------- -------- 1 36 28x100/40G + 8x400G QSFP-DD Ethernet Module N9K-C93600CD-GX ok 27 0 Virtual Supervisor Module N9K-C93600CD-GX active * Mod Sw Hw Slot --- ----------------------- ------ ---- 1 10.5(1) 3.0 NA 27 10.5(1) 3.0 VSUP Mod MAC-Address(es) Serial-Num --- -------------------------------------- ---------- 1 54-51-de-f9-fb-10 to 54-51-de-f9-fb-df FDOREDACTED 27 54-51-de-f9-fb-10 to 54-51-de-f9-fb-df FDOREDACTED Mod Online Diag Status --- ------------------ 1 Pass 27 Pass * this terminal session

A 100Gbps transceiver (specifically, a QSFP-100G-SM-SR, which leverages single-mode duplex fiber with LC connectors; put another way, no breakout cable is used here) is inserted in Ethernet1/29, which is one of the 400Gbps uplink ports on the switch.

N9K-1# show interface Ethernet1/29
Ethernet1/29 is up admin state is up, Dedicated Interface Hardware: 10000/25000/40000/50000/100000/200000/400000 Ethernet, address: 5451.def9.fb17 (bia 5451.def9.fb98) MTU 1500 bytes, BW 100000000 Kbit , DLY 10 usec reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, medium is broadcast full-duplex, 100 Gb/s, media type is 100G Beacon is turned off Auto-Negotiation is turned on FEC mode is Auto Input flow-control is off, output flow-control is off Auto-mdix is turned off Rate mode is dedicated Switchport monitor is off EtherType is 0x8100 EEE (efficient-ethernet) : n/a admin fec state is auto, oper fec state is Rs-fec Last link flapped 00:31:44 Last clearing of "show interface" counters never 1 interface resets Load-Interval #1: 30 seconds 30 seconds input rate 0 bits/sec, 0 packets/sec 30 seconds output rate 0 bits/sec, 0 packets/sec input rate 0 bps, 0 pps; output rate 0 bps, 0 pps Load-Interval #2: 5 minute (300 seconds) 300 seconds input rate 40 bits/sec, 0 packets/sec 300 seconds output rate 40 bits/sec, 0 packets/sec input rate 40 bps, 0 pps; output rate 40 bps, 0 pps RX 0 unicast packets 34 multicast packets 0 broadcast packets 34 input packets 10812 bytes 0 jumbo packets 0 storm suppression bytes 0 runts 0 giants 0 CRC 0 no buffer 0 input error 0 short frame 0 overrun 0 underrun 0 ignored 0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop 0 input with dribble 0 input discard 0 Rx pause 0 Stomped CRC TX 0 unicast packets 34 multicast packets 0 broadcast packets 34 output packets 10812 bytes 0 jumbo packets 0 output error 0 collision 0 deferred 0 late collision 0 lost carrier 0 no carrier 0 babble 0 output discard 0 Tx pause

Here's further evidence that a QSFP-100G-SM-SR is inserted in this port.

N9K-1# show interface Ethernet1/29 transceiver
Ethernet1/29 transceiver is present type is QSFP-100G-SM-SR name is CISCO-INNOLIGHT part number is TR-FC13H-HCI revision is 1C serial number is INLREDACTED nominal bitrate is 25500 MBit/sec per channel Link length supported for 9/125um fiber is 2 km cisco id is 17 cisco extended id number is 220 cisco part number is 10-3220-02 cisco product id is QSFP-100G-SM-SR cisco version id is V02

CDP shows us that on the remote end of the switch, we have another N9K-C93600CD-GX with a remote interface of Ethernet1/29 as well.

N9K-1# show cdp neighbors interface Ethernet1/29
Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater,
                  V - VoIP-Phone, D - Remotely-Managed-Device,
                  s - Supports-STP-Dispute

Device-ID          Local Intrfce  Hldtme Capability  Platform      Port ID
N9K-2(FDOREDACTED)
                    Eth1/29        156    R S s     N9K-C93600CD-GX   Eth1/29       

Total entries displayed: 1

With respect to your second question - Generally speaking for modern Nexus switches running NX-OS, you can safely use one of the "uplink ports" as an access or downlink port without any loss in features or functionality (and vice-versa - use a downlink as an uplink). Depending on the specific model of switch and the specific use case you're working with, there may be scenarios where a feature does not work on an uplink, but does work on a downlink (or vice-versa).

To use an arbitrary example, let's say you're working with an N9K-C93180YC-FX3 running NX-OS 9.3(14). You may have a use case where you need to connect an uplink port (e.g. Ethernet1/49, a 40/100Gbps QSFP28 port) to a WAN (like a dark fiber or DWDM circuit) at 10Gbps by using a QSA (CVR-QSFP-SFP10G). In this scenario, you cannot use a downlink port because all of them are already in use elsewhere to connect downstream servers or network devices. Additionally, you must satisfy a business requirement dictating that any traffic over a WAN must be encrypted, so the use of MACSec is required in this instance.

In this hyper-specific instance, you would encounter the following documented limitation:

"Cisco Nexus 9000 Series switches do not support MACsec on any of the MACsec capable ports when QSA is being used."

Luckily, this limitation is lifted for the N9K-C93180YC-FX3 starting with NX-OS 10.1(2), so a software upgrade is all that is needed to resolve this blocker:

"Beginning with Cisco NX-OS Release 10.1(2), MACsec is supported by Cisco Nexus 9300-FX3 platform switches when QSA is being used."

However, not all limitations like this can be worked around through software; some are rooted in hardware limitations (the design of the underlying ASIC, the length of circuit traces between the ASIC and MACs/PHYs of specific ports, etc.).

In any case, this is a niche example that demonstrates why I would say that generally speaking, you will see feature and functionality parity between downlinks and uplinks. You will typically see any limitations surrounding the lack of parity documented under the Guidelines and Limitations section of a given feature's configuration guide. In your case, since you're looking for a spine switch for an EVPN fabric, the Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide is a great place to start. Note that the aforementioned guide is for the NX-OS 10.5(x) train; make sure to reference the configuration guide for the NX-OS train you plan on deploying into production!

Apologies for the long-winded explanation - networking is a complex field from bottom to top, which makes it difficult to answer simple questions with straightforward, absolute answers. In any case, I hope you find this useful!

-Christopher

View solution in original post

8 Replies 8

Dawei
Cisco Employee
Cisco Employee

Thanks, but this is a 400G switch that CAN provide 100G ports via breakout cables. See this matrix. I don't want that. I want native 100G... no breakouts. 

https://www.cisco.com/c/en/us/products/switches/nexus-9000-series-switches/nexus-9300-400-ge-switches.html

 

Thank you, but I do not need that many ports. I can't believe that Cisco only offers 1 100 gig switch. Cisco is really going downhill.

Hello!

The N9K-C93600CD-GX can provide native 100G (as well as native 40G, if needed) ports without breakout. The previously-cited matrix is indicating the absolute maximum number of 100G links the switch can bring up. The data sheet for the Nexus 9300 GX series of switches confirms that the N9K-C93600CD-GX has 28 100/40Gbps QSFP28 ports acting as downlinks, with 8 400/100Gbps QSFP-DD ports acting as uplinks. If all 8 uplinks were broken out into 4 sets of 100Gbps links, that'd result in a total of 60 100Gbps ports.

With that being said, because the uplinks are 400/100Gbps QSFP-DD ports, they do accept native 100Gbps QSFP28 transceivers that link up at 100Gbps as you desire. We can confirm this by referencing the Cisco Optics-to-Device Compatibility Matrix filtered for this product; as you can see, there are numerous 100G transceiver options for this switch, both for the downlinks and the uplinks.

You've mentioned 100Gbps capabilities thus far, but just in case you have needs for 10/25/40Gbps, I recommend reviewing the "Overview" chapter of the Cisco Nexus 93600CD-GX NX-OS Mode Switch Hardware Installation Guide, which has important information about how the "port quad" mechanism works when 10/25Gbps links speeds are needed for this particular switch (either when working with breakout or through a CVR/QSA adapter).

I hope this helps - thank you!

-Christopher

Hello, Christopher. Thank you so much for that detailed answer. You really clarified a lot of things for me. That matrix I sent you is really good, but they should make things a little clearer. Anyway, thank you so much.

I have 2 quick follow-up questions. First, just to be sure, with regard to those 400 gig ports, I can convert them to 100 gig ports simply by inserting a 100 gig qsfp28 optic correct? I do not need a breakout cable, correct? So in the end, I can have 36x 100 gig ports (natively... no breakouts) altogether. Is that correct?

Secondly, this may seem like a silly question, but, regardless of the switch model, there is no functional difference -- besides the speed -- between an uplink port and an access port, correct? So, on a 93180YC-FX3 switch that has 48x 10/25G access ports and 6x 40/100G uplink ports, I can use one of the 100G "uplink" ports to connect to a host with a 100G interface, right?

Thank you once again! 

Hello!

Yes, you can convert the 400Gbps ports to 100Gbps ports simply by inserting a 100Gbps QSFP28 transceiver/optic; no breakout cable is needed. To remove any doubt, we have a N9K-C93600CD-GX in our lab environment running NX-OS 10.5(1):

N9K-1# show module 
Mod Ports Module-Type Model Status --- ----- ------------------------------------------------ --------------------- -------- 1 36 28x100/40G + 8x400G QSFP-DD Ethernet Module N9K-C93600CD-GX ok 27 0 Virtual Supervisor Module N9K-C93600CD-GX active * Mod Sw Hw Slot --- ----------------------- ------ ---- 1 10.5(1) 3.0 NA 27 10.5(1) 3.0 VSUP Mod MAC-Address(es) Serial-Num --- -------------------------------------- ---------- 1 54-51-de-f9-fb-10 to 54-51-de-f9-fb-df FDOREDACTED 27 54-51-de-f9-fb-10 to 54-51-de-f9-fb-df FDOREDACTED Mod Online Diag Status --- ------------------ 1 Pass 27 Pass * this terminal session

A 100Gbps transceiver (specifically, a QSFP-100G-SM-SR, which leverages single-mode duplex fiber with LC connectors; put another way, no breakout cable is used here) is inserted in Ethernet1/29, which is one of the 400Gbps uplink ports on the switch.

N9K-1# show interface Ethernet1/29
Ethernet1/29 is up admin state is up, Dedicated Interface Hardware: 10000/25000/40000/50000/100000/200000/400000 Ethernet, address: 5451.def9.fb17 (bia 5451.def9.fb98) MTU 1500 bytes, BW 100000000 Kbit , DLY 10 usec reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, medium is broadcast full-duplex, 100 Gb/s, media type is 100G Beacon is turned off Auto-Negotiation is turned on FEC mode is Auto Input flow-control is off, output flow-control is off Auto-mdix is turned off Rate mode is dedicated Switchport monitor is off EtherType is 0x8100 EEE (efficient-ethernet) : n/a admin fec state is auto, oper fec state is Rs-fec Last link flapped 00:31:44 Last clearing of "show interface" counters never 1 interface resets Load-Interval #1: 30 seconds 30 seconds input rate 0 bits/sec, 0 packets/sec 30 seconds output rate 0 bits/sec, 0 packets/sec input rate 0 bps, 0 pps; output rate 0 bps, 0 pps Load-Interval #2: 5 minute (300 seconds) 300 seconds input rate 40 bits/sec, 0 packets/sec 300 seconds output rate 40 bits/sec, 0 packets/sec input rate 40 bps, 0 pps; output rate 40 bps, 0 pps RX 0 unicast packets 34 multicast packets 0 broadcast packets 34 input packets 10812 bytes 0 jumbo packets 0 storm suppression bytes 0 runts 0 giants 0 CRC 0 no buffer 0 input error 0 short frame 0 overrun 0 underrun 0 ignored 0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop 0 input with dribble 0 input discard 0 Rx pause 0 Stomped CRC TX 0 unicast packets 34 multicast packets 0 broadcast packets 34 output packets 10812 bytes 0 jumbo packets 0 output error 0 collision 0 deferred 0 late collision 0 lost carrier 0 no carrier 0 babble 0 output discard 0 Tx pause

Here's further evidence that a QSFP-100G-SM-SR is inserted in this port.

N9K-1# show interface Ethernet1/29 transceiver
Ethernet1/29 transceiver is present type is QSFP-100G-SM-SR name is CISCO-INNOLIGHT part number is TR-FC13H-HCI revision is 1C serial number is INLREDACTED nominal bitrate is 25500 MBit/sec per channel Link length supported for 9/125um fiber is 2 km cisco id is 17 cisco extended id number is 220 cisco part number is 10-3220-02 cisco product id is QSFP-100G-SM-SR cisco version id is V02

CDP shows us that on the remote end of the switch, we have another N9K-C93600CD-GX with a remote interface of Ethernet1/29 as well.

N9K-1# show cdp neighbors interface Ethernet1/29
Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater,
                  V - VoIP-Phone, D - Remotely-Managed-Device,
                  s - Supports-STP-Dispute

Device-ID          Local Intrfce  Hldtme Capability  Platform      Port ID
N9K-2(FDOREDACTED)
                    Eth1/29        156    R S s     N9K-C93600CD-GX   Eth1/29       

Total entries displayed: 1

With respect to your second question - Generally speaking for modern Nexus switches running NX-OS, you can safely use one of the "uplink ports" as an access or downlink port without any loss in features or functionality (and vice-versa - use a downlink as an uplink). Depending on the specific model of switch and the specific use case you're working with, there may be scenarios where a feature does not work on an uplink, but does work on a downlink (or vice-versa).

To use an arbitrary example, let's say you're working with an N9K-C93180YC-FX3 running NX-OS 9.3(14). You may have a use case where you need to connect an uplink port (e.g. Ethernet1/49, a 40/100Gbps QSFP28 port) to a WAN (like a dark fiber or DWDM circuit) at 10Gbps by using a QSA (CVR-QSFP-SFP10G). In this scenario, you cannot use a downlink port because all of them are already in use elsewhere to connect downstream servers or network devices. Additionally, you must satisfy a business requirement dictating that any traffic over a WAN must be encrypted, so the use of MACSec is required in this instance.

In this hyper-specific instance, you would encounter the following documented limitation:

"Cisco Nexus 9000 Series switches do not support MACsec on any of the MACsec capable ports when QSA is being used."

Luckily, this limitation is lifted for the N9K-C93180YC-FX3 starting with NX-OS 10.1(2), so a software upgrade is all that is needed to resolve this blocker:

"Beginning with Cisco NX-OS Release 10.1(2), MACsec is supported by Cisco Nexus 9300-FX3 platform switches when QSA is being used."

However, not all limitations like this can be worked around through software; some are rooted in hardware limitations (the design of the underlying ASIC, the length of circuit traces between the ASIC and MACs/PHYs of specific ports, etc.).

In any case, this is a niche example that demonstrates why I would say that generally speaking, you will see feature and functionality parity between downlinks and uplinks. You will typically see any limitations surrounding the lack of parity documented under the Guidelines and Limitations section of a given feature's configuration guide. In your case, since you're looking for a spine switch for an EVPN fabric, the Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide is a great place to start. Note that the aforementioned guide is for the NX-OS 10.5(x) train; make sure to reference the configuration guide for the NX-OS train you plan on deploying into production!

Apologies for the long-winded explanation - networking is a complex field from bottom to top, which makes it difficult to answer simple questions with straightforward, absolute answers. In any case, I hope you find this useful!

-Christopher

ex-engineer
Level 1
Level 1

Christopher, you are the MAN! Thank you so much for such thorough and helpful answers! THIS is what Cisco used to be like years ago; smart people who went out of their way to help you! Sorry to say, now, not so much. But you are indeed a rock star! Thank you again, buddy. 

Review Cisco Networking for a $25 gift card