Cisco 93128TX Transceiver issues (40G to 10G)

[Mary Ghabrai]

For Cisco 93128TX switches, we bought CVR-QSFP-SFP10G adapters and SFP-10G-SR (FS brand)
on the vendor's page, it says that the adapter supports SFP-10G-SR. but I couldn't get the interface to come up. below is a summary of the issue and what's been set up. Has anyone encountered this issue? 

- I have enabled "service unsupported-transciever"

- on the 40G ports, I have configured "speed-group 10000" according to this forum

- when I do a show interface, I have this error: "Transceiver speed does not match the speed configured on the port"

- configuration on the port :

interface Ethernet2/1
speed-group 10000
no shutdown

[Sergiu.Daniluk]

Hi @Mary Ghabrai 

Can you share the following outputs from your device, while the QSA + SFP are inserted:

show module

show run int eth2/1 all

show interface eth2/1

show interface eth2/1 status

show interface eth2/1 capabilities

show interface eth2/1 transceiver details

Note: Make sure you remove the SNs from the outputs, to avoid disclosing sensitive data about your devices.

Cheers,

Sergiu

[Mary Ghabrai]

So I upgraded the switches from 7.0(3)I7(8) to version 9.3(4) and the error went away but the links won't come up. show command results below:

ELG-COR-0002# show module
Mod Ports Module-Type Model Status
--- ----- ------------------------------------- --------------------- ---------
1 96 1/10G-T Ethernet Module N9K-C93128TX active *
2 8 40G Ethernet Expansion Module N9K-M12PQ ok

Mod Sw Hw Slot
--- ----------------------- ------ ----
1 9.3(4) 2.0 NA
2 NA 1.1 GEM

Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 50-87-89-a2-3c-78 to 50-87-89-a2-3c-df SAL********
2 88-f0-31-bd-41-be to 88-f0-31-bd-41-c9 SAL********

Mod Online Diag Status
--- ------------------
1 Pass
2 Pass
=======================================================================================================

ELG-COR-0002# show run int eth2/1 all

!Command: show running-config interface Ethernet2/1 all
!Running configuration last done at: Thu Apr 30 16:08:56 2020
!Time: Thu Apr 30 18:57:47 2020

version 9.3(4) Bios:version 07.67

interface Ethernet2/1
no description
lacp port-priority 32768
lacp rate normal
cdp enable
switchport
switchport mode access
no switchport monitor
no switchport dot1q ethertype
no switchport priority extend
switchport access vlan 1
switchport trunk native vlan 1
no switchport vlan mapping enable
priority-flow-control mode auto
priority-flow-control watch-dog-interval off
spanning-tree port-priority 128
spanning-tree cost auto
spanning-tree link-type auto
no spanning-tree bpduguard
no spanning-tree bpdufilter
flowcontrol receive off
flowcontrol send off
delay 1
snmp trap link-status
mtu 1500
link debounce time 100
no beacon
no link transmit reset-skip
link mac-up timer 0
speed-group 10000
speed auto
duplex auto
fec auto
link dfe adaptive-tuning
negotiate auto
logging event port link-status default
logging event port trunk-status default
storm-control broadcast level 100.00
storm-control multicast level 100.00
storm-control unicast level 100.00
no storm-control action
bandwidth 40000000
no bandwidth inherit
load-interval counter 1 30
load-interval counter 2 300
no load-interval counter 3
no switchport block multicast
no switchport block unicast
no switchport mac-learn disable
no shutdown lan
no udld disable
no switchport autostate exclude
no shutdown
=======================================================================================================

ELG-COR-0002# show interface eth2/1
Ethernet2/1 is down (Link not connected)
admin state is up, Dedicated Interface
Hardware: 10000/40000 Ethernet, address: 88f0.31bd.41be (bia 88f0.31bd.41be)
MTU 1500 bytes, BW 40000000 Kbit , DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is access
auto-duplex, auto-speed, media type is 40g
Beacon is turned off
Auto-Negotiation is turned on FEC mode is Auto
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
EEE (efficient-ethernet) : n/a
admin fec state is auto, oper fec state is off
Last link flapped never
Last clearing of "show interface" counters never
0 interface resets
Load-Interval #1: 30 seconds
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
Load-Interval #2: 5 minute (300 seconds)
300 seconds input rate 0 bits/sec, 0 packets/sec
300 seconds output rate 0 bits/sec, 0 packets/sec
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
RX
0 unicast packets 0 multicast packets 0 broadcast packets
0 input packets 0 bytes
0 jumbo packets 0 storm suppression packets
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 0 multicast packets 0 broadcast packets
0 output packets 0 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
=======================================================================================================

ELG-COR-0002# show interface eth2/1 status

--------------------------------------------------------------------------------
Port Name Status Vlan Duplex Speed Type
--------------------------------------------------------------------------------
Eth2/1 -- notconnec 1 auto auto --

=======================================================================================================

ELG-COR-0002# show interface eth2/1 capabilities
Ethernet2/1
Model: N9K-C93128TX
Type (SFP capable): unknown
Speed: 10000,40000
Duplex: full
Trunk encap. type: 802.1Q
Channel: yes
Broadcast suppression: percentage(0-100)
Flowcontrol: rx-(off/on),tx-(off/on)
Rate mode: dedicated
Port mode: Routed,Switched
QOS scheduling: rx-(none),tx-(4q)
CoS rewrite: yes
ToS rewrite: yes
SPAN: yes
UDLD: yes
MDIX: no
TDR capable: no
Link Debounce: yes
Link Debounce Time: yes
FEX Fabric: no
dot1Q-tunnel mode: no
Pvlan Trunk capable: yes
Port Group Members: 97-102
EEE (efficient-eth): no
PFC capable: yes
Buffer Boost capable: no
Speed group capable: yes
Breakout capable: no
MACSEC capable: no

=======================================================================================================

ELG-COR-0002# show interface eth2/1 transceiver details
Ethernet2/1
transceiver is not supported

SFP Detail Diagnostics Information (internal calibration)
----------------------------------------------------------------------------
Current Alarms Warnings
Measurement High Low High Low
----------------------------------------------------------------------------
Temperature N/A N/A N/A N/A N/A
Voltage N/A N/A N/A N/A N/A
Current N/A N/A N/A N/A N/A
Tx Power N/A N/A N/A N/A N/A
Rx Power N/A N/A N/A N/A N/A
Transmit Fault Count = 0
----------------------------------------------------------------------------
Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning

*** This QSFP support partial diagnostic data! ***

SFP Detail Diagnostics Information (internal calibration)
----------------------------------------------------------------------------
Current Alarms Warnings
Measurement High Low High Low
----------------------------------------------------------------------------
Temperature N/A N/A N/A N/A N/A
Voltage N/A N/A N/A N/A N/A
Current N/A N/A N/A N/A N/A
Tx Power N/A N/A N/A N/A N/A
Rx Power N/A N/A N/A N/A N/A
Transmit Fault Count = 0
----------------------------------------------------------------------------
Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning

*** This QSFP support partial diagnostic data! ***

SFP Detail Diagnostics Information (internal calibration)
----------------------------------------------------------------------------
Current Alarms Warnings
Measurement High Low High Low
----------------------------------------------------------------------------
Temperature N/A N/A N/A N/A N/A
Voltage N/A N/A N/A N/A N/A
Current N/A N/A N/A N/A N/A
Tx Power N/A N/A N/A N/A N/A
Rx Power N/A N/A N/A N/A N/A
Transmit Fault Count = 0
----------------------------------------------------------------------------
Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning

*** This QSFP support partial diagnostic data! ***

SFP Detail Diagnostics Information (internal calibration)
----------------------------------------------------------------------------
Current Alarms Warnings
Measurement High Low High Low
----------------------------------------------------------------------------
Temperature N/A N/A N/A N/A N/A
Voltage N/A N/A N/A N/A N/A
Current N/A N/A N/A N/A N/A
Tx Power N/A N/A N/A N/A N/A
Rx Power N/A N/A N/A N/A N/A
Transmit Fault Count = 0
----------------------------------------------------------------------------
Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning

*** This QSFP support partial diagnostic data! ***

[Sergiu.Daniluk]

Hello @Mary Ghabrai 

Have you tried configuring "speed 10000" on the interface?  It looks like your interface is still in auto (40G).

Cheers,

Sergiu

[Mary Ghabrai]

yes I've tried that. below is the error I get: (I've also tried to remove the auto-negotiation but I get this error: "ERROR: cannot turn off auto-negotiate with speed auto")

I

[paul driver]

Hello Mary
As stated previously the M6PQ module isn't 10gb supported according to cisco transceiver matrix, So can you confirm what module you have procured?  M6PQ_E does support 10GB

Try defaulting the interface on both sides of the interconnect and then disable negotiation and specify the speed/duplex 
default interface x/x
speed 10000
no negotiate auto  

[Mary Ghabrai]

we have M12PQ

[Sergiu.Daniluk]

Hi @Mary Ghabrai 

So, based on the transceiver matrix, the N9K-M12PQ supports CVR+ SFP-10G.

Are the CVR and SFP you are using both cisco SFPs?

Have you tried with other SFP/CVR? What is the result?

Have you tried the current CVR/SFP in other ports? What is the result?

Have you tried the current CVR/SFP in other switch? What is the result?

Regards,

Sergiu

[Mary Ghabrai]

Hi Sergiu,
- Are the CVR and SFP you are using both cisco SFPs? No, they're the FX brand (unfortunatley I'm remote and didn't have access and budget to test a Cisco optic

- Have you tried with other SFP/CVR? What is the result? yes the current optics are the FS brand and it doesn't work

- Have you tried the current CVR/SFP in other ports? What is the result? yes, none work

- Have you tried the current CVR/SFP in other switch? What is the result? yes, I'm testing this on 4 switches and they all have the same issue

[Sergiu.Daniluk]

Hello,

Yep. Kinda looks like SFPs to be the problem here. Have you enabled "service unsupported-transceiver"? This might help. However, important note: if you experience an issue with a non-Cisco SFP, and the fault or defect can be traced to the use of third-party transceivers, then the Cisco support team will simply tell you to use Cisco SFP.

Cheers,

Sergiu

[paul driver]

Hello

---

@Mary Ghabrai wrote:

For Cisco 93128TX switches, we bought CVR-QSFP-SFP10G adapters and SFP-10G-SR (FS brand)
on the vendor's page, it says that the adapter supports SFP-10G-SR.

---

I cannot see that M6PQ module for10gb being supported for that switch on the capabiltiy matrix? - here

[Sergiu.Daniluk]

Hi @paul driver 

Nexus 93128TX is using M6PQ, M6PQ-E, or M12PQ modules to have 40G ports. So in the matrix you need to look at the modules and not at the Nexus9000 as a platform:  https://tmgmatrix.cisco.com/?tpid=73&npid=137&npid=139 

As you can see, it is supported with a note:

Cheers,

Sergiu