Cisco 9500 48Y4C stackwise virtual uplinks down

[velislavatopuzova]

Hello team,

I have issues with new deployment of stackwise virtual on C950048Y4C. We are using Hu1/0/49 and Hu1/0/51 for stackwise virtual.

Here is the configuration of the VSS:

Master:
On Switch 1:
Device>enable
Switch#switch priority 15
Device#configure terminal
Switch(config)#stackwise-virtual
Switch(config-stackwise-virtual)# domain 101
Switch(config-stackwise-virtual)# exit
Device(config)#interface range hu1/0/49,hu1/0/51 

Device(config-if)#stackwise-virtual link 1
Device(config-if)#end
Device#write memory
Device#reload

On Switch 2:
Device>enable
Switch#switch renumber 2
Device#configure terminal
Switch(config)#stackwise-virtual
Switch(config-stackwise-virtual)# domain 101
Switch(config-stackwise-virtual)# exit
Device(config)#interface range hu1/0/49,hu1/0/51
Device(config-if)#stackwise-virtual link 1
Device(config-if)#end
Device#write memory
Device#reload

Also we are using these QSFP's and DAC cables: 

3 х QSFP-100G-AOC2M=

14 х QSFP-100G-FR-S

They are supported for this model.

The software version is: 17.6.5 and both switches are licensed Network Advantage.

The QSFP's are showing up, but not connected. 

17.1

Twe1/0/48 notconnect 4094 auto auto unknown
Hu1/0/49 notconnect 4094 full 100G QSFP 100G FR-S
Hu1/0/50 notconnect 1 full 100G QSFP 100G FR-S
Hu1/0/51 notconnect 4094 full 100G QSFP 100G FR-S
Hu1/0/52 connected 1 full 100G QSFP 100G AOC2M

17.3

Twe2/0/48 notconnect 4094 auto auto unknown
Hu2/0/49 notconnect 4094 full 100G QSFP 100G FR-S
Hu2/0/50 notconnect 1 full 100G QSFP 100G FR-S
Hu2/0/51 notconnect 4094 full 100G QSFP 100G FR-S
Hu2/0/52 connected 1 full 100G QSFP 100G AOC2M

The optical receiver power stays -40.0:

#show interface HundredGigE1/0/49 transceiver detail
ITU Channel not available (Wavelength not available),
Transceiver is internally calibrated.
mA: milliamperes, dBm: decibels (milliwatts), NA or N/A: not applicable.
++ : high alarm, + : high warning, - : low warning, -- : low alarm.
A2D readouts (if they differ), are reported in parentheses.
The threshold values are calibrated.

High Alarm High Warn Low Warn Low Alarm
Temperature Threshold Threshold Threshold Threshold
Port (Celsius) (Celsius) (Celsius) (Celsius) (Celsius)
--------- ----------------- ---------- --------- --------- ---------
Hu1/0/49 38.3 75.0 70.0 0.0 -5.0

High Alarm High Warn Low Warn Low Alarm
Voltage Threshold Threshold Threshold Threshold
Port (Volts) (Volts) (Volts) (Volts) (Volts)
--------- ----------------- ---------- --------- --------- ---------
Hu1/0/49 3.23 3.63 3.46 3.13 2.97

High Alarm High Warn Low Warn Low Alarm
Current Threshold Threshold Threshold Threshold
Port Lane (milliamperes) (mA) (mA) (mA) (mA)
--------- ---- --------------- ---------- --------- --------- ---------
Hu1/0/49 1 70.1 120.0 110.0 30.0 20.0

Optical High Alarm High Warn Low Warn Low Alarm
Transmit Power Threshold Threshold Threshold Threshold
Port Lane (dBm) (dBm) (dBm) (dBm) (dBm)
--------- ---- --------------- ---------- --------- --------- ---------
Hu1/0/49 1 2.2 5.0 4.0 -3.4 -4.4

Optical High Alarm High Warn Low Warn Low Alarm
Receive Power Threshold Threshold Threshold Threshold
Port Lane (dBm) (dBm) (dBm) (dBm) (dBm)
--------- ---- --------------- ---------- --------- --------- ---------
Hu1/0/49 1 -40.0 5.5 4.5 -7.4 -8.0x

They are physically connected and I can't understand from where might be the issue. Thank you in advance for your help.

[marce1000]

 - Not sure of the intended port(s) are supported for stackwise virtual ; ref https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9000/nb-06-cat-9k-stack-wp-cte-en.html
                            Check Table 2

 M.

[velislavatopuzova]

- Any port (Downlink: 10G/25G; Fixed Uplink: 40G/100G). We are using 100G Qsfp's. I think the uplink supports 100G.

[MHM Cisco World]

Change cable' also check the max length of cable using for this QSFP.

Also as @marce1000  mention there is restrictions of which port can use for svl

From cisco doc.

You can establish SVLs using 40G or 10G Ethernet connections on Cisco Catalyst 9500 Series Switches and 100G, 40G, 25G and 10G Ethernet connections on Cisco Catalyst 9500 Series High Performance Switches.

[velislavatopuzova]

QSFP-100G-FR-S

100GBASE FR QSFP Transceiver, 2km over SMF

[velislavatopuzova]

This model is high performance switch so I think we can use 100G.

Thank you!

[Richard Pidcock]

Checking the ref guide provided by @marce1000 above, this document confirms that the 100G uplink ports can be used for the VSS paring.

[Richard Pidcock]

I don't see in your config that you have an interface configured for DAD (Dual active detection).  That might be what you're needing.  You need an interface on each switch (i.e. 1/0/1 <-- --> 1/0/1) connected to each other for dual active detection.

!
interface wentyFiveGigEx/x/xx
stackwise-virtual dual-active-detection
description VSS_KeepAlive
!

[velislavatopuzova]

I tried to configure it but also without any uplink. Also I think the Dual Active Detection is not mandatory. I will try again and update you with the results.

[Turn2]

Doesn't seem like it's a stackwise virtual issue at this point as the links are not even up. More of a physical issue -  have you tried reversing the fibre strands at one end? Maybe try connecting port 50 at either end without any config and see if that gets a link to eliminate configuration issues. 

[velislavatopuzova]

Yes, I also tested that but the uplink is down.

[Turn2]

If swapping the transmit/receive at one end didn't make a difference, and you tried moving the QSFP's to a different port with no config still didn't get a link, perhaps it's a bad fiber, or transceiver? Try looping the connection at both ends (run a fiber from transmit to receive) and see if you get a link light, to potentially eliminate the QSFP as the problem. Also, anything of interest show up on the interface (show interface hu2/0/49), or in the logs?

[MHM Cisco World]

After check all restrictions I found that

You can not use breakout interface (QSFP) for SVL and DAD in any c9500.

Use other interface 

MHM

[MHM Cisco World]

• SVL and DAD links are not supported on breakout interfaces when operating in SVL mode. Breakout interfaces are only supported for regular data and control traffic carrying ports (not SVL and DAD links) when operating in SVL mode.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9500/software/release/17-11/configuration_guide/ha/b_1711_ha_9500_cg/configuring_cisco_stackwise_virtual.html