25GB Connectivity issue between S3260 server and Nexus switches

[chawkideeb]

Dear Support,

I have a s3260 server (UCSS-S3260) with a network card (UCSC-PCIE-C25Q-04 = Cisco UCS VIC 1455 Quad Port 10/25G SFP28 CNA PCIE) that is installed on the SIOC (UCS-S3260-PCISIOC) card in the server.

This server is connected to a nexus switch (N9K-C93180YC-EX= Nexus 9300 with 48p 10/25G SFP+ and 6p 100G QSFP28) with a cable of 25GB (SFP-H25G-CU3M=  25GBASE-CU SFP28 Cable 3 Meter).

The os on the server is windows 2019 standard.

Mu issue is that the ports are not coming UP on the server . However the server can see the card and if I put a link of 10 GB it works properly.

I did firmware Upgrade for the server and Bios upgrade and I updates the driver ,etc… but still facing the same issue.

Your support is highly appreciated in order to let the connectivity between the server and the nexus switches become UP.

[Kirk J · Accepted solution]

Greetings.

Probably has to do with FEC mode negotiation (or the lack of).

Try changing the FEC mode on the VIC adapter to FC-FEC (cl74) and confirm the N9k port matches.

Kirk...

[sankbhat · Accepted solution]

Hi,

May be issue with Forward Error Correction (FEC) Mode.

The "Auto" FEC mode on the VIC 1400 adapters defaults to cl91 (RS-FEC) instead of actually negotiating with the switch.

Consider making below changes in adapter on server side.

Server# scope chassis

Server /chassis # scope adapter 1

Server /chassis/adapter # scope ext-eth-if 1

Server /chassis/adapter/ext-eth-if # set admin-fec-mode cl74

Server /chassis/adapter/ext-eth-if* # commit

Changes to the network settings will be applied immediately.

You may lose connectivity to the Cisco IMC and may have to log in again.

Do you wish to continue? [y/N] y