C220 M5 wont connect to C3132Q 40Gb with 1387 VIC

[swetie42]

Hello everyone, I have two C220 M5's with 1387 VIC cards. I can not for the life of me get these servers to connect to the switch at 40Gb and don't know if I'm missing anything. I can get them to connect to the switch with a QSA and 10Gb adapter. I also can get my dell to connect at 40Gb. I fell I've narrowed it down to the server but don't know what config I might be missing. Anyone have any advice or clue?

[marce1000]

 - Try playing with speed and duplex settings at the switch side  , use the number of 'allowed combinations so to speak...' , but always disconnect the adapter and connect again when trying with changed settings , also always check switch logs for all attempts and or check if anything is reported , 

 M.

[Wes Austin]

What optics are you using on the VIC adapter side? You may need to adjust the link training settings.

Can you post some screen captures of your VIC config settings in CIMC?

[ngoldstein]

Here are the settings on the VIC.

[Wes Austin]

What optics are you using on the VIC? If you connect the VIC ports back to back do you get link status up? (link light)

[ngoldstein]

I am using the QSFP-H40G-CU1M. If i connect the VIC port from C220-01 to C220-02 then I still get no link. Only when I use a QSA do I get a link after setting the speed to 4x10Gb.

[Steven Tardy]

See CDET CSCvq21079 :: UCS C-series integration with FI fails when 40Gb VIC cards with Link training disabled

Workaround:
Configure server's CIMC in standalone mode via [F8], log into CIMC web gui. Modify VIC card link training to 'enabled'.
  CIMC / Navigation (top left) / Networking / MLOM / External Ethernet Interfaces / Port-0
Set:
  Admin Speed = 40G
  Link Training = ON
  [Save]
"Link State" should change from "Link Down" to "Link Up".
Repeat the same for Port-1.

[ngoldstein]

All of that is set and its still not able to link.

[trtai]

I'm working on this too. The weird part is I had a link at 40Gbs for a moment on one of my 3 hosts with a QSFP-H40G-CU1M but lost it after a reboot and ever since I have not been able to get it back on any of my hosts. QSA with configured to 4x10Gbs comes up without any config changes. On my MLOM I'm not able to toggle any external ethernet settings in the GUI at all, and CIMC via ssh only allows me to adjust the admin-speed, admin-FEC-mode is also an option but attempting to set it to any of the available values result in a message that admin-fec-mode is not configurable on this adapter.

[dsenela]

Facing exact issue with C240 M5 with 1387 VIC. Tried all suggestions here and others from net, simply not able to connect. Showing "link down". Anyone else has any suggestions? I am almost at the point of pullling this MLOM off the unit and installing a Mellanox ConnectX 40GB PCI card as replacement.

[mrcr-ckynet]

Im having the same issue on a C220 M4 with a VIC 1387. What is odd is that the card nor CIMC show a link but my switch does with lots of receive errors on the port. Changing the admin mode on physical Ethernet between 4x10/40 do nothing. I can’t say I’ve ever seen a unidirectional link with a DAC. I tried other cables as well. My other Mellanox cards work fine so it’s surely not a cable issue. I tried reverting firmware to turn off link training but that didn’t seem to help either. 

[Murthi]

We are facing same issue , all our c480 m5 ports suddenly  stopped working all my c480 got disconnected from my FI, if anyone have a solution to this issue please let us now