Configure FC-NVMe Initiator on C240M5 with VIC1455

[subhish_p]

Hi All,

My setup is as follows --

2x 6454 FI's in a cluster.

C240M5 servers with VIC1455 direct-connected to the FI for UCSM integration.

FI is connected via FC uplinks to a Brocade FC switch

I want to create an FC-NVMe initiator on the server and be able to run end-to-end NVMe to a NVMe storage target connected to the FC switch fabric.

From what I understand the hardware supports this, but I am not able to find any information or configuration step for supported OS and/or drivers for the Cisco VIC 1455.

I have tried with Centos 8.3 and SLES 15.2 running on the servers, and created vHBA with adapter policy set to "FCNVMeInitiator". But with this the vHBA login to the switch is of regular FC type. ("FC4 Features [FCP]: Initiator")

Does anybody have experience with this type of setup?

What OS supports FC-NVMe on Cisco VIC 1455 and does it require any specific drivers (fnic) or special configuration on the host OS?

Any help would be greatly appreciated.

Thanks!

[Wes Austin]

What is your UCSM, Server and VIC firmware version?

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/notes/CiscoUCSManager-RN-4-1.html 

New Software Features in Release 4.1(3a)

Software Enablement for New Hardware (Listed in the New hardware section)

Feature Enhancements

<snip>

Support for NVMe over Fibre Channel (FC-NVMe) on UCS 6300 series Fabric Interconnects, UCS 6454, and UCS 64108 Fabric Interconnects with Cisco UCS VIC 13xx series adapters on RHEL 7.8, RHEL 7.9, and RHEL 8.2. This support is also available on Cisco C220 and C240 M5 Standalone rack servers with Cisco UCS 13xx series adapters.

Support for Fibre Channel and NVMe over Fibre Channel (FC-NVME)

Support for NVMe over Fibre Channel (FC-NVMe) on SLES 12 SP4, SLES 12 SP5, SLES 15, SLES 15 SP1 and RHEL 7.6. Unified driver support for Fibre Channel and NVMe over Fibre Channel (FC-NVME) on SLES 12 SP4, SLES 15, and RHEL 7.6. This support is available on UCS 6300 Series Fabric Interconnects and UCS 6454 Fabric Interconnects with Cisco UCS 14xx Series adapters. NVMe over Fibre Channel now supports up to 16 interfaces.

[subhish_p]

UCSM/server/VIC = 4.1.3b

I believe the fnic driver on SLES 15SP2 is 1.6.x

[subhish_p · Accepted solution]

Needed to install the VIC out-of-box drivers for the OS from the 4.1.3b package.

Ended up with RHEL 8.2 OS running fnic version: 2.0.0.69-178.0 and the NVMe initiators successfully logged in.

Thanks!