NPIV/NPV in UCS

[nicksousa]

There are two different configuration options that may or may not complement one another – NPIV enabled VMs within ESXi, and End Host Mode for the FIs. What types of storage ports are they when not using NPIV for VMs?

Please correct the attached diagram. This is for FIs using the default End Host Mode.

[Wes Austin]

This sounds like an exam question.......?

[Walter Dey]

The NPV switch above is the UCS FI, the blades vhba are N-Ports, connecting to a F-Port of the UCS FI. The MDS has to be in NPIV mode !

[nicksousa]

Thanks for the response.

How are the ports configured with Top of Rack switches in between the FI and the Core - NP Ports? Are the ToR switches still considered NPV edge switches, or are they just logically inseparable from the core?

Also how does VMwares implementation of NPIV for VMs play into this? Do the port types change for the ToR/Core switches?

[Walter Dey]

Have a look at the following URL's: I am not so sure, if NPIV is supported on UCS HBA ? at lease for a long time it wasn't:

https://supportforums.cisco.com/t5/unified-computing-system/npiv-with-vic-1240-and-esx/td-p/2688161
https://communities-gbot.vmware.com/thread/520930

https://blogs.vmware.com/vsphere/2011/11/npiv-n-port-id-virtualization.html

https://pubs.vmware.com/vsphere-50/index.jsp#com.vmware.vsphere.vm_admin.doc_50/GUID-C713BCA5-71B4-4539-A4AE-8E781330755C.html?resultof=%2522%256e%2570%2569%2576%2522%2520

And chaining of 2 NPV is not supported: NPV (UCS FC End Host Mode)  -- NPV -- NPIV

[Walter Dey]

https://ucshcltool.cloudapps.cisco.com/public/

Firmware Version 3.2(3) Notes

Please see the list of results for relevant items referring to each note.

• 10 : Virtual Volume (vVol) is supported, minimum FNIC version is 1.6.0.33. Please check with vmware.com and the Storage Array Vendor for details
• 11 : A Cisco OEM Customized ESX ISO maybe required for installation. Custom ISOs contain the minimum driver level for installation, please check the Cisco HCL for the supported driver versions. Custom ISOs are available from VMware.com > Downloads > VMware vSphere > Drivers and Tools > Custom ISOs
• 12 : iSCSI Boot is supported. Both Appliance and Uplink ports are supported unless otherwise noted
• 20 : Minimum FW is 4.2.1. VmWare's HCL may not always be in sync with Cisco's HCL. Please refer to VmWare's KB 2030818
• 21 : Drivers can be found on the latest 3.2(3) B-Series Driver ISO
• 31 : ESX NPIV Host Feature is not supported

[nicksousa]

On the attachment I included in the original post, I had a question mark next to the interface from the NPIV switch to the ToR (or FI). Is this a TNP port?

[Walter Dey]

This design doesn't make sense at all; your NetApp is connected to 2 different MDS ! perfect, it's a physical dual fabric; then you merge everything to a single ToR, which is a single point of failure.

Anyway, your ToR must be NPIV, and the ports between MDS and ToR are E-Port's.

TF-TNP link between an F port in the NPIV core switch (ToR in your case) , and an NP port in the NPV switch (UCS FI in your case):

Enable the F port trunking and channeling protocol on the MDS core switch:

switch(config)# feature fport-channel-trunk

[nicksousa]

Thanks for continuing forward with the discussion Walter. I still don't fully understand the basics of the port types. Could you label the attached diagram?

[Walter Dey]

Hi

I mentioned several times, that your FC design is simply not correct and state of the art.

Take the time and go through this document, and you hopefully understand what I mean;

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/flexpod_esxi65u1_n9fc.html

 

[nicksousa]

That is one design. For a large datacenter not everything will run directly from the FI to the core. The original diagram was logical, not physical - I added another ToR to the latest diagram regardless.

Other than FlexPod architectures, do you have any other designs to share? The last place I worked had hundreds of terabytes of consolidated storage with multiple SAN switches. They had a FlexPod as well, but that wasn't the primary design.

[Walter Dey]

Here a Whitepaper for large SAN design

https://www.cisco.com/c/en/us/products/collateral/storage-networking/mds-9700-series-multilayer-directors/white-paper-c11-729697.html

[claudedube]

Hi
The NPIV/NPV in UCS blade is not properly configured, the UCS blades are B200 M5 series
The SAN storage is provided through MDS9706 fabric and the NPIV protocol is able on MDS fabrics and working for Oracle virtual machine and AIX/vios environments

The storage is provided through MDS9706 fabric, connected through UCS FI, and the UCS blades are booted on SAN and installed on Red Hat virtualization version 7.7 and it's properly working.

However, the 4 provided HBA are not virtualization capable, so how these 4 HBA ports can be enabled for supporting virtulization capable.

Thanks, Claude

virsh nodedev-list --cap scsi_host
scsi_host0
scsi_host1
scsi_host2
scsi_host3

#
virsh nodedev-list --cap vports
< empty>

Discovery

Discovery of HBA(s) capable of NPIV is provided through the virsh command "virsh nodedev-list --cap vports"
which will return a list of scsi_host's capable of generating a vHBA.
These scsi_host's will be described as the parent. If no HBA is returned, then the host configuration should be checked.

It does work on RedHat powerPC linux, it looks like there is a setting missing on UCS bios.

https://wiki.libvirt.org/page/NPIV_in_libvirt

https://www.ibm.com/developerworks/linux/library/l-npiv-kvm/l-npiv-kvm-pdf.pdf

[nicksousa]

No it's not. It's my relentless pursuit of knowledge.