Showing results for 
Search instead for 
Did you mean: 

Nexus VMware VM-FEX Video Configuration Guide



A virtualized server implementation consists of one or more VMs running as 'guests' on a single physical server. The guest VMs are hosted and managed by a software layer called the hypervisor or virtual machine manager (VMM). The hypervisor typically presents a virtual network interface to each VM and performs Layer 2 switching of traffic from a VM to other local VMs or to a physical interface to the external network. Working with a Cisco virtual interface card (VIC) adapter, Cisco Virtual Machine Fabric Extender (VM-FEX)bypasses software-based switching of VM traffic by the hypervisor in favor of external hardware-based switching in the fabric interconnect. This method results in a reduced load on the server CPU, faster switching,and the ability to apply a rich set of network management features to local and remote traffic. VM-FEX extends the (prestandard) IEEE 802.1Qbh port extender architecture to the VMs, providing each VM interface with a virtual Peripheral Component Interconnect Express (PCIe) device and a virtual port ona switch. This solution allows precise rate limiting and quality of service (QoS) guarantees on the VM interface.

Infrastructure Requirements

  • Nexus VMware VM-FEX is available with Nexus 5500 Series and UCS standalone C series with VIC (P81E,VIC1225, VIC1225T)
  • Each VIC card supports up to 96 Virtual Machine Interface
  • Enterprise Plus License required (as is for any DVS) on Host
  • Standard License and above is required for vCenter
  • Hypervisor features are supported for both emulated and hypervisor bypass mode

     – vMotion, HA, DRS, Snapshot and Hot add/remove virtual device, Suspend/Resume

  • VMDirectPath (Hypervisor Bypass) with VM-FEX is supportted with ESXi5.0+
  • VM-FEX upgrade is supportted from ESXi4.X to ESXi5.X with Customized ISO and VMware Update Manager

VM-FEX Mode of Operations




Hypervisor Bypass

Hypervisor Version

ESX 4.0 U1 +

ESXi 5.0 +

Minmum NX-OS Version

Minimum VIC Firmware Version

P81E:         1.6(2a)

VIC1225:     HUU 1.5.1(b)

VIC1225T:   HUU 1.5.1(b)

P81E:             1.6(2a)

VIC1225:         HUU 1.5.1(b)

VIC1225T:       HUU 1.5.1(b)

Minimum CIMC Version

P81E:         1.4(2.3)

VIC1225:     HUU 1.5.1(b)

VIC1225T:   HUU 1.5.1(b)

P81E:             1.4(2.3)

VIC1225:         HUU 1.5.1(b)

VIC1225T:       HUU 1.5.1(b)

VMotion / Live Migration

Support Guest OS Any OS support by ESX/ ESXi

RHEL 6.0+ , SLES 11+

Window 2008 SP2, Windows 2008 R2, Windows 2012

Guest OS/vSphere Version
vSphere 5.0
vSphere 5.0 U1
vSphere 5.1
Windows 2012N/A1.2.24.0 (vmwaretools) (vmwaretools)
Windows 2008 R21.2.22.0 (vmwaretools) (vmwaretools) (vmwaretools)
Windows 2008 SP21.2.22.0 (vmwaretools) (vmwaretools) (vmwaretools)
RHEL (inbox) (inbox) (inbox)
RHEL (inbox) (inbox) (inbox)
RHEL 6.2N/A (inbox) (vmwaretools) (inbox) (vmwaretools)

SLES111.0.36.0 (vmwaretools) (vmwaretools) (vmwaretools)
SLES11 SP11.0.36.0 (vmwaretools) (vmwaretools) (vmwaretools)

VM-FEX Scalability

UCSM Configuration Limit 2.1 Release -

Cisco UCS 6100 / 6200 Series


Half-Width Blade with Single VIC

Full-Width Blade with Dual VIC

ESX 4.0 – 4.1 (DirectPath I/O)

56 (54 vNIC + 2 vHBA)

56 (54 vNIC + 2 vHBA)

ESXi 5.0 – 5.1 (DirectPath I/O)

116 (114 vNIC + 2 vHBA)

116 (114 vNIC + 2 vHBA)*

Windows 2012 (SR-IOV)

116 (114 vNIC + 2 vHBA)

232 (228 vNIC + 4 vHBA)

KVM 6.1 – 6.3 (SR-IOV)

116 (114 vNIC + 2 vHBA)

232 (228 vNIC + 4 vHBA)

* Additional VIC will NOT increase the total VIF count due to OS limitation

* Multiple VICs are Supported for full width blade and B200M3

Nexus 5500  Series
HypervisorAdapter FEXVM-FEX

ESX 4.1 – ESXi 5.1

96 vNIC

96 vNIC

* Only one VIC (P81E/VIC1225) is Supported for each C series rack server

Nexus Verified Scalability 6.0(2)N1(1) release

Verified Topology
Verified Maximum
Number of VFCs over Virtual Ethernet Interfaces40 40
Number of Port Profiles1,0001,000
Number of Virtual Machines (VMs) Concurrently not VMotioned5 VMs with 10 vNICs each5 VMs with 10 vNICs each
Number of Virtual Ethernet Interfaces Enabled with vNIC Shaping2,0002,000
Number of Virtual Ethernet Interfaces Enabled with Untagged CoS2,0002,000
Server - Number of Adapters per Server11
Server - Number of vNICs per Server5096

VM-FEX Feature Comparision



Adapter FEX






Port Security YesYesYes
IGMP Snooping YesYes
Qos Shaping YesYesYes
QoS Classification Yes YesYes
PVLANYes YesIsloate
L2 Multipath Edge Port Edge Port
TrustSec Yes Yes
DHCP Snooping YesYes
IP Source Guard YesYes


Reference Documentation

Nexus VM-FEX Configuration Guide

Cisco-Live-BRKCOM-2005-Virtual Machine Fabric Extension (VM-FEX)-2013

VM-FEX Configuration & Best Practice for Multiple Hypervisor

Please refer to the attachmenta

Cisco VM-FEX Performance Testing verus VMware vSwitch

External (Light Reading) VM-FEX Peroformance Testing verus VMware vSwitch


VM-FEX Performance Testing with SAP

VM-FEX Performance Testing with Microsoft SharePoint 2010

VM-FEX Performance Testing with Microsoft SQL Server 2012

VM-FEX Performance Testing with Citrix Xenapp 6.5

Microsoft Private Cloud Fast Track 3.0 with VM-FEX

Video Configuration Demo Topology

  • Nexus 5548/C22 M3/ VIC1225/ESXi
  • Nexus 5548-A is pre-configured, focus on 5548-B with the same configuration in the demo
  • Uplink Redundancy – Each static vNIC configure as Active/Standby

     -No need for OS teaming software

     -Required for hypervisor uplink

  • Each dynamic vNIC attach to uplink in Round Robin fashion
  • vPC Doman and Peer Link is configured to synchronize veth numbering for the same VM

          -Not used for the forwarding plane

nexus 5k VM-FEX.png

Nexus 5500 VM-FEX Configuration Workflow


Video Link : 11359



Dumb question from my part and maybe someone can clear it up:)

As each vNIC has a failover (uplink failover) shouldn't we define vethernet 1, respectively vethernet 2 on both Nexus?


on both

interface vethernet 2

bind interface eth102/1/32 channel 2


interface vethernet 1

bind interface ethernet 102/1/32 channel 1

Hope someone can help clear this up,


Content for Community-Ad