Overview: A virtualized server implementation consists of one or more VMs running as 'guests' on a single physical server. The guest VMs are hosted and managed by a software layer called the hypervisor or virtual machine manager (VMM). The hypervisor typically presents a virtual network interface to each VM and performs Layer 2 switching of traffic from a VM to other local VMs or to a physical interface to the external network. Working with a Cisco virtual interface card (VIC) adapter, Cisco Virtual Machine Fabric Extender (VM-FEX)bypasses software-based switching of VM traffic by the hypervisor in favor of external hardware-based switching in the fabric interconnect. This method results in a reduced load on the server CPU, faster switching,and the ability to apply a rich set of network management features to local and remote traffic. VM-FEX extends the (prestandard) IEEE 802.1Qbh port extender architecture to the VMs, providing each VM interface with a virtual Peripheral Component Interconnect Express (PCIe) device and a virtual port ona switch. This solution allows precise rate limiting and quality of service (QoS) guarantees on the VM interface. Infrastructure Requirements Nexus VMware VM-FEX is available with Nexus 5500 Series and UCS standalone C series with VIC (P81E,VIC1225, VIC1225T) Each VIC card supports up to 96 Virtual Machine Interface Enterprise Plus License required (as is for any DVS) on Host Standard License and above is required for vCenter Hypervisor features are supported for both emulated and hypervisor bypass mode      – vMotion , HA, DRS, Snapshot and Hot add/remove virtual device, Suspend/Resume VMDirectPath (Hypervisor Bypass) with VM-FEX is supportted with ESXi5.0+ VM-FEX upgrade is supportted from ESXi4.X to ESXi5.X with Customized ISO and VMware Update Manager VM-FEX Mode of Operations VMware VMware VM-FEX Mode Emulation Hypervisor Bypass Hypervisor Version ESX 4.0 U1 + ESXi 5.0 + Minmum NX-OS Version 5.1(3)N1(1) 5.1(3)N1(1) Minimum VIC Firmware Version P81E:         1.6(2a) VIC1225:     HUU 1.5.1(b) VIC1225T:   HUU 1.5.1(b) P81E:             1.6(2a) VIC1225:         HUU 1.5.1(b) VIC1225T:       HUU 1.5.1(b) Minimum CIMC Version P81E:         1.4(2.3) VIC1225:     HUU 1.5.1(b) VIC1225T:   HUU 1.5.1(b) P81E:             1.4(2.3) VIC1225:         HUU 1.5.1(b) VIC1225T:       HUU 1.5.1(b) VMotion / Live Migration Support Support Support Guest OS Any OS support by ESX/ ESXi RHEL 6.0+ , SLES 11+ Window 2008 SP2, Windows 2008 R2, Windows 2012 Guest OS/vSphere Version vSphere 5.0 vSphere 5.0 U1 vSphere 5.1 Windows 2012 N/A 1.2.24.0 (vmwaretools) 1.3.9.0 (vmwaretools) Windows 2008 R2 1.2.22.0 (vmwaretools) 1.2.24.0 (vmwaretools) 1.3.9.0 (vmwaretools) Windows 2008 SP2 1.2.22.0 (vmwaretools) 1.2.24.0 (vmwaretools) 1.3.9.0 (vmwaretools) RHEL 6.0 1.0.14.0-k (inbox) 1.0.14.0-k (inbox) 1.0.14.0-k (inbox) RHEL 6.1 1.0.14.0-k (inbox) 1.0.14.0-k (inbox) 1.0.14.0-k (inbox) RHEL 6.2 N/A 1.1.18.0-k (inbox) 1.0.47.0 (vmwaretools) 1.1.18.0-k (inbox) 1.1.32.0 (vmwaretools) SLES11 1.0.36.0 (vmwaretools) 1.0.47.0 (vmwaretools) 1.1.32.0 (vmwaretools) SLES11 SP1 1.0.36.0 (vmwaretools) 1.0.47.0 (vmwaretools) 1.1.32.0 (vmwaretools) VM-FEX Scalability UCSM Configuration Limit 2.1 Release -http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/configuration_limits/2.1/b_UCS_Configuration_Limits_2_1.html Cisco UCS 6100 / 6200 Series Hypervisor Half-Width Blade with Single VIC Full-Width Blade with Dual VIC ESX 4.0 – 4.1 ( DirectPath I/O) 56 (54 vNIC + 2 vHBA ) 56 (54 vNIC + 2 vHBA ) ESXi 5.0 – 5.1 ( DirectPath I/O) 116 (114 vNIC + 2 vHBA ) 116 (114 vNIC + 2 vHBA ) * Windows 2012 (SR-IOV) 116 (114 vNIC + 2 vHBA ) 232 (228 vNIC + 4 vHBA ) KVM 6.1 – 6.3 (SR-IOV) 116 (114 vNIC + 2 vHBA ) 232 (228 vNIC + 4 vHBA ) * Additional VIC will NOT increase the total VIF count due to OS limitation * Multiple VICs are Supported for full width blade and B200M3 Nexus 5500  Series Hypervisor Adapter FEX VM-FEX ESX 4.1 – ESXi 5.1 96 vNIC 96 vNIC * Only one VIC (P81E/VIC1225) is Supported for each C series rack server Nexus Verified Scalability 6.0(2)N1(1) release http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/configuration_limits/b_N5500_Config_Limits_602N11_chapter_01.html#reference_BCDAA1182FD641C3B350AC9C1825050A Feature Verified Topology Verified Maximum Number of VFCs over Virtual Ethernet Interfaces 40 40 Number of Port Profiles 1,000 1,000 Number of Virtual Machines (VMs) Concurrently not VMotioned 5 VMs with 10 vNICs each 5 VMs with 10 vNICs each Number of Virtual Ethernet Interfaces Enabled with vNIC Shaping 2,000 2,000 Number of Virtual Ethernet Interfaces Enabled with Untagged CoS 2,000 2,000 Server - Number of Adapters per Server 1 1 Server - Number of vNICs per Server 50 96 VM-FEX Feature Comparision Feature N5K Adapter FEX N5K VM-FEX UCS VM-FEX ERSPAN Yes SPAN Yes Yes FCoE Yes Yes Yes Port Security Yes Yes Yes IGMP Snooping Yes Yes Qos Shaping Yes Yes Yes QoS Classification Yes Yes Yes PVLAN Yes Yes Isloate L2 Multipath Edge Port Edge Port TrustSec Yes Yes IPv4/IPv6/MAC ACL Yes Yes DHCP Snooping Yes Yes IP Source Guard Yes Yes SNMP Yes Yes Netflow+ Yes Reference Documentation Nexus VM-FEX Configuration Guide http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/mkt_ops_guides/513_n1_1/n5k_ops_vmfex.html#wp1075280 Cisco-Live-BRKCOM-2005-Virtual Machine Fabric Extension (VM-FEX)-2013 VM-FEX Configuration & Best Practice for Multiple Hypervisor Please refer to the attachmenta Cisco VM-FEX Performance Testing verus VMware vSwitch http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/whitepaper_c11-727581.html External (Light Reading) VM-FEX Peroformance Testing verus VMware vSwitch http://www.lightreading.com/document.asp?doc_id=693092                                                           VM-FEX Performance Testing with SAP http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/whitepaper_c11_703103.pdf VM-FEX Performance Testing with Microsoft SharePoint 2010 http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_sharepoint2010_flexpod_vmware.html#wp402742 VM-FEX Performance Testing with Microsoft SQL Server 2012 http://www.cisco.com/en/US/prod/collateral/ps10265/ps10281/whitepaper_c11-711915.html VM-FEX Performance Testing with Citrix Xenapp 6.5 http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/citrix_emc_ucs_XDXAscale.pdf Microsoft Private Cloud Fast Track 3.0 with VM-FEX http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_flexpod_mspc30_sc12_deploy.pdf Video Configuration Demo Topology Nexus 5548/C22 M3/ VIC1225/ ESXi Nexus 5548-A is pre-configured, focus on 5548-B with the same configuration in the demo Uplink Redundancy – Each static vNIC configure as Active/Standby      - No need for OS teaming software      - Required for hypervisor uplink Each dynamic vNIC attach to uplink in Round Robin fashion vPC Doman and Peer Link is configured to synchronize veth numbering for the same VM           - Not used for the forwarding plane Nexus 5500 VM-FEX Configuration Workflow Video Link : 11359 ... View more

KVM VM-FEX Standalone Video Configuration Guide ... View more

Nexus VMware VM-FEX Video Configuration Guide   ... View more

UCS VMware VM-FEX Video Configuration Guide ... View more

Hyper-V VM-FEX with SCVMM Video Configuration Guide   ... View more

Overview: A virtualized server implementation consists of one or more VMs running as 'guests' on a single physical server. The guest VMs are hosted and managed by a software layer called the hypervisor or virtual machine manager (VMM). The hypervisor typically presents a virtual network interface to each VM and performs Layer 2 switching of traffic from a VM to other local VMs or to a physical interface to the external network. Working with a Cisco virtual interface card (VIC) adapter, Cisco Virtual Machine Fabric Extender (VM-FEX)bypasses software-based switching of VM traffic by the hypervisor in favor of external hardware-based switching in the fabric interconnect. This method results in a reduced load on the server CPU, faster switching,and the ability to apply a rich set of network management features to local and remote traffic. VM-FEX extends the (prestandard) IEEE 802.1Qbh port extender architecture to the VMs, providing each VM interface with a virtual Peripheral Component Interconnect Express (PCIe) device and a virtual port ona switch. This solution allows precise rate limiting and quality of service (QoS) guarantees on the VM interface. Infrastructure Requirements The host must be managed by Cisco UCS Manager Release 2.1 or later. The host operating system must be an edition of Windows Server 2012 or later that includes Hyper-V support. For more information about installing Microsoft Hyper-V, see the Microsoft Windows Server documentation. In the host BIOS Processor settings, you must enable Virtualization Technology (VT) and Direct Cache Access. In the host BIOS Intel Directed I/O settings, you must enable VT For Directed I/O and Interrupt Remap. For more information about configuring UCS server BIOS settings, see the Cisco UCS Manager GUI Configuration Guide. HTTPS communication must be enabled on the Cisco UCS Manager. For more information about enabling HTTPS communication, see the Cisco UCS Manager GUI Configuration Guide. One or more Cisco VIC adapters must be installed in the host. For more information about installing a Cisco VIC adapter, see the Cisco UCS 5108 Server Chassis Hardware Installation Guide. An SR-IOV PF cannot be used as an iSCSI vNIC. An existing iSCSI vNIC cannot be defined as a PF. VM-FEX Mode of Operations Windows Windows VM-FEX Mode Emulation Hypervisor Bypass with SR-IOV Minimum Hypervisor Version Windows 2012 Windows 2012 Minmum ENIC Driver Version UCSM 2.1(1a) UCSM 2.1(1a) Minimum VIC Firmware Version UCSM 2.1(1a) UCSM 2.1(1a) VM-FEX Tool Pleaese refer to attachment Please refer to attachment VMotion / Live Migration Support Support Support Guest OS Any OS support by Hyper-V Windows 2012 only VM-FEX Scalability UCSM Configuration Limit 2.1 Release -http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/configuration_limits/2.1/b_UCS_Configuration_Limits_2_1.html Cisco UCS 6100 / 6200 Series Hypervisor Half-Width Blade with Single VIC Full-Width Blade with Dual VIC ESX 4.0 – 4.1 ( DirectPath I/O) 56 (54 vNIC + 2 vHBA ) 56 (54 vNIC + 2 vHBA ) ESXi 5.0 – 5.1 ( DirectPath I/O) 116 (114 vNIC + 2 vHBA ) 116 (114 vNIC + 2 vHBA ) * Windows 2012 (SR-IOV) 116 (114 vNIC + 2 vHBA ) 232 (228 vNIC + 4 vHBA ) KVM 6.1 – 6.3 (SR-IOV) 116 (114 vNIC + 2 vHBA ) 232 (228 vNIC + 4 vHBA ) * Additional VIC will NOT increase the total VIF count due to OS limitation * Multiple VICs are Supported for full width blade and B200M3 Video Configuration Demo Topology Hyper-V VM-FEX Configuration Workflow Video Link : 11298 Reference Documentation UCSM Hyper-V VM-FEX GUI Configuratoin Guide http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/vm_fex/hyperv/gui/config_guide/2.1/b_GUI_Hyper-V_VM-FEX_UCSM_Configuration_Guide_chapter_010.html UCSM Hyper-V VM-FEX CLI Configuratoin Guide http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/vm_fex/hyperv/cli/config_guide/2.1/b_CLI_Hyper-V_VM-FEX_UCSM_Configuration_Guide_chapter_010.html Hyper-V VM-FEX Configuration Example - TAC http://www.cisco.com/en/US/products/ps10281/products_configuration_example09186a0080c1780d.shtml Cisco-Live-BRKCOM-2005-Virtual Machine Fabric Extension (VM-FEX)-2013 VM-FEX Configuration & Best Practice for Multiple Hypervisor Please refer to the attachmenta Cisco VM-FEX Performance Testing verus VMware vSwitch http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/whitepaper_c11-727581.html External (Light Reading) VM-FEX Peroformance Testing verus VMware vSwitch http://www.lightreading.com/document.asp?doc_id=693092                                                          VM-FEX Performance Testing with SAP http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/whitepaper_c11_703103.pdf VM-FEX Performance Testing with Microsoft SharePoint 2010 http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_sharepoint2010_flexpod_vmware.html#wp402742 VM-FEX Performance Testing with Microsoft SQL Server 2012 http://www.cisco.com/en/US/prod/collateral/ps10265/ps10281/whitepaper_c11-711915.html VM-FEX Performance Testing with Citrix Xenapp 6.5 http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/citrix_emc_ucs_XDXAscale.pdf Microsoft Private Cloud Fast Track 3.0 with VM-FEX http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_flexpod_mspc30_sc12_deploy.pdf ... View more

Hyper-V VM-FEX Standalone Video Configuration Guide ... View more