cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3684
Views
1
Helpful
1
Comments
timoma
Level 1
Level 1

Overview:

A virtualized server implementation consists of one or more VMs running as 'guests' on a single physical server. The guest VMs are hosted and managed by a software layer called the hypervisor or virtual machine manager (VMM). The hypervisor typically presents a virtual network interface to each VM and performs Layer 2 switching of traffic from a VM to other local VMs or to a physical interface to the external network. Working with a Cisco virtual interface card (VIC) adapter, Cisco Virtual Machine Fabric Extender (VM-FEX)bypasses software-based switching of VM traffic by the hypervisor in favor of external hardware-based switching in the fabric interconnect. This method results in a reduced load on the server CPU, faster switching,and the ability to apply a rich set of network management features to local and remote traffic. VM-FEX extends the (prestandard) IEEE 802.1Qbh port extender architecture to the VMs, providing each VM interface with a virtual Peripheral Component Interconnect Express (PCIe) device and a virtual port ona switch. This solution allows precise rate limiting and quality of service (QoS) guarantees on the VM interface.

Infrastructure Requirements

  • Nexus VMware VM-FEX is available with Nexus 5500 Series and UCS standalone C series with VIC (P81E,VIC1225, VIC1225T)
  • Each VIC card supports up to 96 Virtual Machine Interface
  • Enterprise Plus License required (as is for any DVS) on Host
  • Standard License and above is required for vCenter
  • Hypervisor features are supported for both emulated and hypervisor bypass mode

     – vMotion, HA, DRS, Snapshot and Hot add/remove virtual device, Suspend/Resume

  • VMDirectPath (Hypervisor Bypass) with VM-FEX is supportted with ESXi5.0+
  • VM-FEX upgrade is supportted from ESXi4.X to ESXi5.X with Customized ISO and VMware Update Manager

VM-FEX Mode of Operations


VMware
VMware

VM-FEX Mode

Emulation

Hypervisor Bypass

Hypervisor Version

ESX 4.0 U1 +

ESXi 5.0 +

Minmum NX-OS Version

5.1(3)N1(1)5.1(3)N1(1)
Minimum VIC Firmware Version

P81E:         1.6(2a)

VIC1225:     HUU 1.5.1(b)

VIC1225T:   HUU 1.5.1(b)

P81E:             1.6(2a)

VIC1225:         HUU 1.5.1(b)

VIC1225T:       HUU 1.5.1(b)

Minimum CIMC Version

P81E:         1.4(2.3)

VIC1225:     HUU 1.5.1(b)

VIC1225T:   HUU 1.5.1(b)

P81E:             1.4(2.3)

VIC1225:         HUU 1.5.1(b)

VIC1225T:       HUU 1.5.1(b)

VMotion / Live Migration

SupportSupport
Support Guest OS Any OS support by ESX/ ESXi

RHEL 6.0+ , SLES 11+

Window 2008 SP2, Windows 2008 R2, Windows 2012

Guest OS/vSphere Version
vSphere 5.0
vSphere 5.0 U1
vSphere 5.1
Windows 2012N/A1.2.24.0 (vmwaretools)1.3.9.0 (vmwaretools)
Windows 2008 R21.2.22.0 (vmwaretools)1.2.24.0 (vmwaretools)1.3.9.0 (vmwaretools)
Windows 2008 SP21.2.22.0 (vmwaretools)1.2.24.0 (vmwaretools)1.3.9.0 (vmwaretools)
RHEL 6.01.0.14.0-k (inbox)1.0.14.0-k (inbox)1.0.14.0-k (inbox)
RHEL 6.11.0.14.0-k (inbox)1.0.14.0-k (inbox)1.0.14.0-k (inbox)
RHEL 6.2N/A

1.1.18.0-k (inbox)

1.0.47.0 (vmwaretools)

1.1.18.0-k (inbox)

1.1.32.0 (vmwaretools)

SLES111.0.36.0 (vmwaretools)1.0.47.0 (vmwaretools)1.1.32.0 (vmwaretools)
SLES11 SP11.0.36.0 (vmwaretools)1.0.47.0 (vmwaretools)1.1.32.0 (vmwaretools)




VM-FEX Scalability

UCSM Configuration Limit 2.1 Release -http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/configuration_limits/2.1/b_UCS_Configuration_Limits_2_1.html

Cisco UCS 6100 / 6200 Series

Hypervisor

Half-Width Blade with Single VIC

Full-Width Blade with Dual VIC

ESX 4.0 – 4.1 (DirectPath I/O)

56 (54 vNIC + 2 vHBA)

56 (54 vNIC + 2 vHBA)

ESXi 5.0 – 5.1 (DirectPath I/O)

116 (114 vNIC + 2 vHBA)

116 (114 vNIC + 2 vHBA)*

Windows 2012 (SR-IOV)

116 (114 vNIC + 2 vHBA)

232 (228 vNIC + 4 vHBA)

KVM 6.1 – 6.3 (SR-IOV)

116 (114 vNIC + 2 vHBA)

232 (228 vNIC + 4 vHBA)

* Additional VIC will NOT increase the total VIF count due to OS limitation

* Multiple VICs are Supported for full width blade and B200M3

Nexus 5500  Series
HypervisorAdapter FEXVM-FEX

ESX 4.1 – ESXi 5.1

96 vNIC

96 vNIC

* Only one VIC (P81E/VIC1225) is Supported for each C series rack server

Nexus Verified Scalability 6.0(2)N1(1) release

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/configuration_limits/b_N5500_Config_Limits_602N11_chapter_01.html#reference_BCDAA1182FD641C3B350AC9C1825050A

Feature
Verified Topology
Verified Maximum
Number of VFCs over Virtual Ethernet Interfaces40 40
Number of Port Profiles1,0001,000
Number of Virtual Machines (VMs) Concurrently not VMotioned5 VMs with 10 vNICs each5 VMs with 10 vNICs each
Number of Virtual Ethernet Interfaces Enabled with vNIC Shaping2,0002,000
Number of Virtual Ethernet Interfaces Enabled with Untagged CoS2,0002,000
Server - Number of Adapters per Server11
Server - Number of vNICs per Server5096

VM-FEX Feature Comparision

Feature

N5K

Adapter FEX

N5K

VM-FEX

UCS

VM-FEX

ERSPANYes

SPAN YesYes
FCoEYesYesYes
Port Security YesYesYes
IGMP Snooping YesYes
Qos Shaping YesYesYes
QoS Classification Yes YesYes
PVLANYes YesIsloate
L2 Multipath Edge Port Edge Port
TrustSec Yes Yes
IPv4/IPv6/MAC ACLYesYes
DHCP Snooping YesYes
IP Source Guard YesYes
SNMP YesYes
Netflow+

Yes




Reference Documentation

Nexus VM-FEX Configuration Guide

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/mkt_ops_guides/513_n1_1/n5k_ops_vmfex.html#wp1075280

Cisco-Live-BRKCOM-2005-Virtual Machine Fabric Extension (VM-FEX)-2013

VM-FEX Configuration & Best Practice for Multiple Hypervisor

Please refer to the attachmenta

Cisco VM-FEX Performance Testing verus VMware vSwitch

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/whitepaper_c11-727581.html

External (Light Reading) VM-FEX Peroformance Testing verus VMware vSwitch

http://www.lightreading.com/document.asp?doc_id=693092

                                                         

VM-FEX Performance Testing with SAP

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/whitepaper_c11_703103.pdf

VM-FEX Performance Testing with Microsoft SharePoint 2010

http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/ucs_sharepoint2010_flexpod_vmware.html#wp402742

VM-FEX Performance Testing with Microsoft SQL Server 2012

http://www.cisco.com/en/US/prod/collateral/ps10265/ps10281/whitepaper_c11-711915.html

VM-FEX Performance Testing with Citrix Xenapp 6.5

http://www.cisco.com/en/US/docs/unified_computing/ucs/UCS_CVDs/citrix_emc_ucs_XDXAscale.pdf

Microsoft Private Cloud Fast Track 3.0 with VM-FEX

Video Configuration Demo Topology

  • Nexus 5548/C22 M3/ VIC1225/ESXi
  • Nexus 5548-A is pre-configured, focus on 5548-B with the same configuration in the demo
  • Uplink Redundancy – Each static vNIC configure as Active/Standby

     -No need for OS teaming software

     -Required for hypervisor uplink

  • Each dynamic vNIC attach to uplink in Round Robin fashion
  • vPC Doman and Peer Link is configured to synchronize veth numbering for the same VM

          -Not used for the forwarding plane

nexus 5k VM-FEX.png

Nexus 5500 VM-FEX Configuration Workflow

nexus-vmfex-workflow.png

Video Link : 11359

Comments
mihaitanasescu
Level 1
Level 1

Hi,

Dumb question from my part and maybe someone can clear it up:)

As each vNIC has a failover (uplink failover) shouldn't we define vethernet 1, respectively vethernet 2 on both Nexus?

Ex:

on both

interface vethernet 2

bind interface eth102/1/32 channel 2

and

interface vethernet 1

bind interface ethernet 102/1/32 channel 1

Hope someone can help clear this up,

Mihai

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: