Introduction
This document describes a procedure that can be used to recover an ESX or ESXi host that has become isolated from the network due to a failed or misconfigured VEM. When deploying the Nexus 1000V Virtual Ethernet Module (VEM), some configuration errors and failures may cause the ESX host to become completely isolated from the network. An example of a configuration error that would cause this is failing to make the Control vlan a system vlan in the uplink port-profile and then moving the service console or vmkernel behind the VEM. The VEM will not bring any non-system VLANs up before it has established connectivity to the VSM, and if the Control vlan is not a system vlan then the VEM will never be able to establish communication with the VSM. The following section describes a recovery technique that can be used in the event the vmkernel or service console has been moved behind the VEM, and all network connectivity to the host has been lost due to a configuration or other type of error.
VSM and VEM
Cisco Nexus 1000V has 2 parts:
Virtual supervisor module (VSM) - This is the control software of the Cisco Nexus 1000V distributed virtual switch. It runs on a virtual machine (VM) and is based on NX-OS software.
Virtual Ethernet module (VEM) - This is the part of Cisco Nexus 1000V that actually switches data traffic. It runs on a VMware ESX 4.0 host. Several VEMs are controlled by one VSM. All the VEMs that form a switch domain should be in the same virtual Data Center as defined by VMware VirtualCenter.
Recovering Network Connectivity
In order to recover network connectivity to the server, the service console or vmkernel needs to be moved back to the vSwitch. In addition any uplinks that were moved to the VEM must be moved back to the vSwitch. Since network connectivity has been lost to the host, the host must be accessed via the physical console or through a KVM system. A Cisco UCS blade can be accessed using the embedded KVM. The below procedure can be used to do this. The procedure must be performed using Tech Support mode on the ESX or ESXi host.
1) Access the console of the host physically or through KVM.
2) Log into Tech Support Mode:
Prior to ESX 4.1, at the main console screen enter ALT-F1. Type in "unsupported" (without quotes), the characters will not be echoed on the screen. Hit enter and a prompt for userid/password should appear.
In ESX 4.1 and above code Local Tech Support is disabled by default. To enable it press F2, select Troubleshooting Options, and enable Local Tech Support. Then log out and enter ALT-F1. At this point there should be a prompt for userid/password.
3) Use the "esxcfg-vswitch" command to remove an uplink from the VEM (DVS) and put it on the vSwitch. Do esxcfg-vswitch -l | more to determine the DVPortID for VMNIC0, vmk0, and the DVS name. These will be used as input to the commands below.
Remove an uplink from the DVS:
esxcfg-vswitch -V [DV Port ID] -Q [vmnic] [DVS Name]
Add the uplink to the vSwitch:
esxcfg-vswitch -L [vmnic] [vswitch]
4) Move the Service Console in the case of ESX, or vmkernel in the case of ESXi back to the vSwitch:
For ESX systems migrate vswif0 from DVS to vSwitch:
esxcfg-vswif -p {Port Group Name] [vswif]
For ESXi systems vmk0 must be deleted and re-added:
esxcfg-vmknic -d -s [dvs name] -v [dv-port-id]
esxcfg-vmknic -a -i [ip address] -n [netmask] [port-group-name]
To get out of tech support mode and back to the main screen without rebooting type exit, then press ALT-F2.
Removing a Port Profile
Enter the following commands
a. config t
b. no port-profile profile_name
c. show port-profile name profile_name
d. copy running-config startup-config
Obtaining a File of Core Memory Information
Cisco customer support engineers often use files from your system for analysis. One of these is a file containing memory information, and is referred to as a core dump. The file is sent to a TFTP server or to a Flash card in slot0: of the local switch. You should set up your switch to generate this file under the instruction of your customer support representative, and send it to a TFTP server so that it can be e-mailed to them.
To generate a file of core memory information, or a core dump, use the command in the following example.
n1000v# system cores tftp://10.91.51.200/jsmith_cores
Related Information
How to remove an orphaned 1000v DVS
Install and Configure Nexus 1000v on VMware
