topic Problem Deploying Nexus 1000v in Server Networking

Problem Deploying Nexus 1000v

arikgelman — Mon, 05 Aug 2013 19:06:30 GMT

Hi,

I would very happy if someone can help me.

I am trying to install cisco 1000v on 5 X Esxi.

I was able to install the VSM and the VEM on each of the ESXi but I wasn't able to connect between them

Every time I am trying to add host to my nexus switch I am getting this error:

"vDS operation failed on host hostnamexxxx, An error occurred during host configuration. got (vim.fault.PlatformConfigFault) exception"

I don't have any svs neighbors and my "show module" shows only the VSM.

I am using Nexus 1000V Release 4.2(1)SV2(2.1) and ESXI version 5.1

I tried to use the install app that comes with this version but when I am trying to deploy the VSM I am getting some kind of error.

Thanks in advance

Problem Deploying Nexus 1000v

calopez2 — Mon, 05 Aug 2013 19:20:29 GMT

Hi,

Please ensure the VEM is actually installed in the VMWARE host.

If not, please manually install the VEM in the Host and check if this resolves the problem.

Regards,

Carlos

Problem Deploying Nexus 1000v

arikgelman — Mon, 05 Aug 2013 19:37:19 GMT

What does it mean on each host ?

I installed the VEM manually on each of the servers (on each ESXI server), Is this what you mean?

Problem Deploying Nexus 1000v

calopez2 — Mon, 05 Aug 2013 20:17:01 GMT

Hi,

Great. You said you have manually installed/verified the VEM installation on the server.

If using Layer 3 connectivity, can you please verify the VMKernel mapping for the VSM-VEM connectivty?

Problem Deploying Nexus 1000v

arikgelman — Tue, 06 Aug 2013 02:51:21 GMT

How do I do this? How do I configure the VEM to work with the VSM ? I haven't found any configuration commands for the VEM.

Please help me with this...Thanks

Problem Deploying Nexus 1000v

suryakant.chavan — Wed, 07 Aug 2013 07:15:07 GMT

Hi Team,

In lab we have one physical server. we have created two Esxi servers on it and planning to implement nexus 1000v in HA on it. Will it work.

Thanks and regards,
Surya

Problem Deploying Nexus 1000v

Robert Burns — Wed, 07 Aug 2013 13:40:02 GMT

Yes. I've created a doc detailing how to setup the N1K in a Nested-ESX environment.

https://learningnetwork.cisco.com/servlet/JiveServlet/previewBody/17233-102-1-67123/How%20to%20Deploy%20a%20Nexus%201000v%20lab%20with%20a%20single%20ESX%20host.pdf

Regards,

Robert

Problem Deploying Nexus 1000v

arikgelman — Sun, 11 Aug 2013 10:21:45 GMT

I was able to add finally the esxi to the Nexus 1000V but now every port I am adding I have:

"port blocked by admin"

does someone knows what to do in this situation?

Problem Deploying Nexus 1000v

sprasath — Sun, 11 Aug 2013 10:34:14 GMT

Do you see the VEM modules inserted into the VSM - can check using 'show module' on the VSM?

Where are you seeing port blocked by admin?

If it's for VMs, does the port-profile have a 'no shutdown' and are the interfaces enabled on the OS level?

Thanks,

Shankar

Problem Deploying Nexus 1000v

arikgelman — Sun, 11 Aug 2013 10:40:53 GMT

Hi,

Thanks for the fast repay.

I don't see the VEM under 'show module'

I see it in the vcenter under the port-profiles

The port-profile are with 'no shutdown' command

Problem Deploying Nexus 1000v

sprasath — Sun, 11 Aug 2013 10:48:08 GMT

Well, there should be a communication between the VEM and the VSM for your ports to come up. If 'show module' does not show your VEM, then this is not up. It's a little complicated to troubleshoot this in here. To start with:

Are you using the L2 mode or the L3 mode?

If you are using the L2 mode, you need to make sure that there are no blockages on the control VLAN between the VSM's NIC 1 and the VEM uplinks

If you are using the L3 mode, you need to make sure that the vmk IP address that you chose as l3-capable should be able to talk to the VSM's interface that you are choosing as the L3 point of contact.

Thanks,

Shankar

Problem Deploying Nexus 1000v

arikgelman — Sun, 11 Aug 2013 11:04:34 GMT

I was able to make the uplinks go UP,

Now the vethernet ports are the only one that are down.

Problem Deploying Nexus 1000v

sprasath — Sun, 11 Aug 2013 11:12:56 GMT

Can you tell me what changes you did to make the uplinks come up?

And do you see the modules showing up on the VSM?

If yes, then you should proceed looking at the Vethernet ports.

Post the output of 'show module' and 'show int br' from the VSM, 'vemcmd show port' and 'vemcmd show card' from the ESX host

Thanks,

Shankar

Problem Deploying Nexus 1000v

arikgelman — Sun, 11 Aug 2013 11:51:01 GMT

I added the uplink interface to one of the MGMT interfaces of the ESXi.

I still don't see the VEM under 'show module'

Problem Deploying Nexus 1000v

sprasath — Mon, 12 Aug 2013 06:38:58 GMT

Like I said, it might be hard to troubleshoot this over here. Get the following outputs and we can give it a shot:

From the VSM:

- show svs domain

- Port-profile that you are using for the uplink interface

- If L3 mode, port-profile that you are using for the l3 capable vmknic

From the VEM:

- vemcmd show card

- vemcmd show port

- esxcfg-vmknic -l -> in case of using the L3 mode

The above would be useful to see if there is an error with your configuration within the Nexus 1000v.

Try to explain the topology between the ESX host and the VSM.

Please see if you can follow the deployment guide:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/guide_c07-556626.html

If you are using the UCS:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/guide_c07-704280.html

Thanks,

Shankar

Problem Deploying Nexus 1000v

suryakant.chavan — Mon, 12 Aug 2013 13:17:17 GMT

Hi Team,

I have configured VSM and VEM on same EXS server ( 10.10.1.3) but unable to have communication between them.

I am not able to see VEM in VSM sh module command . VSM and Vcentre (10.10.1.12) are communicating properly. VSM is in L2 mode.

Below are some outputs

~ # vemcmd show trunk

Trunk port 16 native_vlan 1 CBL 1

vlan(1) cbl 1, vlan(3972) cbl 1, vlan(3970) cbl 1, vlan(3969) cbl 1, vlan(3968) cbl 1, vlan(3971) cbl 1, vlan(111) cbl 1, vlan(113) cbl 1,

~ # vemcmd show card info

Card UUID type 2: 44454c4c-5900-104b-804d-cac04f393253

Card name: Esxi03

Switch name: Nexus1000V

Switch alias: DvsPortset-0

Switch uuid: 7c 0d 38 50 5c 0c 2b 26-b0 d9 34 b3 9e df 29 27

Card domain: 1

Card slot: 1

VEM Tunnel Mode: L2 Mode

VEM Control (AIPC) MAC: 00:02:3d:10:01:00

VEM Packet (Inband) MAC: 00:02:3d:20:01:00

VEM Control Agent (DPA) MAC: 00:02:3d:40:01:00

VEM SPAN MAC: 00:02:3d:30:01:00

Primary VSM MAC : 00:50:56:b8:68:58

Primary VSM PKT MAC : 00:50:56:b8:38:97

Primary VSM MGMT MAC : 00:50:56:b8:34:7b

Standby VSM CTRL MAC : ff:ff:ff:ff:ff:ff

Management IPv4 address: 10.10.1.3

Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000

Secondary VSM MAC : 00:00:00:00:00:00

Secondary L3 Control IPv4 address: 0.0.0.0

Upgrade : Default

Max physical ports: 32

Max virtual ports: 216

Card control VLAN: 111

Card packet VLAN: 113

Card Headless Mode : Yes

Processors: 4

Processor Cores: 4

Processor Sockets: 1

Kernel Memory: 16767124

Port link-up delay: 5s

Global UUFB: DISABLED

Heartbeat Set: False

PC LB Algo: source-mac

Datapath portset event in progress : no

Licensed: No

~ # vemcmd show port vlans

Native VLAN Allowed

LTL VSM Port Mode VLAN State Vlans

18 Eth1/2 A 1 BLK 1

~ #

~ # vemcmd show port

LTL VSM Port Admin Link State PC-LTL SGID Vem Port Type

18 Eth1/2 DOWN UP BLK 0 vmnic1

Nexus1000V# sh svs domain
SVS domain config:
Domain id:    1
Control vlan: 111
Packet vlan: 113
L2/L3 Control mode: L2
L3 control interface: NA
Status: Config push to VC successful.
Nexus1000V# sh s
scheduler        ssh              svs
snmp             startup-config   switchname
sockets          stun             system
Nexus1000V# sh svs connections

connection LM_NSK:
    ip address: 10.10.1.12
    remote port: 80
    protocol: vmware-vim https
    certificate: default
    datacenter name: DC1
    admin:
    max-ports: 8192
    DVS uuid: 7c 0d 38 50 5c 0c 2b 26-b0 d9 34 b3 9e df 29 27
    config status: Enabled
    operational status: Connected
    sync status: Complete
    version: VMware vCenter Server 5.1.0 build-799731
    vc-uuid: 6643B3C4-31A9-4A62-A9D5-6B214FB62CE5
Nexus1000V#
Nexus1000V#

~ # vemcmd show trunk
Trunk port 16 native_vlan 1 CBL 1
vlan(1) cbl 1, vlan(3972) cbl 1, vlan(3970) cbl 1, vlan(3969) cbl 1, vlan(3968)                                                                               cbl 1, vlan(3971) cbl 1, vlan(111) cbl 1, vlan(113) cbl 1,
~ # vemcmd show card info
Card UUID type 2: 44454c4c-5900-104b-804d-cac04f393253
Card name: Esxi03
Switch name: Nexus1000V
Switch alias: DvsPortset-0
Switch uuid: 7c 0d 38 50 5c 0c 2b 26-b0 d9 34 b3 9e df 29 27
Card domain: 1
Card slot: 1
VEM Tunnel Mode: L2 Mode
VEM Control (AIPC) MAC: 00:02:3d:10:01:00
VEM Packet (Inband) MAC: 00:02:3d:20:01:00
VEM Control Agent (DPA) MAC: 00:02:3d:40:01:00
VEM SPAN MAC: 00:02:3d:30:01:00
Primary VSM MAC : 00:50:56:b8:68:58
Primary VSM PKT MAC : 00:50:56:b8:38:97
Primary VSM MGMT MAC : 00:50:56:b8:34:7b
Standby VSM CTRL MAC : ff:ff:ff:ff:ff:ff
Management IPv4 address: 10.10.1.3
Management IPv6 address: 0000:0000:0000:0000:0000:0000:0000:0000
Secondary VSM MAC : 00:00:00:00:00:00
Secondary L3 Control IPv4 address: 0.0.0.0
Upgrade : Default
Max physical ports: 32
Max virtual ports: 216
Card control VLAN: 111
Card packet VLAN: 113
Card Headless Mode : Yes
       Processors: 4
Processor Cores: 4
Processor Sockets: 1
Kernel Memory:   16767124
Port link-up delay: 5s
Global UUFB: DISABLED
Heartbeat Set: False
PC LB Algo: source-mac
Datapath portset event in progress : no
Licensed: No

~ # vemcmd show port vlans
                          Native VLAN   Allowed
LTL   VSM Port Mode    VLAN    State Vlans
   18     Eth1/2   A          1   BLK    1
~ #
~ # vemcmd show port
LTL   VSM Port Admin Link State PC-LTL SGID Vem Port Type
   18     Eth1/2   DOWN   UP    BLK       0          vmnic1

I am new to this technology, So just want to know whether there is communication between Vcentre and VEM.

Thanks and Regards,

Surya

Problem Deploying Nexus 1000v

suryakant.chavan — Tue, 13 Aug 2013 13:16:17 GMT

Hi Team,

Below are some outputs, Can you help me to understand why first link is showing as blocking. I have configured vlan 111 as control vlan . but it's not showing.

vemcmd show port
LTL VSM Port Admin Link State PC-LTL SGID Vem Port Type

18 DOWN UP BLK 0 vmnic1
49 UP UP FWD 0 Nexus1000V-4.2...2-Primary.et

~ # vemcmd show port vlans
Native VLAN Allowed
LTL VSM Port Mode VLAN State Vlans

18 A 1 BLK 1
49 A 113 FWD 113

Thanks and regards,

surya

Problem Deploying Nexus 1000v

suryakant.chavan — Tue, 13 Aug 2013 13:17:39 GMT

Updating output again as it's not showing correctly in previous comment

vemcmd show port

LTL VSM Port Admin Link State PC-LTL SGID Vem Port Type

18 DOWN UP BLK 0 vmnic1

49 UP UP FWD 0 Nexus1000V-4.2...2-Primary.et

~ # vemcmd show port vlans

Native VLAN Allowed

LTL VSM Port Mode VLAN State Vlans

18 A 1 BLK 1

49 A 113 FWD 113

Problem Deploying Nexus 1000v

Robert Burns — Tue, 13 Aug 2013 13:19:54 GMT

1. Paste your running config from the VSM

2. The upstream Switch's Interface (show run interface x/y) where the VEM uplinks connect.

Regards,

Robert

Problem Deploying Nexus 1000v

suryakant.chavan — Tue, 13 Aug 2013 14:28:53 GMT

Hi Robert,

Appreciate your prompt response.

Tomarrow I will provide the running config.

Here is my topology

ESXserver(10.10.1.3/24)-----Dlink Switch---Vcenter(10.10.1.12/24)

VSM (10.10.1.160/24) and VEM on same ESX server(10.10.1.3).

I have configured Control Vlan 111, Management Vlan 1 and Packet Vlan 113.

There is no problem between VSM and Vcentre communication.

What I have query is , as VSM and VEM on same server, is it really matter the upstream config as implemented L2 mode and what I read is VSM and VEM communicate on control vlan.

What I see in VEM is I can see Packet vlan 113 but not control vlan 111.

here are outputs

vemcmd show port

LTL VSM Port Admin Link State PC-LTL SGID Vem Port Type

18 DOWN UP BLK 0 vmnic1

49 UP UP FWD 0 Nexus1000V-4.2...2-Primary.et

~ # vemcmd show port vlans

Native VLAN Allowed

LTL VSM Port Mode VLAN State Vlans

18 A 1 BLK 1

49 A 113 FWD 113