cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
915
Views
5
Helpful
2
Replies
Faisal syed
Beginner

Nexus 1000v. Only first VM is able to ping outside vlan interface.

Hello everyone,

I am trying to setup Nexus 1000V VSM and VEM.  My ESXi host has four network ports. For management I am using only one of the ports on the quad nic card as a trunk.  It is trunked to Cisco 3750 switch.  Another one of the quad nic cards is connected to the VDS.  Both ports are in port channel 1.  I moved three VMs from vSwitch to vds . But somehow only one vm which I powered on first is able to ping outside vlan interface, others won't. They only work after I issued command in port-channel 1 shut and no shut. When I reboot the host, it is back to where the first vm can ping outside vlan interface.

Here is the configuration for the Nexus port profiles and the 3750 switch.
port-profile Unused_Or_Quarantine_Uplink
type: Ethernet
description: Port-group created for Nexus1000V internal usage. Do not use.
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  shutdown
evaluated config attributes:
assigned interfaces:
port-group: Unused_Or_Quarantine_Uplink
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile Unused_Or_Quarantine_Veth
type: Vethernet
description: Port-group created for Nexus1000V internal usage. Do not use.
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  shutdown
evaluated config attributes:
assigned interfaces:
port-group: Unused_Or_Quarantine_Veth
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VMK-Control
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
assigned interfaces:
  Vethernet2
port-group: VMK-Control
system vlans: 9
capability l3control: yes
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN2
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  capability iscsi-multipath
  switchport access vlan 2
  no shutdown
evaluated config attributes:
  switchport mode access
  capability iscsi-multipath
  switchport access vlan 2
  no shutdown
assigned interfaces:
  Vethernet1
port-group: VM_VLAN2
system vlans: 2
capability l3control: no
capability iscsi-multipath: yes
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN3
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 3
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 3
  no shutdown
assigned interfaces:
  Vethernet3
  Vethernet4
port-group: VM_VLAN3
system vlans: 3
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN6
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 6
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 6
  no shutdown
assigned interfaces:
  Vethernet7
port-group: VM_VLAN6
system vlans: 6
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN8
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 8
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 8
  no shutdown
assigned interfaces:
port-group: VM_VLAN8
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN9
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
assigned interfaces:
  Vethernet5
  Vethernet6
port-group: VM_VLAN9
system vlans: 9
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile uplink_Connection
type: Ethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode trunk
  switchport trunk allowed vlan 1-10
  channel-group auto mode on mac-pinning
  no shutdown
evaluated config attributes:
  switchport mode trunk
  switchport trunk allowed vlan 1-10
  channel-group auto mode on mac-pinning
  no shutdown
assigned interfaces:
  port-channel1
  Ethernet3/5
  Ethernet3/10
port-group: uplink_Connection
system vlans: 1-3,6,8-10
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

********************************3750 switch configuration************************************

Switch>en
Switch#show run
Building configuration...

Current configuration : 3862 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname Switch
!
!
no aaa new-model
switch 1 provision ws-c3750-24ts
system mtu routing 1500
ip subnet-zero
ip routing
!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
spanning-tree vlan 9 priority 4096
!
vlan internal allocation policy ascending
!
!
!
!
interface Port-channel1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
!
interface Port-channel2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2
switchport mode trunk
!
interface FastEthernet1/0/1
!
interface FastEthernet1/0/2
switchport access vlan 2
spanning-tree portfast
spanning-tree port-priority 0
!
interface FastEthernet1/0/3
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2
switchport mode trunk
channel-group 2 mode active
spanning-tree portfast
!
interface FastEthernet1/0/4
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2
switchport mode trunk
channel-group 2 mode active
spanning-tree portfast
!

!
interface FastEthernet1/0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
channel-group 1 mode active
!
interface FastEthernet1/0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
channel-group 1 mode active
spanning-tree portfast
spanning-tree port-priority 0
!
interface FastEthernet1/0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet1/0/1
!
interface GigabitEthernet1/0/2
!
interface Vlan1
no ip address
!
interface Vlan2
ip address 20.12.162.1 255.255.255.0
!
interface Vlan3
ip address 20.12.161.13 255.255.255.0
!
interface Vlan4
ip address 20.12.165.1 255.255.255.0
!
interface Vlan6
ip address 20.12.166.1 255.255.255.0
!
interface Vlan9
ip address 20.12.167.1 255.255.255.0
!
interface Vlan10
ip address 20.12.168.1 255.255.255.0
!
router eigrp 100
network 20.12.161.0 0.0.0.255
network 20.12.162.0 0.0.0.255
network 20.12.165.0 0.0.0.255
network 20.12.166.0 0.0.0.255
network 20.12.167.0 0.0.0.255
network 20.12.168.0 0.0.0.255
no auto-summary
!
ip classless
ip route 0.0.0.0 0.0.0.0 20.12.165.2
ip http server
!
!
!
control-plane
!
!
line con 0
line vty 0 4
login
line vty 5 15
login
!
end

Switch#

**************************Trunk-status*******************************************

switch# show int trunk

--------------------------------------------------------------------------------
Port       Native  Status        Port
           Vlan                  Channel
--------------------------------------------------------------------------------
Eth3/5     1       trnk-bndl     Po1
Eth3/10    1       trnk-bndl     Po1
Po1        1       trunking      --

--------------------------------------------------------------------------------
Port       Vlans Allowed on Trunk
--------------------------------------------------------------------------------
Eth3/5     2-3,6,8-9
Eth3/10    2-3,6,8-9
Po1        2-3,6,8-9

--------------------------------------------------------------------------------
Port       Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------
Po1        none

************************************MAC pinning**************************************************************

VEM modules are loaded

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         4           128               1500    vmnic5
vSwitch1         128         4           128               1500    vmnic8
vSwitch2         128         1           128               1500
DVS Name         Num Ports   Used Ports  Configured Ports  MTU     Uplinks
switch           1024        20          1024              1500    vmnic9,vmnic4

VEM Agent (vemdpa) is running


~ # vemcmd show iscsi pinning
Vmknic   LTL      Pinned_Uplink    LTL
vmk2     49       vmnic4           21
~ #


vmk2 is pinned to the right vmnic. It supposed to be pinned with vmnic9


Thank you all for your help in advance.

Also, I need some guidance on binding vmnic to vmkernal in VEM.

Regards,

Faisal

2 REPLIES 2
sprasath
Beginner

Hi Faisal,

- Please paste the outputs of 'vemcmd show port' and 'vemcmd show port vlans'

- Mention the name of the VM that is working and the ones that are not working

- vmk/vmnic/vmhba binding is done at the ESX level. The VEM is not involved in this.

Thanks,

Shankar

Thanks Shankar for your response. I figured it out. It was port-channel configuration issue. I had connected the uplink directly into vlan 2. I should have  added that interface into port-channel.

Thanks again.

Content for Community-Ad
This widget could not be displayed.