cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
801
Views
5
Helpful
2
Replies
Highlighted
Beginner

Nexus 1000v. Only first VM is able to ping outside vlan interface.

Hello everyone,

I am trying to setup Nexus 1000V VSM and VEM.  My ESXi host has four network ports. For management I am using only one of the ports on the quad nic card as a trunk.  It is trunked to Cisco 3750 switch.  Another one of the quad nic cards is connected to the VDS.  Both ports are in port channel 1.  I moved three VMs from vSwitch to vds . But somehow only one vm which I powered on first is able to ping outside vlan interface, others won't. They only work after I issued command in port-channel 1 shut and no shut. When I reboot the host, it is back to where the first vm can ping outside vlan interface.

Here is the configuration for the Nexus port profiles and the 3750 switch.
port-profile Unused_Or_Quarantine_Uplink
type: Ethernet
description: Port-group created for Nexus1000V internal usage. Do not use.
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  shutdown
evaluated config attributes:
assigned interfaces:
port-group: Unused_Or_Quarantine_Uplink
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile Unused_Or_Quarantine_Veth
type: Vethernet
description: Port-group created for Nexus1000V internal usage. Do not use.
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  shutdown
evaluated config attributes:
assigned interfaces:
port-group: Unused_Or_Quarantine_Veth
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VMK-Control
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
assigned interfaces:
  Vethernet2
port-group: VMK-Control
system vlans: 9
capability l3control: yes
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN2
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  capability iscsi-multipath
  switchport access vlan 2
  no shutdown
evaluated config attributes:
  switchport mode access
  capability iscsi-multipath
  switchport access vlan 2
  no shutdown
assigned interfaces:
  Vethernet1
port-group: VM_VLAN2
system vlans: 2
capability l3control: no
capability iscsi-multipath: yes
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN3
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 3
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 3
  no shutdown
assigned interfaces:
  Vethernet3
  Vethernet4
port-group: VM_VLAN3
system vlans: 3
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN6
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 6
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 6
  no shutdown
assigned interfaces:
  Vethernet7
port-group: VM_VLAN6
system vlans: 6
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN8
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 8
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 8
  no shutdown
assigned interfaces:
port-group: VM_VLAN8
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile VM_VLAN9
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
evaluated config attributes:
  switchport mode access
  switchport access vlan 9
  no shutdown
assigned interfaces:
  Vethernet5
  Vethernet6
port-group: VM_VLAN9
system vlans: 9
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

port-profile uplink_Connection
type: Ethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
  switchport mode trunk
  switchport trunk allowed vlan 1-10
  channel-group auto mode on mac-pinning
  no shutdown
evaluated config attributes:
  switchport mode trunk
  switchport trunk allowed vlan 1-10
  channel-group auto mode on mac-pinning
  no shutdown
assigned interfaces:
  port-channel1
  Ethernet3/5
  Ethernet3/10
port-group: uplink_Connection
system vlans: 1-3,6,8-10
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vservice: no
port-profile role: none
port-binding: static

********************************3750 switch configuration************************************

Switch>en
Switch#show run
Building configuration...

Current configuration : 3862 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname Switch
!
!
no aaa new-model
switch 1 provision ws-c3750-24ts
system mtu routing 1500
ip subnet-zero
ip routing
!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
spanning-tree vlan 9 priority 4096
!
vlan internal allocation policy ascending
!
!
!
!
interface Port-channel1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
!
interface Port-channel2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2
switchport mode trunk
!
interface FastEthernet1/0/1
!
interface FastEthernet1/0/2
switchport access vlan 2
spanning-tree portfast
spanning-tree port-priority 0
!
interface FastEthernet1/0/3
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2
switchport mode trunk
channel-group 2 mode active
spanning-tree portfast
!
interface FastEthernet1/0/4
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2
switchport mode trunk
channel-group 2 mode active
spanning-tree portfast
!

!
interface FastEthernet1/0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
channel-group 1 mode active
!
interface FastEthernet1/0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
channel-group 1 mode active
spanning-tree portfast
spanning-tree port-priority 0
!
interface FastEthernet1/0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2,3,6,9
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet1/0/1
!
interface GigabitEthernet1/0/2
!
interface Vlan1
no ip address
!
interface Vlan2
ip address 20.12.162.1 255.255.255.0
!
interface Vlan3
ip address 20.12.161.13 255.255.255.0
!
interface Vlan4
ip address 20.12.165.1 255.255.255.0
!
interface Vlan6
ip address 20.12.166.1 255.255.255.0
!
interface Vlan9
ip address 20.12.167.1 255.255.255.0
!
interface Vlan10
ip address 20.12.168.1 255.255.255.0
!
router eigrp 100
network 20.12.161.0 0.0.0.255
network 20.12.162.0 0.0.0.255
network 20.12.165.0 0.0.0.255
network 20.12.166.0 0.0.0.255
network 20.12.167.0 0.0.0.255
network 20.12.168.0 0.0.0.255
no auto-summary
!
ip classless
ip route 0.0.0.0 0.0.0.0 20.12.165.2
ip http server
!
!
!
control-plane
!
!
line con 0
line vty 0 4
login
line vty 5 15
login
!
end

Switch#

**************************Trunk-status*******************************************

switch# show int trunk

--------------------------------------------------------------------------------
Port       Native  Status        Port
           Vlan                  Channel
--------------------------------------------------------------------------------
Eth3/5     1       trnk-bndl     Po1
Eth3/10    1       trnk-bndl     Po1
Po1        1       trunking      --

--------------------------------------------------------------------------------
Port       Vlans Allowed on Trunk
--------------------------------------------------------------------------------
Eth3/5     2-3,6,8-9
Eth3/10    2-3,6,8-9
Po1        2-3,6,8-9

--------------------------------------------------------------------------------
Port       Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------
Po1        none

************************************MAC pinning**************************************************************

VEM modules are loaded

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         128         4           128               1500    vmnic5
vSwitch1         128         4           128               1500    vmnic8
vSwitch2         128         1           128               1500
DVS Name         Num Ports   Used Ports  Configured Ports  MTU     Uplinks
switch           1024        20          1024              1500    vmnic9,vmnic4

VEM Agent (vemdpa) is running


~ # vemcmd show iscsi pinning
Vmknic   LTL      Pinned_Uplink    LTL
vmk2     49       vmnic4           21
~ #


vmk2 is pinned to the right vmnic. It supposed to be pinned with vmnic9


Thank you all for your help in advance.

Also, I need some guidance on binding vmnic to vmkernal in VEM.

Regards,

Faisal

2 REPLIES 2
Highlighted
Beginner

Nexus 1000v. Only first VM is able to ping outside vlan interfac

Hi Faisal,

- Please paste the outputs of 'vemcmd show port' and 'vemcmd show port vlans'

- Mention the name of the VM that is working and the ones that are not working

- vmk/vmnic/vmhba binding is done at the ESX level. The VEM is not involved in this.

Thanks,

Shankar

Highlighted
Beginner

Nexus 1000v. Only first VM is able to ping outside vlan interfac

Thanks Shankar for your response. I figured it out. It was port-channel configuration issue. I had connected the uplink directly into vlan 2. I should have  added that interface into port-channel.

Thanks again.

CreatePlease to create content
Content for Community-Ad

Cisco COVID-19 Survey