Solved: UCS: Can't reach ESXi server after install on UCS B230 M2 Blade

Luke Akpor · ‎03-29-2013

Hi,

I installed ESXi 5.1.0 on UCS B230 M2 Blade, after IP address Configuration then appears the below:

"Download tools to manage this host from

But i can't ping this IP Address and I can't reach it by http.

I use the same IP address as KVM IP pool, I can ping KVM IP addresses.

I plugged and configured management ports only on UCS Fabric interconnect; and uplink ports looks ok. I can ping the SP management and the Server IP addresses from anywhere on the network, but cant ping the ESXi IP address which resides in the same subnet.

I guess I am missing something but can’t really work this out?

Can someone please assist?

Thanks

Luke Akpor · ‎04-07-2013

Hi All, thanks for response.

Just thought I share what the issue was eventually after raising a TAC case.

Anyway following trobleshooting with TAC, there is a known issue with duplicate mac address with vMware which was causing the issue I had.

Once the vmknic was deleted using the ESXi Shell and re-created, the mac address changed from the Cisco range I had of 00:25:B5 to 00:50:56 which belongs to vMware.

After this changed the issue was resolved.

Here is the link to the vMware known issue incase it helps someone else.

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1031111&sliceId=1&docTypeID=DT_KB_1_1&dialogID=770362736&stateId=1%200%20770392599

Thanks for all our response.

View solution in original post

padramas · ‎03-29-2013

Hello,

to start with, make sure you have added the right VLANs to vNIC via service profile and are correctly tagged within ESXi vswitch

esxcfg-vmknic -l

esxcfg-vswitch -l

Verify that the vmkernel mac address maps to correct VLAN on FI server port.

Padma

Luke Akpor · ‎03-29-2013

Hi Padma,

Thanks for you response.

But the issue is I can't even log on to the ESXi as its un-pingable.

Even the Fabric Interconnect can't ping the ip address even when it sits on the same subnet.

Is there anything else I can check?

I have the vNIC configured with Mgt-Vlan and default vlan, with the default vlan being the native vlan.

Is this wrong?

Thanks

padramas · ‎03-29-2013

Hello,

You can SSH into FI and check the mac address table from NX-OS prompt.

For ESXi, you can use KVM console to execute those commands

Padma

dfilenko · ‎03-29-2013

Hi Luke,

You can login to ESX CLI using KVM. In troubleshooting tools, enable console and press alt+F1 to get into CLI prompt on ESX host.

Few things to check:

do you have any native vlan configuration on vNIC in service profile?
if you do, is this the same vlan that you are using tryting to communicate to network?
If it is, make sure that you are not adding vlan in ESX management vmk configuration.
Login to UCSM console, switch to NXOS (connect nxos) and check for vmk0 mac address in mac address table.
If it's there, check check for the same in upstream switch.

Luke Akpor · ‎03-29-2013

Thanks Padma and Dmitri for you response;

Answers to your questions:

do you have any native vlan configuration on vNIC in service profile? Ans - Yes (Mgmt-Vlan)
if you do, is this the same vlan that you are using tryting to communicate to network? Ans- Yes
If it is, make sure that you are not adding vlan in ESX management vmk configuration.Ans - No
Login to UCSM console, switch to NXOS (connect nxos) and check for vmk0 mac address in mac address table. Ans - On the mac address table as shown below
If it's there, check check for the same in upstream switch Ans - Yes in upstream switch as well

Guys please see some output and screen shot which may help place me in the right direction

This is the Eth0 interface on the Server where the ESXi is installed

N6K-FI1-A(nxos)# sh int vethernet 758

Vethernet758 is up

Bound Interface is port-channel1280

Hardware: Virtual, address: 547f.eec0.7460 (bia 547f.eec0.7460)

Description: server 1/1, VNIC eth0

Encapsulation ARPA

Port mode is trunk

EtherType is 0x8100

Rx

0 unicast packets 1 multicast packets 5 broadcast packets

6 input packets 394 bytes

0 input packet drops

Tx

0 unicast packets 0 multicast packets 33 broadcast packets

33 output packets 11550 bytes

0 flood packets

0 output packet drops

N6K-FI1-A(nxos)#

N6K-FI1-A(nxos)# sh mac address-table

Legend:

* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC

age - seconds since last seen,+ - primary entry using vPC Peer-Link

VLAN MAC Address Type age Secure NTFY Ports

---------+-----------------+--------+---------+------+----+------------------

* 10 0025.b500.009f static 0 F F Veth758

N5K# sh mac address-table

Legend:

* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC

age - seconds since last seen,+ - primary entry using vPC Peer-Link

VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID

---------+-----------------+--------+---------+------+----+------------------

* 10 0025.b500.007f dynamic 210 F F Po11

N5K# sh mac address-table

Legend:

* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC

age - seconds since last seen,+ - primary entry using vPC Peer-Link

VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID

---------+-----------------+--------+---------+------+----+------------------

* 10 0025.b500.007f dynamic 10 F F Po11

N6K-FI1-A(local-mgmt)# ping 192.168.60.101

PING 192.168.60.101 (192.168.60.101) from 192.168.60.162 : 56(84) bytes of data.

From 192.168.60.162 icmp_seq=1 Destination Host Unreachable

From 192.168.60.162 icmp_seq=2 Destination Host Unreachable

From 192.168.60.162 icmp_seq=3 Destination Host Unreachable

--- 192.168.60.101 ping statistics ---

4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3011ms

, pipe 3

N6K-FI1-A(local-mgmt)#

dfilenko · ‎03-29-2013

N5K upstream is not learning MAC 0025.b500.009f and this might be the problem.

From ESX configuration, check what MAC got assigned to vmk0 port (usually it's MAC address of vmnic0). Then make sure that same MAC is learned on upstream N5K switch.

Luke Akpor · ‎03-29-2013

Hi Dmitri,

Thanks for your response,

I am unable to check the MAC that got assigned to the vmk0 port on the ESXi as using the Alt + F1 just shows a console blinking with any commands not getting accepted.

I am stuck really. Do I have to re-install the ESXi?

Thanks

dfilenko · ‎03-29-2013

No need to reinstall ESX. By default, CLI is disabled.

Go back into main screen (Alt+F2), log-in and select Troubleshooting Options->Enable ESXi Shell. This will enable ESX console.

Luke Akpor · ‎04-02-2013

Hi Guy,

Sorry for late response as I needed to be in the office to try this out again.

Now I have checked the ESXi configuration and the vmnic0 is binding to the vNIC0 interface of the blade server.

I have also configured the VLAN on the ESXi to vlan 10 which is the management VLAN but still unable to ping the rest of the network.

I then changed it to VLAN 4095 but still unable to ping.

The thing i am not sure about is why am I not seeing the vNIC0 mac-address on the upstream switches (N5K)?

Any idea what I need to check for again please?

Thanks for your help

edgar.morales · ‎04-02-2013

Luke...i am not a networking guru but had a thought. I see you have vlan 10 set as the native vlan for the management vnic under this server profile. Do you also have the native vlan set to vlan 10 on the upstream 5K from the FIs?

Under the LAN, LAN cloud and VLANs, do you have VLAN 10 set as native as well. If so, try unchecking the "native vlan" option from the actual management vnic under that specific service profile and try pinging the ESX Host.

==== Disregard my post....If I would have read all the previously posts I would have seen that Dmitri asked similar questions and you had answered them. ======

Luke Akpor · ‎04-02-2013

Hi Edgar,

Thanks for the response. You are right with your point as per the native vlan note completely set across the network. I didnt do this on the 5K. But once I did it I still could not ping the ESXi with the vNIC which has the VLAN 10 as native vlan. But surprisingly I could ping another ESXi which is using the default VLAN 1 as the native vlan.

I am still trobleshooting this so any further assistance will be much appreciated.

Thanks

edgar.morales · ‎04-03-2013

Luke,

Just to confirm you said you have logged into the KVM on the ESXi host and verified you are tagging VLAN 10 on the managment network (vmnic0)? If this is true and you have the native vlan set to vlan 10 on the vnic in the service profile of UCS and vlan 10 set as the native vlan upstream this may be your issue.

You should be able to simply remove VLAN 10 from vmnic0 on the ESXi host via the KVM console and traffic should flow normally.

Luke Akpor · ‎04-03-2013

Thanks Edgar,

This is the case. I have no VLANs configured on the ESXi host now.

Can I confrm that when I start creating extral vlans on the ESXi , e.g for vMotion etc, I still have to leave this option for VLAN ID as unset?

Thanks very much for your help.

edgar.morales · ‎04-03-2013

Luke

As long as the you have the native vlan setup appropriately then yes you would leave the management vlan blank on the ESXi side. When creating additional port groups in vmware (vmotion, iscsi, vm traffic etc.) you would specify the appropriate vlans rather than leave them blank as these services should be on seperate vlans (not part of vlan 10 native)

There are a number a ways to set this up. Another way of setting it up would be to not set the native vlan on the vnics in each service profile and tag the vlans on the ESXi hosts by specifying a vlan for each Port Group you create (ie. vmotion, iscsi, vm traffic etc.)