VMs unable to pxe boot from any nic other than vnic0

[twongibm]

I got 4 vmnics on a setvice profile and have same vlans assigned. I am unable to pxe boot the vms on the esx hosts on a vlan on all the nics except vmnic0

any idea? The same vlans is configured on all the 4 nics

[Kirk J]

The Option ROM that Provides the PXE boot functionality is not VLAN aware, so you have to make sure your vlan in question is marked as 'native'.

VMK0 management interface generally assumes the MAC of the physical (1st vnic)

I'm assuming you are changing the LAN boot entry to reference the other vnic names, which is what is failing?

In your PXE boot server, are you specifying a specific mac address for the server?

Kirk...

[twongibm]

i have the VM booting fine from vnic0.

I tried removing vmnic0 and adding vnic1,2,3 and none of them work pxe booting.

vnic0 is exactly the same as vnic1,2,3

why does it not boot from vnic1,2,3?

[twongibm]

i did mark it as native but still does not work

[twongibm]

i do not think its related to vmk0

its only able to boot fro, vnic0. I moved vmnic0 to another vswitch where there is no vmk0 and still boots

[twongibm]

no I do not have a mac addresss reservation for the client. if thats what you are asking

my dhcp requests is going trhough a helper/forwarder

dhcp server is on different vlan than pxe boot clients

[Kirk J]

The vNICs 0,1,2,3 are not the same as they will all have different MAC addresses.

Which specific mac address are you referencing in your PXE/bootp/DHCP settings?

Why don't you try disabling vnic 0,,, and manually apply vnic0 mac address to vnic1.

Else, try resetting mac address for vnic0, and see if booting stops working for vnic0.

I'm wondering if your PXE server has a dynamic reservation for vnic0's mac , which may have been the first one used, that consumed a 'profile'

Kirk...

[twongibm]

I am not unstanding the question

My VM on the esx host is having the problem with pxe boot on the vswitch using vnic1,2,3

the dhcp server does not have any mac address restrictions or reservations

[Kirk J]

Sorry, thought we were talking about Host booting PXE.

So your vswitch port group will be what you tag the correct vlan with.  You won't use native vlan at the vnic/service profile level, as the traffic will get correctlly tagged at the vswitch port group level.  

I would compare mac address of the VMNICs you have associated with your vswitch and pxe boot port-group/network to make sure vnic1 is actually vmnic1, etc

Also, for the vswitch, what kind of teaming do you have setup (i.e. default originating port-id, ip-hash, etc).

You may want to check the guestVM (that you are wanting to pxe boot) nic's mac address, and confirm it is being learned on the right vlan on one of the FIs (ssh UCSM IP, connect nxos a|b, show mac address-table | in xxxx

Once you can confirm which physical vnic your VMNICs are active in your vswitch, please check the vnic 0,1,2,3 network config and make sure none of them have the pxe vlan marked as native.  Native is not needed as the vswitch port group is adding the needed vlan tag on to the frames. (this is assuming the pxe vlan is actually allowed on vnics 1,2,3 etc ;)

Kirk...

[twongibm]

i just reset the mac on vmnic0 in ucsm and tried pxebooting agaain on the vm and it still boots

[twongibm]

i just swapped the mac address from vmnic0 to vmnic1 and vice versa. 

Then i tried pxebooting the vm on vmnic1 and still wont boot. 

Then i tried pxebooting the vm on vmnic0 and it will boot. 

so the issue is its tied to vmnic0 not related the the physical mac address of vmnic

[Kirk J]

Sorry, I sent you on mac address chase, as I was incorrectly thinking we were referring to the ESXi host trying to PXE boot.

If you can only get the pxe based traffic to work out of one vnic,,, then there is something wrong with the vnic service profile setting (i.e. your other vnics are missing the needed vlan,,, or that vlan is marked as native).

Kirk...

[twongibm]

but the vmnics are not missing vlans. They all have native vlan assigned. they all have same vlans assigned