New C220m6 ESX can ping itself but not gateway

[Chris Austin]

I have 3 out of box C220 m6.  2 on the network and 1 waiting.

The 2 on the network have the same issue so it has to be something i missed in the config/install of ESX.

CIMC is reachable.

ESX can ping itself but not the gateway.  Both CIMC and ESX are on the same subnet.  ESX is connected to network via trunk port on core switch.  Simple config:

int gi1/7/33

switchport

switchport mode trunk

This is a refresh project so I mirrored the ESX/trunk port connecting to the current c220.

I used the onboard LAN1 port for connection on the c220.  In the core switch I can see the mac of that port in the mac address table but nothing shows up in arp.  The new is on the same subnet as the current and have the same gateway.  In esx, I see the vswitch that contains the 'onboard lan1'.   There is one port group created by default. It contains the default port groups VM Network and Management Network. Both current and new c220 seems to be configured the same.  However, I think I missed something in the initial configuration/install of ESX.

I connected to the c220 via monitor and keyboard to set up CIMC.  NIC mode for this is dedicated.  I then used that CIMC connection to install ESX.  I did the get esx7.0.3d from vmware and made sure to get the custom cisco iso.  It installed with no issues.  Once it was installed I logged in via KVM and added my IP, subnet, DNS and domain.  The vswitch was created automatically.

In CIMC, at one time I did not see any VNICs so I created 2 manually.  Rebooted and now there are 4.  Regardless they all have the same configuration.  I went through and mirrored the VNIC settings from the current to the new.  The current is a c220m4 and was set up 6 years ago so there are a couple of feature differences.  For the most part I went with default.

I am at a loss as why I can see the mac in the switch, esx can ping itself but cannot ping the default gateway.  Just to be clear, I cannot ping esx from the switch.  ESX cannot ping anything other than itself.  In my mind, it seems that the vswitch that has the onboard lan1 is not passing data to the right place.

Any ideas or suggestions as to what I can verify or do to determine what the issue is?

FYI I have a TAC case but I am impatient in waiting for the engineer to reach back to me.

[Wes Austin]

Can you send me a direct message with your TAC SR?

If you are learning the mac address for the correct vmnic on the switch in the correct VLAN, traffic seems to be making it to the right place. 

Can you confirm how your setup is configured from ESXi console? Are you tagging a VLAN? Are you using multiple vmnic for management? Where is the default gateway located you are trying to reach?

If you are not using the Cisco VIC, the vNIC in CIMC are not related to the issue.

[Chris Austin]

We are connected via trunk and are not tagging a vlan.  There are no vlans configured in esx.  For management I am using the dedicated mgmt port.  The default gateway is the core switch, which we are plugged into.  The switch that is seeing the mac is the default gateway.

[Chris Austin]

TAC suggested moving the cable from the dual 1G/10G onboard LAN1 port to a port on the 1gig adapter.  We moved the cable and updated ESX to use that vmnic.  We can see the mac of the new vmnic but still cannot ping the gateway or ping from the switch back to ESX.

[Chris Austin · Accepted solution]

Got this working by adding switchport trunk native vlan XXX to the switch interface configuration.  I cannot tell you why it was not required on the c220 that is in production but I am now able to ping the ESX host and the ESX host can ping out and resolve DNS.

I will just throw this out there as well..  I made the same add to the C220 that is still on the dual 1g/10 onboard LAN1 and it started working as well.