UCS Uplinks to Core Network Issue

ssaunders42 · ‎08-19-2013

Hi,

we recently purchased a UCS setup to run our EXS hosts on. we have two 6248 Fabric switches and one 5108 chassis. I have setup for the most part everything I would think I needed to get a VM Host up and running for testing. Created IP pool for KVM connection, which works and I can gain access to the ESX console from this IP. I created all the VLAN's and setup Port-Channels to my core 6506's. here is a run down of the physical devices and software running on them.

dual 6506 (VSS) - w/ VS-SUP2T-10G, x6816-10GE line card (s2t54-adventerprisek9-mz.SPA.151-1.SY1.bin)

UCS running latets running 2.1(2a)

I setup ports 13/14 on the 6200's to be a port channel to one of the 6500's

6500 Interface setup (ESX host mgmt IP is on vlan 210)

interface Port-channel10

description Uplink FI-A 13/14

switchport

switchport trunk allowed vlan 110-114,120,130,140,145,210-214,220,230,240,245

switchport mode trunk

interface TenGigabitEthernet1/1/1

description Uplink-UCS-A

switchport

switchport trunk allowed vlan 110-114,120,130,140,145,210-214,220,230,240,245

switchport mode trunk

channel-group 10 mode active

interface TenGigabitEthernet1/1/2

description Uplink-UCS-A

switchport

switchport trunk allowed vlan 110-114,120,130,140,145,210-214,220,230,240,245

switchport mode trunk

channel-group 10 mode active

UCS setting

interface Ethernet1/13

description U: Uplink

pinning border

switchport mode trunk

switchport trunk native vlan 210

switchport trunk allowed vlan 1,110,210-213,220,230

channel-group 10 mode active

no shutdown

interface Ethernet1/14

description U: Uplink

pinning border

switchport mode trunk

switchport trunk native vlan 210

switchport trunk allowed vlan 1,110,210-213,220,230

channel-group 10 mode active

no shutdown

interface port-channel10

description U: Uplink

switchport mode trunk

pinning border

switchport trunk native vlan 210

switchport trunk allowed vlan 1,110,210-213,220,230

speed 10000

Issue is I cannot ping/connect the ESX host that is sitting on the blade. All links are green and they show no errors. 6200's are in FC Switch mode because we are directily connecting it to a VNX SAN. Network Adapter shows a status of connected from the ESX host on the correct vnic. any help would be greatly appreciated.

shawn

padramas · ‎08-19-2013

Hello Shawn,

It is all L2 within UCS and we need to trace the mac.

Please note the vmkernel interface mac address and VLAN configuration virtual switch.

Is UCS vNIC ( in service profile ) configured with corresponding VLAN ?

Marked as native if vSwitch is not performing VLAN tagging ?

From FI NX-OS, can you please check the mac address table and VLAN mapping ?

connect nx-os

show mac address-table

Check on upstream Cat6K, the mac address is learned on the right VLAN.

On a side note, as FI Ethernet is operating in end host mode, you can configure Cat6K switch ports in STP portfast trunk mode.

Padma

ssaunders42 · ‎08-19-2013

Hi Padramas - the vnic is set to the correct vlan and that vlan is set to native. I checked the mac address table on the FI and vlan 210 shows up with the correct mac address of the ESX host. the same mac address is not showing up in the table on the 6500's. I tried turning on STP on the 6500's but it just shows them as edge now since they are trunk ports.

ssaunders42 · ‎08-19-2013

I recreated the port channels and the interfaces just to clean everything up. I now see the mac address on both the FI and the 6500's. I still connot ping the devices though. From the FI local-mgmt interface I can ping the default gateway network for VLAN 210, but not the ESX. Same thing from the 6500's as well as the ESX hosts them selves. oh, and I built another ESX host and put its mgmt nic on the same vlan, they cannot ping eachother. very odd.....

Walter Dey · ‎08-19-2013

Most likely, this is a trivial cause

When you setup the ESXi host, specifiying IP, mask, gwy, did you specify a Vlan id ??

If you did, this might be your problem. Either you specify a vlan id, and remove native vlan, or have no vlan id, and native vlan set

LukaszTJB · ‎08-23-2013

Hey shawn,

to me it looks like the vlan 210 is configured as native vlan on the Fabric interconnects (untagged), while on the 6500 Trunks vlan 210 is tagged and the default vlan1 is native. Could that be the issue ? How is the vlan 210 configured on the management vnic for the ESXi host on the UCS (tagged or untagged) and how is the configuration inside the ESXi OS for the Management VMkernel ?

Greetings

Lukasz

ssaunders42 · ‎08-23-2013

I tried removing the vlan setting on the ESX and only having it set as native on the UCS but it still wouldnt work.

this seems odd, but I switched everything over to the mgmt network and it started working. not sure if that is a must for hosts on the UCS. I was then able to create a test server on the ESX and give it an IP on the 210 network and it works just fine.

not going to worry about this, it is not like we will have over 200 esx hosts. we only have 6 right now.

thank you for everyones help though!