UCS and ESXI vMotion failure

[Jean Paul Enerst]

Hi Pros,

Need  your help to figure out an issue here. The issue is between UCS and Esxi servers. Apparently when we tried to vmotion a guest server from  one Esxi host to another, we lost network connectivity during the migration. And this  issue only affected one specific  VLAN(200). The fix is usual disconnect and re-connect the adaptor in vCenter. VMware engineer has claimed it's an uplink issue. I have double check and triple check the config in the UCS side but can't see anything wrong. Below is the configuration for one of the host.... hopefully one of you can spot something or give me a plausible answer.

1. Fabric-A

interface Ethernet1/17
description U: Uplink
pinning border
switchport mode trunk
switchport trunk allowed vlan 1,100-105,200,230-231
channel-group 1 mode active
no shutdown

interface Ethernet1/18
description U: Uplink
pinning border
switchport mode trunk
switchport trunk allowed vlan 1,100-105,200,230-231
channel-group 1 mode active
no shutdown

interface Vethernet935

description server 2/3, VNIC vmnic2
switchport mode trunk
hardware vethernet mac filtering per-vlan
no pinning server sticky
pinning server pinning-failure link-down
switchport trunk allowed vlan 101,104,200,230-231
bind interface port-channel1293 channel 935
service-policy type queuing input default-in-policy
no shutdown

2. Fabric B

interface Ethernet1/17
description U: Uplink
pinning border
switchport mode trunk
switchport trunk allowed vlan 1,100-105,200,230-231
channel-group 1 mode active
no shutdown

interface Ethernet1/18
description U: Uplink
pinning border
switchport mode trunk
switchport trunk allowed vlan 1,100-105,200,230-231
channel-group 1 mode active
no shutdown

interface Vethernet936
description server 2/3, VNIC vmnic3
switchport mode trunk
hardware vethernet mac filtering per-vlan
no pinning server sticky
pinning server pinning-failure link-down
switchport trunk allowed vlan 101,104,200,230-231
bind interface port-channel1292 channel 936
service-policy type queuing input default-in-policy
no shutdown

Thanks alot guys

[Wes Austin]

When you lose connectivity, are you still learning the vmotion vmkernel mac address on the fabric interconnects in VLAN 200? Is it learned on the upstream switch?

What is the upstream switch running configuration for ports connecting to UCS?

[Jean Paul Enerst]

Morning Wes,

Thanks for your reply.

Vmware engineer has taken a tcpdump but did not see traffic coming from the VLAN. And the interfaces connected to the FIs are trunked and pass all the VLAN down., as per below.

version 7.1(4)N1(1)

interface Ethernet1/17
description *** colo624801 e1/17 ***
switchport mode trunk
channel-group 1 mode active
no shutdown

colo554801# sh run int Eth1/18

!Command: show running-config interface Ethernet1/18
!Time: Mon May 16 08:10:18 2016

version 7.1(4)N1(1)

interface Ethernet1/18
description *** colo624801 e1/18 ***
switchport mode trunk
channel-group 1 mode active
no shutdown

However, I think that i might figure it out though.  All the VLAN in the previous config have their interfaces, and layer 3 routing configured in the N5K switches and have HSRP configured as well, but VLAN 200 is a DMZ network. Therefore, its interface is configured at the ASA and does not have HSRP. Thus, when one of the N5K reboots or a path changes, in the case of vmotion, for example, we lost connectivity. I did test a bit further, beyond the VM environment, and had the same result with a hard-wired PC! When the path changes, we lost connectivity.

Therefore, I have to come up with a solution to load balance the traffic from the ASA itself before sends it to the N5Ks downstream. Maybe I will have to configure HSRP in the ASA, but if you have any other solution/suggestion, I'll be happy to explore it.

Thanks,

[Jean Paul Enerst]

Hey Gents,

Quick update here, and I wanted to pick your brain before i proceed with my plan. As i mentioned, the VLAN that i am having issues to seamless vmotion to has its interface at the ASA which is in Failover mode, and all other VLAN has their interface in the N5K downstream of the ASA and do have HSRP configure as well. 

I was planning to configured HSRP in the ASA, but ASA does not support HSRP. Now, i am planning to fall back to the N5K, but the issue here is the N5K has received EIGRP route for vlan200 from the ASA. Now, this is a prod network, i can't plan too much with it. So my question is can i still configure an SVI in the N5K for VLAN 200 even though that i received EIGRP route for that VLAN from the ASA? I have a feeling that may create a routing loop for that VLAN which i don't want.

Thanks,

[Jean Paul Enerst · Accepted solution]

This issue has been fully resolved....no HSRP configured as the ASA is already in failover mode. Issue was layer 2, downsstream switches were not configured with the required vlan, adding the VLAN in questions and configured trunk port properly had fixed this issue and a few other ones as well.

Thank you all