Host loses FcoE storage Nexus 1010 and 5000

[anthony_douwes]

Host loses connectivity to SAN after adding both adapters to system_uplink.

Qlogic dual port CNA cards - ql8152

vsphere 4.1

VEM/VSM 1.3(b)

Found I can resolve by not binding the mac address to the vfc, but rather to the port channel - however I can no longer boot from san.

[Robert Burns]

Why are you binding the MAC address?  You should just need to bind the vfc to the Port Channel.

I assume you're using a vPC from your N5K's?  Are you using LACP or statuc Po?

Rob

[wickusb]

Hi Robert

Our setup is as follows.

2 x 6500's in VSS mode

2 x nexus 5020

2 x Nexus 1010 + 2 1000v software installed.

vsan 2 is created on 5k-1

sh vlan fcoe

Original VLAN ID        Translated VSAN ID      Association State
----------------        ------------------      -----------------

      30                        2                Operational

vsan 3 is created on 5k-2

sh vlan fcoe

Original VLAN ID        Translated VSAN ID      Association State
----------------        ------------------      -----------------

      31                        3                Operational

Each of the nexus 5020 is connected via 10gig links to each 6500 and all the links is in a port-channel. the 2 nexus 1010 is setup in active/standby and connected to nexus 2k fabic extender which is also connected to the 5k's.

Each ESX host is connected to a 5020 via the qlogic cna 10 gig card.

We've setup port-channels for each of the 10gig links and added vpc on the port-channel.

i will paste the interface config, port-channel config and vfc config.

Switch 5k-1

interface Ethernet1/5
  untagged cos 0
  switchport mode trunk
  switchport trunk allowed vlan 1,10-30,101-104
  spanning-tree port type edge trunk
  channel-group 5 mode active

interface port-channel5
  switchport mode trunk
  vpc 5
  switchport trunk allowed vlan 1,10-30,101-104
  spanning-tree port type edge trunk
  spanning-tree bpdufilter enable
  speed 10000

interface vfc5
  bind interface port-channel5
  no shutdown

Switch 5k-2

interface Ethernet1/5
  untagged cos 0
  switchport mode trunk
  switchport trunk allowed vlan 1,10-31,101-104
  spanning-tree port type edge trunk
  channel-group 5 mode active

interface port-channel5
  switchport mode trunk
  vpc 5
  switchport trunk allowed vlan 1,10-31,101-104
  spanning-tree port type edge trunk
  spanning-tree bpdufilter enable
  speed 10000

interface vfc5
  bind interface port-channel5
  no shutdown

In the above scenario we can see both storage paths to the san on the storage adapters. However when the ESX host is rebooted the boot device is no longer available.

If u go into the fastutil and scan for fibre devices nothing is found. We then change the bind interface po5 to bind to the mac address. Rescan for fibre devices is successfull. restart the server. boot from san is working again.

once the ESX host is up and we go to the storage adapters. one of the hba's shows 0 targets the other hba shows 2 targets. When the binding on the vfc is changed back to port-channel the storage paths recovers and we can see 2 targets on both hba's.

Let me know if there is anymore info u need.

Thanks

[anthony_douwes]

Robert

The only way we can get the server to boot from SAN using the Qlogic BIOS is to bind the mac address - at least thats what we've found so far.

Once the server is booted and the dvswitch becomes active, then the paths to storage disappear - on some servers all paths, and some servers only paths via one adapter.

Also if a server is not configured to use the dvswitch or only has one adapter in the dvswitch then storage access is ok.

My colleague Wickus replied with the cisco config:

Anthony