FCoE Issue with UCS C220 M3/VIC 1225/Nexus 5548up

[burke.brooks]

                   I have an issue with 3 UCS C220 M3's with VIC 1225's installed seeing FC Storage from a NetApp 2240-2 connected to a pair of Nexus 5548up switches. 

1.  On each of the Nexus 5548up switches I am using port 1/29 and 1/30 as native FC interfaces

2.  1/29 and 1/30 on each Nexus 5548up is connected to onboard FC interfaces on my NetApp 2240-2 HA cluster

3.  I can see the NetApp interfaces logging into the A-Side and B-Side fabric on the 5548's

4.  Each C220 M3 VIC 1225 is connected to an upstream Nexus 5548up Ethernet interface 1/3, 1/4, 1/5

5.  On the 5548's I have created VFC's tied to the ethernet interfaces that the C220 M3's are connected

6.  I am binding the VFC's to the ethernet interfaces that the VIC 1225's are connected to

7.  In the vSAN database I have the VFC's as well as the FC interfaces declared

8.  I am unable to see the C220 M3's log into the fabric at this time

Configuration for interfaces below

interface ethernet 1/3

switchport mode trunk

switchport trunk allowed vlan (production vlan's/FCoE vlan)

spanning-tree port type edge trunk

interface vfc3

bind interface Ethernet1/3

switchport trunk allowed vsan 536

no shutdown

vsan database

vsan 536 interface vfc3

vsan 536 interface fc1/29

vsan 536 interface fc1/30

[Cristian Munoz]

Hi Burke,

What is the status for the interfaces in your cisco card?

UCS #

UCS # scope chassis

UCS / chassis # show adapter

UCS / chassis # scope adapter x (where x is the slot for the Cisco VIC card)

UCS /chassis/adapter # show ext-eth-if

Is FC SAN boot option enabled in the vHBA?

What is the default VLAN?

Default VLAN = FCoE VLAN on switch

Chris,

[burke.brooks]

Chris,

Thank you for the response.  Status for the interfaces on the VIC 1225 is link up.  Default Vlan is 526 which is mapped to FCoE Vsan 536.  So Default Vlan on Adapter Card 1, vHBA tab is 526.  FC SAN Boot is not enabled either.

sho flogi data is showing the adapters from my NetApp FC interfaces logging into vsan 536.  No hosts are logging into the fabric.

[Cristian Munoz]

Hi Burke,

Just to verify, was FC SAN boot enabled?

Please also check if FIP is enabled in the adapter.

ucs-c210-m2 /chassis # scope adapter 1

ucs-c210-m2 /chassis/adapter # show detail

PCI Slot 1:

   Product Name: UCS VIC 1225

.................................................

   Current FW Version: X

   NIV: Disabled

  FIP: Enabled

Please let me know.

Chris,

[burke.brooks]

Chris, FC SAN Boot is not enabled and FIP is enabled on the adapter.

[Cristian Munoz]

Hi Burke,

Did it work after enabling FC SAN Boot?

[burke.brooks]

Chris, even after enabling FC SAN Boot on each FC interface on the vHBA I'm still not seeing a flogi for any host.  I had read another thread about QoS and checked to make sure I had the proper FCoE system qos commands on my 5k's.  On the ethernet interfaces I was missing the native vlan command and have corrected that as well and made that the default vlan on the vHBA tab in the CIMC as well.  Still no joy.  I appreciate your assistance in this for sure.

[padramas]

Hello,

I assume you have N5K configured for vPC and the switch interfaces connecting to VIC are part of vPC.

Please configure the "default VLAN " field of vHBA on VIC to FCoE VLAN ID

Reboot the server for the configuration changes to take effect.

Are you trying to boot from SAN or local boot the OS and access the storage ?

If you still have issues after the reboot, please provide output of following commands from one of the Nexus switch.

show version

show run fcoe
show vsan
show vlan fcoe
show interface vfc 3
show vpc

show int eth 1/3

Padma

[burke.brooks]

Padma, after continued troubleshooting on Friday we have come to the conclusion that the issue may reside with the OS on the server. We installed Windows Server 2012 r2 and think that the drivers for the OS need to be updated. This was the first install with Windows Server 2012 on a C-Series that we have done. Our normal configuration is with B200 M3's running vSphere. Thank you and I will post the resolution fairly soon.

Sent from Cisco Technical Support iPhone App

[padramas]

Thanks for the update

Just for your reference, example configuration is available here

FCoE Configuration Between VIC Adapter on UCS Rack Server and Nexus 5500 Switch

http://www.cisco.com/en/US/products/ps9670/products_configuration_example09186a0080c20a0c.shtml

[craigbruenderman]

Padma, I can't seem to scope to the P81e VIC adapter at all on my C200 M1. I've run HUU to update everything, but it's like the CIMC doesn't acknowledge the VIC card existing, even though it functions in the OS. I'd still like access to it in order to change the FCoE VLAN.

See screenshots

[padramas]

Hello,

You can update the settings of VIC card from CIMC  GUI. 

If you cannot, I would suggest you to open TAC service request for it. 

Thanks

[burke.brooks]

Problem solved, once we updated host firmware and applied network/storage drivers within Server 2012 r2 OS both adapters were recognized. Host is now logged into storage fabric

Thank you everyone for your help and support!

Sent from Cisco Technical Support iPhone App