FCoE issue in ucs with 5548up

[GouthamReddy Mamidi]

Hello

I'm facing issue with FCoE ..

I'm using Fi-6248 UP nexus 5548UP

from 5k to Fi it is running with vpc and I have created two iseland SAN-A and SAN-B .

SAN-A is connected between 5548Up and FI-6248Up as well as SAN-B is connected between others two 548Up and FI-6248Up

I'm Using FCoE from both the sides to boot server from SAN..

I'm facing the issue like FCoE up link is not available ..from UCS

My Fi's are running with NPV mode .Initially before booting the server I can see the flogi of FI's in both the 5k's 5548 UP but when I try to boot the server those FI's flogi's are replaced with server vhba flogi.I should get both the flogi's Fi-A and Fi-B in both 5k 's but both sides Fi flogi's are replacing. with server vhba-0 and vhba-1...

from UCS it is showing FCoE uplink is not available...

FCoE Interface State is Intermediate and faults I'm getting F1082 and F1083..

[AFROJ AHMAD]

Hi ,

It must be a configuration issue ..

Please make sure you are using different VSAN IDand FCoE vlan ID on both N5ks. 

and needs to configure two different vsan on both sides for this to work.

Thanks-

Afroz

***Ratings Encourages Contributors ****

[GouthamReddy Mamidi]

Hi Afroz ,

I'm using two different vsan and vlan on both the sides..

with same configuration It was working but suddenly it is not working vfc is giving big issue..

I applied same configuration on other devices it is working .

I don't think so it is configuration issue.

Thanks-

Goutham

[Cristian Munoz]

Hi Goutham, 

Since you have running VPC from the N5K to the FI, I don't see this is supported. It might work but you might be facing some issues.  You will need to use separtate links for FCoE. 

Do you see the same issue with FCoE dedicated link?

I attach some really helpful articles that can help you also to review the config. 

UCS FCoE Uplink Configuration Example

http://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-manager/116188-configure-fcoe-00.html

UCS Multihop FCoE In Under an Hour.

http://blogs.cisco.com/datacenter/ucs-multihop-fcoe-in-under-an-hour

Regards,

Chris

[Walter Dey]

- what type of uplink ? FCoE (only FCoE or storage) uplink ? or Unified (FCoE and IP) uplink ?

- vpc ususally breaks the FC fabric A / B separation ?

- you will see 2 flogi from FI to N5k link; and 2 flogi from vhba-0 and 1 from your servers

- you need to setup zoning with this vhba-0 and 1

- how does the boot policy look like ?

As a reference, have a look at

http://www.cisco.com/c/en/us/support/docs/switches/nexus-5000-series-switches/116248-configure-fcoe-00.html

[GouthamReddy Mamidi]

Hi Walter Dey

This only FCoE uplink .Intially NPV(FI) switch flogi is coming on N5K but when blade boots only server flogi is coming on N5K flogi is not showing. I did flapping the physical interface from 5k which is connected to NPV switch (FI).After flapping NPV flogi is coming and server flogi is not coming up .

But only one flogi is coming up on N5K .This NPV flogi will comes up when we flapping the physical interface and at the end again only server flogi coming up on the N5K...

I appilied the same configuration on other switches which are same model... it was working on them..

I can replace the switches but this is not permanent solution...

Thank you

[Walter Dey]

Something basic must be wrong; did you check if your vhba0 and 1 are in the proper VSAN ?

[GouthamReddy Mamidi]

Hi Walter Dey,

They are in proper VSAN and VLAN

I received the vhba flogi's of servers on both 5k vhba0 5k1 and vhb1 5k2.

But NPV (Fi-A and Fi-B)flogi is not coming UP when server try to Boot.I can see ony vhba0 and vhba1 flogi.

I never face this issue ..

when I applied same configuration on other devices it is working..

Thanks-

Goutham
 

[Walter Dey]

Can you please post a screenshot of "configured and Actuaal Boot order".

Problem could well be on Zoning configuration and / or lun masking / mapping on the storage array.

[GouthamReddy Mamidi]

I'm attaching boot order and configuration on 5k and UCS

I'm getting errors like F1082 and F1083 fcoe link status is showing Intermediate.

I check the vif-path also it is pinned and showing up for both vhba's fc0 and fc1 pinned to e1/5 on the both the sides .I changed fcoe uplink but still same issue on other links also.

In active zoning both Initiator and Target showing with " * "  indication.

I'm getting shell prompt in KVM console.

Thank you-

Goutham

[Walter Dey]

Q. you always talk about FCoE boot problem; did you sucessfully install the OS (which one) on the boot lun ?

[GouthamReddy Mamidi]

Hello walter Dey,

Yes I installed ESXi .It is working with FC boot but not FCoE....

I directly connected to 5k and converted fc ports it is working .

But when I try to with FCoE It is not booting  with the same ports ...

Thank you-

Goutham

[Walter Dey]

Hi Goutham

The problem must be the boot policy ! Do you have end to end FCoE ? (meaning multihop FCoE)

Please post configured and actual boot policy ?

see attachment

[GouthamReddy Mamidi]

Hello Walter Dey,

This is the configuration I did on N5k

vsan database
vsan 10
exit

vlan 20
fcoe vsan 20
exit

interface e1/5
sw mode trunk
sw trunk allowed vlan 20
spanning-tree port type edge trunk
no sh

int vfc 15
bind interface e1/5
switchport mode f
switchport trunk allowed vsan 20
no sh

vsan database
vsan 10 interface 15

from UCS side I enable port number 5 as FCoE uplink

and I mapped with FCOE vlan/vsan 20

when I converted this ports are fc ports it is booting.

Thank you-

Goutham

[Walter Dey]

I would map FCoE vlan to a higher number, not 20, but maybe 3020 !