configure to bring Fibre Channel between UCS Fiber Interconnect and Nexus 5548UP.

[Shuja Tirmizi]

using following  configuration between Nexus5k and UCS FI. the channel status comes to ini and change to notConnected .

-------------------------------------------------------------------------------
fc1/31     431    F      on      notConnected     swl    --           100
fc1/32     431    F      on      notConnected     swl    --           100

-----------------------------------------------------------------------------------

 System version: 5.2(1)N1(1b)

fcoe                  1         enabled
fex                   1         enabled
fport-channel-trunk   1         enabled
interface-vlan        1         enabled
lacp                  1         enabled
lldp                  1         enabled
npiv                  1         enabled
sshServer             1         enabled
tacacs                1         enabled
telnetServer          1         enabled
udld                  1         enabled
vpc                   1         enabled

interface san-port-channel 100
  channel mode active
  switchport mode F
  switchport trunk allowed vsan 431

interface fc1/31
  switchport description "SAN PO to UCS -A"
  channel-group 100 force
  no shutdown

interface fc1/32
  switchport description "SAN PO to UCS -A"
  channel-group 100 force
  no shutdown

interface fc1/31
  switchport mode F
interface fc1/32
  switchport mode F

Please help

[Manuel Velasco]

Syed,

If you bring the interfaces up without the port channel does this works?

What firmware is your UCS running?

ssh into one of the fabrics interconnects and and provide the output of the following commands

F-A# connect nxos

F-A(nxos)# show interface ethernet 1/1

[Shuja Tirmizi]

Thanks manvelas, Yes I can bring the Interface up without port channel.

Firmware UCS: 2.1.2a

this is the output for

F-A(nxos)# show interface ethernet 1/1

Ethernet1/1 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a57.ddc8 (bia 002a.6a57.ddc8)
Description: S: Server
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is fex-fabric
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 3d01h
Last clearing of "show interface" counters never
30 seconds input rate 19664 bits/sec, 2458 bytes/sec, 6 packets/sec
30 seconds output rate 1696 bits/sec, 212 bytes/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 23.68 Kbps, 4 pps; output rate 1.02 Kbps, 0 pps
RX
2368372 unicast packets 84239 multicast packets 1 broadcast packets
2452612 input packets 802589522 bytes
317103 jumbo packets 0 storm suppression bytes
0 giants 0 input error 0 short frame 0 overrun 0 underrun 0 watchdog 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 85804 multicast packets 10 broadcast packets
85814 output packets 55596514 bytes
0 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble
0 Tx pause
2 interface resets

------------------------------------------------------------------------------------

this the output for FC port on FI connect to 5k

fc1/31 is down (Error disabled - port reinit limit reached)
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Port WWN is 20:1f:00:2a:6a:57:dc:40
Admin port mode is NP, trunk mode is off
snmp link state traps are enabled
Port vsan is 431
Receive data field Size is 2112
Beacon is turned off
Belongs to san-port-channel 100
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1007 frames input, 251944 bytes
20 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
1217 frames output, 129304 bytes
0 discards, 0 errors
49 input OLS, 47 LRR, 26 NOS, 0 loop inits
90 output OLS, 11 LRR, 27 NOS, 0 loop inits
last clearing of "show interface" counters never
Interface last changed at Thu Apr 24 08:46:49 2014

[Manuel Velasco]

Hi Syed,

Looks like the issue is with your nexus switch.  The problem is that the OUI information of the fabric interconnect is not in the 5k data base.  See defect below

Add MAC OUI "002a6a", "8c604f", "00defb" for 5k/UCS-FI

https://tools.cisco.com/bugsearch/bug/CSCty04686

==================LOG info==============

F-A(nxos)# show interface ethernet 1/1

Ethernet1/1 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a57.ddc8 (bia 002a.6a57.ddc8)
Description: S: Server
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is fex-fabric

Let me know if this helps.

[Shuja Tirmizi]

I have this info in my running config added, i just look it was there. did not make any changes

fcdomain fcid database
  vsan 1 wwn 20:1f:00:2a:6a:57:dc:40 fcid 0x170000 dynamic
  vsan 1 wwn 20:20:00:2a:6a:57:dc:40 fcid 0x170040 dynamic

[Manuel Velasco]

This is not a configuration issue.

Please read defect below

https://tools.cisco.com/bugsearch/bug/CSCty04686

[Shuja Tirmizi]

I did remove the trunking TF from sanport channel, but still no go

[Walter Dey]

I don't understand what this should have anything to do with ethernet 1/1 !

Did you enable fc trunking on UCSM ?? see attachment

Did you enable the port-channel on the N5k ?

[Manuel Velasco]

Hi wdey,

I asked Syed, to run this command only to find out the OUI for the fabric interconnect. You are correct eth 1/1 doesn't has to do with the issue. 

[Manuel Velasco]

Did you get a chance to upgrade your n5k switches?

[Shuja Tirmizi]

yes I did upgrade to ver 6.0.2.N2.4 and it resolve ths issue,

[Shuja Tirmizi]

H manvelas,

I have one more question.

can I use 5548up use as

A Fibre Channel Bridge connects an FCoE network to a Fibre Channel network.

I have two port connect to FI is FCOE mode and two ports connect NetApps in FC mode, I am getting flogi from NetApp and FI, but FI has no flogi entries.

Do you have any suggestion?

[Manuel Velasco]

Syed,

what do you mean by BRIDGE?

on your UCS are you able to a flogi for the 5548?

Are you selecting the correct vsans?

[Shuja Tirmizi]

No, there is flogi for 5k in ucs.

at this time we are out of 8GB sfps to use in UCS and there is no FCOE enable adapter in NetApp,  in this case we would like to connect

UCS--fcoe--5k--fc--netapp.

I am getting flogi in 5k but not in ucs

[Walter Dey]

Could you please post a diagram of the setup ! I'm very confused: you say that Netapp is connected with fc to N5k; on the other hand you say that there is no FCoE adaptor on Netapp ?

If UCS is working in FC NPV mode, you will not see any flogi on UCS !