10-24-2013 02:01 AM
We are building a Datacenter with UCS 6248U FIs and Nexus 5548UP Switches.
The problem ist the FCoE configuration on the Nexus. After creating the vfC-Interface it is member of the default-vsan1. When we try to assign it to an other vsan (here vsan 100), it goes down with "error disabled".
We have only the Interface e1/7 in port-channel 60 bound to vfc60, vsan 100 associated to vlan 710
I hope anybody has an idea what's the problem.
Here is an extract of our configuration:
RZ1NX001(config-if)# show run int e1/7
interface Ethernet1/7
description FCoE Link to RZ2FI-A eth1/19
switchport mode trunk
switchport trunk allowed vlan 710
channel-group 60 mode active
--------------------------------------
RZ1NX001(config-if)# show int e1/7
Ethernet1/7 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a53.5c2e (bia 002a.6a53.5c2e)
Description: FCoE Link to RZ2FI-A eth1/19
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
-----------------------------------------
RZ1NX001(config-if)# show run int po60
interface port-channel60
switchport mode trunk
switchport trunk allowed vlan 710
spanning-tree port type edge trunk
speed 10000
-----------------------------------------
RZ1NX001(config-if)# show int po60
port-channel60 is up
Hardware: Port-Channel, address: 002a.6a53.5c2e (bia 002a.6a53.5c2e)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s
Input flow-control is off, output flow-control is off
Switchport monitor is off
EtherType is 0x8100
Members in this channel: Eth1/7
------------------------------------------
RZ1NX001(config-if)# show run int vfc60
interface vfc60
bind interface port-channel60
switchport trunk allowed vsan 100
no shutdown
------------------------------------------
RZ1NX001(config-if)# show int vfc60
vfc60 is down (Error disabled)
Bound interface is port-channel60
Hardware is Ethernet
Port WWN is 20:3b:00:2a:6a:53:5c:7f
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port vsan is 100
-------------------------------------------
RZ1NX001(config-if)# show vlan fcoe
Original VLAN ID Translated VSAN ID Association State
---------------- ------------------ -----------------
710 100 Operational
Solved! Go to Solution.
10-24-2013 06:21 AM
Hello Robert,
The port-channel status is SD and member link is still I state. Have you tried flapping the port-channel interfaces ?
Also, NPV feature is enabled on N5K and I don't see NPIV enabled on it.
Are you planning to run N5K in NPV mode ? If yes, I need to verify if FI can be connected to FC switch operating in NPV mode.
I guess FI is in FC End host mode ( NPV ) and it needs to be connected to FC switch with NPIV feature.
Padma
10-24-2013 03:16 AM
Hello Robert,
What is the UCSM and N5K version ?
From N5K
show feature | grep -i enabled
show fcoe
From UCS FI
connect nxos
show port-channel summary
--- same set of commands as you shared from N5K
Padma
10-24-2013 04:04 AM
Hello Padma,
N5K-Version is version 5.1(3)N2(1)
UCS-Manager is 2.1(2a)
We've configured the UCS with UCSM but the result looks similar to the N5K:
N5K:
--------------------------------------------
RZ1NX001# show feature | grepp i enabled
^
% Invalid command at '^' marker.
RZ1NX001# show feature | grep -i enabled
fcoe 1 enabled
fex 1 enabled
interface-vlan 1 enabled
lacp 1 enabled
lldp 1 enabled
npv 1 enabled
ospf 1 enabled (not-running)
ospf 2 enabled (not-running)
ospf 3 enabled (not-running)
ospf 4 enabled (not-running)
sshServer 1 enabled
vpc 1 enabled
----------------------------------------------
RZ1NX001# show fcoe
RZ1NX001#
(no output)
-----------------------------------------------
-----------------------------------------------
FI:
RZ1FI001-A(nxos)# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
60 Po60(SD) Eth LACP Eth1/19(I)
1026 Po1026(SU) Eth NONE Eth1/1(P) Eth1/2(P) Eth1/3(P)
Eth1/4(P)
1283 Po1283(SD) Eth NONE Eth2/1/9(D) Eth2/1/11(D)
1290 Po1290(SD) Eth NONE Eth2/1/13(D) Eth2/1/14(D) Eth2/1/15(D)
Eth2/1/16(D)
1292 Po1292(SD) Eth NONE Eth2/1/17(D) Eth2/1/18(D) Eth2/1/19(D)
Eth2/1/20(D)
1293 Po1293(SD) Eth NONE Eth2/1/21(D) Eth2/1/22(D) Eth2/1/23(D)
Eth2/1/24(D)
1295 Po1295(SD) Eth NONE Eth2/1/29(D) Eth2/1/30(D) Eth2/1/31(D)
Eth2/1/32(D)
1307 Po1307(SU) Eth NONE Eth2/1/1(P) Eth2/1/2(P) Eth2/1/3(P)
Eth2/1/4(P)
1309 Po1309(SU) Eth NONE Eth2/1/5(P) Eth2/1/6(P) Eth2/1/7(P)
Eth2/1/8(P)
--------------------------------------------------------
RZ1FI001-A(nxos)# show run int e1/19
!Command: show running-config interface Ethernet1/19
!Time: Thu Oct 24 10:42:28 2013
version 5.0(3)N2(2.11.2a)
interface Ethernet1/19
description C: FcoeUplink
pinning border
switchport mode trunk
switchport trunk allowed vlan 710,4048
channel-group 60 mode active
no shutdown
----------------------------------------------------------
RZ1FI001-A(nxos)# show int e1/19
Ethernet1/19 is up
Hardware: 1000/10000 Ethernet, address: 002a.6a3b.523a (bia 002a.6a3b.523a)
Description: C: FcoeUplink
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
-----------------------------------------------------------
RZ1FI001-A(nxos)# show run int po60
!Command: show running-config interface port-channel60
!Time: Thu Oct 24 10:45:04 2013
version 5.0(3)N2(2.11.2a)
interface port-channel60
description C: FcoeUplink
switchport mode trunk
pinning border
switchport trunk allowed vlan 710,4048
speed 10000
------------------------------------------------------------
RZ1FI001-A(nxos)# show int po60
port-channel60 is down (No operational members)
Hardware: Port-Channel, address: 002a.6a3b.523a (bia 002a.6a3b.523a)
Description: C: FcoeUplink
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
auto-duplex, 10 Gb/s
Beacon is turned off
Input flow-control is off, output flow-control is off
Switchport monitor is off
EtherType is 0x8100
Members in this channel: Eth1/19
---------------------------------------------------------------
RZ1FI001-A(nxos)# show vlan fcoe
Original VLAN ID Translated VSAN ID Association State
---------------- ------------------ -----------------
4048 1 Operational
710 100 Operational
-----------------------------------------------------------------
RZ1FI001-A(nxos)# show run int vfc 792
interface vfc792
bind interface port-channel60
switchport mode NP
no shutdown
------------------------------------------------------------------
RZ1FI001-A(nxos)# show int vfc792
vfc792 is down (Error disabled)
Bound interface is port-channel60
Hardware is Virtual Fibre Channel
Port WWN is 23:17:00:2a:6a:3b:52:7f
Admin port mode is NP, trunk mode is on
snmp link state traps are enabled
Port vsan is 100
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 discards, 0 errors
0 frames output, 0 bytes
0 discards, 0 errors
last clearing of "show interface" counters never
------------------------------------------------------------------
RZ1FI001-A(nxos)# show vsan 100
vsan 100 information
name:VSAN0100 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
10-24-2013 06:21 AM
Hello Robert,
The port-channel status is SD and member link is still I state. Have you tried flapping the port-channel interfaces ?
Also, NPV feature is enabled on N5K and I don't see NPIV enabled on it.
Are you planning to run N5K in NPV mode ? If yes, I need to verify if FI can be connected to FC switch operating in NPV mode.
I guess FI is in FC End host mode ( NPV ) and it needs to be connected to FC switch with NPIV feature.
Padma
10-24-2013 09:03 AM
Hello Padma,
this was the solution. After enabling npiv the vfc shows "NPV upstream port not available". After disabling npv everthing is fine. I confused npv and npiv.
Thank you for your help
Robert
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide