07-15-2016 02:06 AM - edited 03-01-2019 12:49 PM
Hi All,
Its new installation of Cisco UCS with Brocade SAN switch 6510.
I have followed the Cisco UCS to Brocade Connectivity Guide.
Even though the on both Switch and FI side the Link is not coming up. No lights are glowing.
When I checked the switchshow @Brocade 6510, the ports are showing No_Light status. Where in the other ports it shows the WWN number of my Storage device.
At FI, when i click FC ports, its showing Over all Status - link Down.
Anyone face this issue.?
Am i missing some configuration steps?
Regards,
Praveen
Solved! Go to Solution.
07-15-2016 06:12 AM
This must be either a trivial or a very complex problem ! I would rather think trivial.
The Brocade is a 16G FC switch; the negotiation down to 8G seems to work (storage subsystem).
Maybe you could try to set the speed on the Brocade port fix to 8G !
Or something with the cables is wrong ? send / receive crossed ??
Are the SFP's MMF short wave ? patch cables as well ?
Can you connect cable of port eg. 31 to 33, 32 to 34 and see if this works ?
07-15-2016 02:30 AM
Hi Praveen
- is the UCS FI in FC end host mode ?
- did you configure FC ports on the UCS FI ?
- do you have a FC 2/4/8 G SFP
- is the FC port : FC uplink port ?
- did you configure VSAN's (I hope you are not using default VSAN 1, but create 2 different VSAN's on FI-A and FI-B, e.g 20 and 30; and then put the FC ports in the apropriate VSAN.
Walter.
PS have a look at
http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/whitepaper_C07-730016.pdf
07-15-2016 02:35 AM
Hi Walter,
Thanks for the quick reply.
Please find my answers as below.
- is the UCS FI in FC end host mode ? - Yes. Attached screen shot.
- did you configure FC ports on the UCS FI ? - Yes, Port 31 & 32 are FC ports
- do you have a FC 2/4/8 G SFP - 8G SFP
- is the FC port : FC uplink port ? - Yes, its FC Uplink
- did you configure VSAN's (I hope you are not using default VSAN 1, but create 2 different VSAN's on FI-A and FI-B, e.g 20 and 30; and then put the FC ports in the apropriate VSAN.
- Yes, 2 different VSAN configured - Attached screen shots.
Thanks for your help,
Praveen
07-15-2016 02:47 AM
Connect via ssh to each FI, then
connect nx-os
show npv status ...... plus any other available show npv commands
07-15-2016 03:32 AM
Please find the details
#show npv status
npiv is enabled
disruptive load balancing is disabled
External Interfaces:
====================
Interface: fc1/31, State: Pre-Initialized
Interface: fc1/32, State: Pre-Initialized
Number of External Interfaces: 2
Server Interfaces:
==================
Interface: vfc703, VSAN: 20, State: Waiting for External Interface
Interface: vfc739, VSAN: 5, State: Waiting for External Interface
Interface: vfc751, VSAN: 5, State: Waiting for External Interface
Interface: vfc763, VSAN: 5, State: Waiting for External Interface
Number of Server Interfaces: 4
# show npv flogi-table
No flogi sessions found.
show npv internal info
NPV Globals:
============
NPV mode: ENABLED
Switch-Name: CISLABFI-A
Switch Mgmt IP Address: 192.168.47.31
proxy logo Retries: 3
Internal FLOGI max timeout Retries: 3
NS Registration max timeout Retries: 5
Disruptive Load Balance: Disabled
Load Balancing Timer (8), id: 00, active: FALSE time remaining: 0
Server Interface Block Timer is OFF
Server Interface Block Timer(10), id: 00, active: FALSE time remaining: 0
Recent External Interface Up Time: Sun Jul 3 18:06:28 2016
Switch WWN: 20:00:8c:60:4f:c8:df:c0
Port group information
=======================
External Interface Info:
========================
Interface Information:
ifindex: fc1/31, Port VSAN: 20, Internal FLOGI VSAN: 0, fcid: 0x000000
Internal FLOGI Done: False, In Progress: False
FSM current state: NPIVP_EXT_IF_ST_PREINIT_DONE
fcf_mac: 8c60:4fc8:dfe6
Pinned Server Intf Count: 0
Port Parameters:
Operational port mode: NP(0x4000), Logical Ifindex: (0x0)
Rx B2B Credits: 0, Multiplier: 0, Buff Size: 0
Tx B2B Credits: 0, Multiplier: 0, Buff Size: 0, bbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
Virtual Fabric Parameters:
VF Tagging Enabled: FALSE, Core supports VF: FALSE, Port Mode: NP
Physical Login Done: FALSE Port Channel: FALSE
Timer & Retry Information:
Busy Timer (1), id: 00, active: FALSE time remaining: 0
Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0
Statistics:
flogi retry count : 0
ns registration retry count : 0
Others:
NPIV not enabled on core sysloged: FALSE
VSAN mis-match on core sysloged: FALSE
FC Delete Response Received: FALSE
VSAN 20:
Timers:
FDISC Response Timer (2), id: 00, active: FALSE time remaining: 0
Error Clear Timer (6), id: 07, active: TRUE time remaining: 408
Statistics:
number of LS_RJTs rcvd : 0
LS_RJT timestamp : Thu Jan 1 00:00:00 1970
retry count : 0
pending queue size : 0
FLOGIs on this VSAN:
Interface Information:
ifindex: fc1/32, Port VSAN: 20, Internal FLOGI VSAN: 0, fcid: 0x000000
Internal FLOGI Done: False, In Progress: False
FSM current state: NPIVP_EXT_IF_ST_PREINIT_DONE
fcf_mac: 8c60:4fc8:dfe7
Pinned Server Intf Count: 0
Port Parameters:
Operational port mode: NP(0x4000), Logical Ifindex: (0x0)
Rx B2B Credits: 0, Multiplier: 0, Buff Size: 0
Tx B2B Credits: 0, Multiplier: 0, Buff Size: 0, bbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
Virtual Fabric Parameters:
VF Tagging Enabled: FALSE, Core supports VF: FALSE, Port Mode: NP
Physical Login Done: FALSE Port Channel: FALSE
Timer & Retry Information:
Busy Timer (1), id: 00, active: FALSE time remaining: 0
Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0
Statistics:
flogi retry count : 0
ns registration retry count : 0
Others:
NPIV not enabled on core sysloged: FALSE
VSAN mis-match on core sysloged: FALSE
FC Delete Response Received: FALSE
VSAN 20:
Timers:
FDISC Response Timer (2), id: 00, active: FALSE time remaining: 0
Error Clear Timer (6), id: 07, active: TRUE time remaining: 408
Statistics:
number of LS_RJTs rcvd : 0
LS_RJT timestamp : Thu Jan 1 00:00:00 1970
retry count : 0
pending queue size : 0
FLOGIs on this VSAN:
Server Interface Info:
======================
Interface Information:
ifindex: vfc703, VSAN: 20, NPIV requested: FALSE, lcp init done: FALSE
Selected External Interface:
FSM current state: NPIVP_SVR_IF_ST_WAITING_EXTERNAL_INTERFACE
Port Parameters:
rxbbcredit: 0 rxbufsize: 0
txbbcredit: 0 txbufsize: 0 txbbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
is_fip: FALSE
Statistics:
Traffic Map: not configured
Interface Information:
ifindex: vfc739, VSAN: 5, NPIV requested: FALSE, lcp init done: FALSE
Selected External Interface:
FSM current state: NPIVP_SVR_IF_ST_WAITING_EXTERNAL_INTERFACE
Port Parameters:
rxbbcredit: 0 rxbufsize: 0
txbbcredit: 0 txbufsize: 0 txbbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
is_fip: FALSE
Statistics:
Traffic Map: not configured
#
07-15-2016 04:10 AM
Hi Praveen
The link between UCS FI and Brocade is down !
Please provide the output of the following commands on the Brocade
switchshow
and of the ports connecting the UCS FI
portshow
portcfgshow
07-15-2016 04:20 AM
Hi Walter,
I have connected the FI A Port 31 & 32 are connected to Brocade 6510 Port 34 and 35.
One more point, to note, this Brocade switch has 3 Context. I am using Context 2 for my UCS connection.
Please find the details.
#switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: ON
LS Attributes: [FID: 2, Base Switch: No, Default Switch: No, Address Mode 0]
Index Port Address Media Speed State Proto
==================================================
32 32 020000 id N8 Online FC F-Port 50:06:0e:80:13:35:25:03
33 33 020100 id N8 Online FC F-Port 50:06:0e:80:13:35:25:23
34 34 020200 id N16 No_Light FC
35 35 020300 id N16 No_Light FC
#portshow 34
portIndex: 34
portName: port34
portHealth: No Fabric Watch License
Authentication: None
portDisableReason: None
portCFlags: 0x1
portFlags: 0x1 PRESENT U_PORT
LocalSwcFlags: 0x0
portType: 24.0
POD Port: Port is licensed
portState: 2 Offline
Protocol: FC
portPhys: 4 No_Light portScn: 0
port generation number: 0
state transition count: 0
portId: 020200
portIfId: 43020025
portWwn: 20:22:50:eb:1a:65:09:42
portWwn of device(s) connected:
Distance: normal
portSpeed: N16Gbps
FEC: Inactive
Credit Recovery: Inactive
LE domain: 0
FC Fastwrite: OFF
Interrupts: 0 Link_failure: 0 Frjt: 0
Unknown: 0 Loss_of_sync: 0 Fbsy: 0
Lli: 10 Loss_of_sig: 0
Proc_rqrd: 0 Protocol_err: 0
Timed_out: 0 Invalid_word: 0
Rx_flushed: 0 Invalid_crc: 0
Tx_unavail: 0 Delim_err: 0
Free_buffer: 0 Address_err: 0
Overrun: 0 Lr_in: 0
Suspended: 0 Lr_out: 0
Parity_err: 0 Ols_in: 0
2_parity_err: 0 Ols_out: 0
CMI_bus_err: 0
#portcfgshow 34
Area Number: 2
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)
AL_PA Offset 13: OFF
Trunk Port ON
Long Distance OFF
VC Link Init OFF
Locked L_Port OFF
Locked G_Port OFF
Disabled E_Port OFF
Locked E_Port OFF
ISL R_RDY Mode OFF
RSCN Suppressed OFF
Persistent Disable OFF
LOS TOV enable OFF
NPIV capability ON
QOS Port AE
Port Auto Disable: OFF
Rate Limit OFF
EX Port OFF
Mirror Port OFF
SIM Port OFF
Credit Recovery ON
F_Port Buffers OFF
E_Port Credits OFF
Fault Delay: 0(R_A_TOV)
NPIV PP Limit: 126
CSCTL mode: OFF
D-Port mode: OFF
D-Port over DWDM: OFF
Compression: OFF
Encryption: OFF
FEC: ON
07-15-2016 05:12 AM
This is what I see
#switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
portState: 2 Offline
portSpeed: N16Gbps
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)
NPIV capability ON
Question: did you enable port 34, 35 ?
portenable 34,35
Are the 2 devices logged in on port 32/33 storage subsystems ?
Zoning OFF on the switch is very strange ?
Btw. do you see any information in the log file on the UCS system ?
07-15-2016 06:00 AM
Hi Walter,
Yes, I had enabled the Ports earlier also. Again i did the portenable and same status.
The other 2 devices showing the WWN are Storage ports.
Since, I have not seen the UCS ports, Zoning is not done.
Which logs has to check on UCS ?
switch_2:FID2:admin> portenable 34
switch_2:FID2:admin> portenable 35
switch_2:FID2:admin> switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: ON
LS Attributes: [FID: 2, Base Switch: No, Default Switch: No, Address Mode 0]
Index Port Address Media Speed State Proto
==================================================
32 32 020000 id N8 Online FC F-Port 50:06:0e:80:13:35:25:03
33 33 020100 id N8 Online FC F-Port 50:06:0e:80:13:35:25:23
34 34 020200 id N16 No_Light FC
35 35 020300 id N16 No_Light FC
switch_2:FID2:admin>
07-15-2016 06:12 AM
This must be either a trivial or a very complex problem ! I would rather think trivial.
The Brocade is a 16G FC switch; the negotiation down to 8G seems to work (storage subsystem).
Maybe you could try to set the speed on the Brocade port fix to 8G !
Or something with the cables is wrong ? send / receive crossed ??
Are the SFP's MMF short wave ? patch cables as well ?
Can you connect cable of port eg. 31 to 33, 32 to 34 and see if this works ?
07-18-2016 03:16 AM
Hi Walter,
Yes, it was problem with SFP.
The SFP which I had in FI, was LW and at the Brocade was SW.
Luckily I had 2 old (4Gbps ) SW SFP and used in FI. now the connection is showing UP and I can see the WWN in portshow.
Thanks a lot for your support and Guidance.
Regards,
Praveen
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide