cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
898
Views
0
Helpful
5
Replies

Unable to fcping Enode over FCoE

Dave Mumford
Level 1
Level 1

Hi ,

Im wondering how i should troubleshoot an issue i have. I have a nexus 56128P switch with connected Storage (IBM) and Enodes connected to Lenovo CN4093 switches. I have a port-channel connecting the Lenovo to cisco and have this setup to run FCoE.

The storage devices are logged into the fabric and so is the Enode see below:

MEDC1ASW04# sh flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
fc2/1 2203 0x680041 50:0a:09:85:80:51:ad:33 50:0a:09:80:80:51:ad:33 <<< This is one of the storage devices
[DC1_2203_ONT-CLU-001-01_0f]
... output ommitted
[DC1_2203_ONT-CLU-001-04_fc_lif_2]
vfc201 2203 0x680100 10:00:00:90:fa:be:d1:f7 20:00:00:90:fa:be:d1:f7  <<< This is the enode connected to the Lenovo switch
[DC1_2203_VMW-ESX-001-B]

From the Nexus I can fcping the storage device (direct FC) but not the enode (over FCoE), see below:

MEDC1ASW04# fcping pwwn 50:0a:09:85:80:51:ad:33 vsan 2203  <<< Storage device above directly connected to the same Nexus 56128P switch.
28 bytes from 50:0a:09:85:80:51:ad:33 time = 326 usec
28 bytes from 50:0a:09:85:80:51:ad:33 time = 120 usec
28 bytes from 50:0a:09:85:80:51:ad:33 time = 155 usec
28 bytes from 50:0a:09:85:80:51:ad:33 time = 157 usec
28 bytes from 50:0a:09:85:80:51:ad:33 time = 122 usec

5 frames sent, 5 frames received, 0 timeouts
Round-trip min/avg/max = 120/176/326 usec
MEDC1ASW04#
MEDC1ASW04#
MEDC1ASW04#
MEDC1ASW04# fcping pwwn 10:00:00:90:fa:be:d1:f7 vsan 2203 <<< Enode connected over FCoE to Lenovo switch.
No response from the destination.
MEDC1ASW04#

I cannot do a successfull fcping from the CN4093 switch to either the attached enode or to the storage device over the FCoE link see below:

MEDC1FSW02#fcping 10:00:00:90:fa:be:d1:f7  <<<< direct attached enode
28 bytes from local switch to 10:00:00:90:fa:be:d1:f7 failed! time us 0
28 bytes from local switch to 10:00:00:90:fa:be:d1:f7 failed! time us 0
28 bytes from local switch to 10:00:00:90:fa:be:d1:f7 failed! time us 0
28 bytes from local switch to 10:00:00:90:fa:be:d1:f7 failed! time us 0
28 bytes from local switch to 10:00:00:90:fa:be:d1:f7 failed! time us 0
MEDC1FSW02#
MEDC1FSW02#
MEDC1FSW02#fcping 50:0a:09:85:80:51:ad:33 <<< FCoe attached Storage
28 bytes from local switch to 50:0a:09:85:80:51:ad:33 failed! time us 0
28 bytes from local switch to 50:0a:09:85:80:51:ad:33 failed! time us 0
28 bytes from local switch to 50:0a:09:85:80:51:ad:33 failed! time us 0
28 bytes from local switch to 50:0a:09:85:80:51:ad:33 failed! time us 0
28 bytes from local switch to 50:0a:09:85:80:51:ad:33 failed! time us 0
MEDC1FSW02#

Anyone able to assist ? Help appreciated.

Thanks,

Dave.

5 Replies 5

jihicks
Cisco Employee
Cisco Employee

Hi Dave,

First, yes you should be able to FCPING a device connected via FCoE.

I am not aware of this not working but I have no experience with a Lenovo switch.

The first thing I would check is the system QOS on the Nexus 56128P.

Here is the default QOS:

policy-map type network-qos fcoe-default-nq-policy
  class type network-qos class-fcoe
    pause no-drop
    mtu 2158
  class type network-qos class-default
    multicast-optimize
system qos
  service-policy type qos input fcoe-default-in-policy
  service-policy type queuing input fcoe-default-in-policy
  service-policy type queuing output fcoe-default-out-policy
  service-policy type network-qos  fcoe-default-nq-policy

If the current QOS does not match or does not contain a no-drop policy for FCoE, this could be the problem.

A jumbo policy might look like:

policy-map type network-qos fcoe-jumbo
  class type network-qos class-default
    mtu 9216
   class type network-qos class-fcoe
    pause no-drop
    mtu 2158
   

Notice it has both the jumbo MTU and a network-qos for FCoE.

To apply the above:

conf t

system qos
service-policy type network-qos fcoe-jumbo

Is FCPING the only thing not working?  Can the FCoE hosts see storage? 

If the answer is YES, then I would ask look into if the host or switch support the  fcping command which is the Fibre Channel FC-LS-2 ECHO ELS.   I don't remember running into a case where fcping was unsupported.

If the problem is more than FCPING failing and QOS is good, then you might want to open a Cisco TAC case for this.

Best regards,

Jim

Hi Jim,

I have this configured, i changed the default policy to allow for jumbo frames, see below:

policy-map type network-qos Jumbo-and-FcoE
class type network-qos class-fcoe
pause no-drop
mtu 2158
class type network-qos class-default
mtu 9216
system qos
service-policy type queuing input fcoe-default-in-policy
service-policy type queuing output fcoe-default-out-policy
service-policy type qos input fcoe-default-in-policy
service-policy type network-qos Jumbo-and-FcoE

And no the initiator cant see the targets , which was why I started looking at the FCPING command. I have since found that I can ping the initiator from another switch upstream of the new N56128 we have just put in. so this is what is not working / working ?

initiator(lenovo)---(FCoe)---N56128 = when i fcping initiator from here i get no response

initiator(lenovo)---(FCoe)---N56128---(FC)---N5596UP = when i fcping initiator from here i get a response

Thanks,

Dave.

Hi Dave,

What is the firmware version that is running on the switch?

That is a strange set of events.

The N56128 is not in NPV mode is it?

Is the Lenovo in NPV mode?

What features are enabled on the N56128?

Where are you seeing it in the flogi table?

What does the FCNS entry look like for the initiator?

Thanks,

Jim

Hi Jim,

The N56128 is not in NPV mode is it? NO 

Is the Lenovo in NPV mode? NO

What features are enabled on the N56128?

feature fcoe
fabric-mode 40G
hostname MEDC1ASW04
feature npiv
no feature telnet
feature tacacs+
cfs eth distribute
feature udld
feature interface-vlan
feature lacp
feature vpc
feature lldp
feature fex

Flogi table:

MEDC1ASW04# sh flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
fc2/1 2203 0x680041 50:0a:09:85:80:51:ad:33 50:0a:09:80:80:51:ad:33
[DC1_2203_ONT-CLU-001-01_0f]
fc2/1 2203 0x680042 20:02:00:a0:98:99:65:80 20:0a:00:a0:98:99:65:80
[DC1_2203_ONT-CLU-001-01_fc_lif_1]
fc2/3 2203 0x680000 50:0a:09:85:80:31:ac:d3 50:0a:09:80:80:31:ac:d3
[DC1_2203_ONT-CLU-001-02_0f]
fc2/3 2203 0x680001 20:05:00:a0:98:99:65:80 20:0a:00:a0:98:99:65:80
[DC1_2203_ONT-CLU-001-02_fc_lif_2]
fc2/4 2203 0x680020 50:0a:09:83:80:31:ac:d3 50:0a:09:80:80:31:ac:d3
[DC1_2203_ONT-CLU-001-02_0h]
fc2/5 2203 0x680080 50:0a:09:84:80:e1:a4:45 50:0a:09:80:80:e1:a4:45
[DC1_2203_ONT-CLU-001-03_0g]
fc2/5 2203 0x680081 20:07:00:a0:98:99:65:80 20:0a:00:a0:98:99:65:80
[DC1_2203_ONT-CLU-001-03_fc_lif_2]
fc2/7 2203 0x6800a0 50:0a:09:84:80:11:a4:b1 50:0a:09:80:80:11:a4:b1
[DC1_2203_ONT-CLU-001-04_0g]
fc2/7 2203 0x6800a1 20:09:00:a0:98:99:65:80 20:0a:00:a0:98:99:65:80
[DC1_2203_ONT-CLU-001-04_fc_lif_2]
vfc201 2203 0x680100 10:00:00:90:fa:be:d1:f7 20:00:00:90:fa:be:d1:f7
[DC1_2203_VMW-ESX-001-B]
vfc202 2203 0x680120 10:00:00:90:fa:ba:8a:89 20:00:00:90:fa:ba:8a:89
[DC1_2203_VMW-ESX-002-B]
vfc401 2203 0x6800c0 10:00:00:90:fa:ba:8c:0f 20:00:00:90:fa:ba:8c:0f
[DC1_2203_VMW-ESX-015-B]
vfc402 2203 0x680140 10:00:00:90:fa:ba:8c:87 20:00:00:90:fa:ba:8c:87
[DC1_2203_VMW-ESX-016-B]

sh fcns data:

MEDC1ASW04# sh fcns data

VSAN 2203:
--------------------------------------------------------------------------
FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE
--------------------------------------------------------------------------
output omitted....

0x680000 N 50:0a:09:85:80:31:ac:d3 (NetApp)
[DC1_2203_ONT-CLU-001-02_0f]
0x680001 N 20:05:00:a0:98:99:65:80 (NetApp) scsi-fcp:target
[DC1_2203_ONT-CLU-001-02_fc_lif_2]
0x680020 N 50:0a:09:83:80:31:ac:d3 (NetApp)
[DC1_2203_ONT-CLU-001-02_0h]
0x680041 N 50:0a:09:85:80:51:ad:33 (NetApp)
[DC1_2203_ONT-CLU-001-01_0f]
0x680042 N 20:02:00:a0:98:99:65:80 (NetApp) scsi-fcp:target
[DC1_2203_ONT-CLU-001-01_fc_lif_1]
0x680080 N 50:0a:09:84:80:e1:a4:45 (NetApp)
[DC1_2203_ONT-CLU-001-03_0g]
0x680081 N 20:07:00:a0:98:99:65:80 (NetApp) scsi-fcp:target
[DC1_2203_ONT-CLU-001-03_fc_lif_2]
0x6800a0 N 50:0a:09:84:80:11:a4:b1 (NetApp)
[DC1_2203_ONT-CLU-001-04_0g]
0x6800a1 N 20:09:00:a0:98:99:65:80 (NetApp) scsi-fcp:target
[DC1_2203_ONT-CLU-001-04_fc_lif_2]
0x6800c0 N 10:00:00:90:fa:ba:8c:0f
[DC1_2203_VMW-ESX-015-B]
0x680100 N 10:00:00:90:fa:be:d1:f7
[DC1_2203_VMW-ESX-001-B]
0x680120 N 10:00:00:90:fa:ba:8a:89
[DC1_2203_VMW-ESX-002-B]
0x680140 N 10:00:00:90:fa:ba:8c:87
[DC1_2203_VMW-ESX-016-B]

Regards,

Dave.

Jim,

This is now resolved. I hit bug :

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCug84860/?reffering_site=dumpcr

We are running recommended version 7.1(0)N1(1). All is OK if we bind the interface to the vfc instead of the mac address.

Thanks,

Dave.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: