Nexus 5548p-2 fcoe initiator wont stay logged in

[niranjandas]

Hi,

I am seeing a weird behaviour with this Nexus 5548p-2 switch. My interfaces are configured for FCoE. Initiator logs into the switch stay on name server for a few minutes and then logs out. If i go and do a disable/enable at the pcie device on the windows host, it logs in back but then again after few minutes it logs out. I did a write erase in the switch, begin the config from scratch but no help.

If i look at the eth config fcoe status it shows up. Why not pwwn printed in the command below?

c5548p-2# show interface ethernet 1/1 fcoe

Ethernet1/1 is FCoE UP

    vfc1 is Up

vfc shows,

c5548p-2# show int vfc1

vfc1 is trunking (Not all VSANs UP on the trunk)

    Bound interface is Ethernet1/1

    Hardware is Virtual Fibre Channel

    Port WWN is 20:00:00:05:73:cf:2c:7f

    Admin port mode is F, trunk mode is on

    snmp link state traps are enabled

    Port mode is TF

    Port vsan is 2001

    Trunk vsans (admin allowed and active) (1,2001)

    Trunk vsans (up)                       ()

    Trunk vsans (isolated)                 ()

    Trunk vsans (initializing)             (1,2001)

    1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

    1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

      26 frames input, 3360 bytes

        0 discards, 0 errors

      11 frames output, 1700 bytes

        0 discards, 0 errors

    last clearing of "show interface" counters never

    Interface last changed at Wed Nov  9 05:30:43 2011

I want to add one more question to the original one. This is regarding vfc. From the above "show int vfc1" look at the "initializing" key word. (highlighted above). I guess this started happening after release of 5.0.x firmware. In 4.2.x i did not see this. Now what this "initializing" stands for? Also why do we need Trunking at a port where a initiator to be connected ?

I used to remove trunking from vfc and add only the vsan which it is part of. I am not sure if doing that is right. But many times it resolves my initializing problem.

c5548p-2(config-if)# no switchport trunk allowed vsan all

c5548p-2(config-if)# switchport trunk allowed vsan 2001

Here are my running config for the interfaces.

c5548p-2# show running-config interface eth1/1

!Command: show running-config interface Ethernet1/1

!Time: Wed Nov  9 22:36:48 2011

version 5.0(3)N2(2a)

interface Ethernet1/1

  switchport mode trunk

  spanning-tree port type edge trunk

c5548p-2#

c5548p-2# show running-config interface vfc1

!Command: show running-config interface vfc1

!Time: Wed Nov  9 22:36:53 2011

version 5.0(3)N2(2a)

interface vfc1

  bind interface Ethernet1/1

  switchport trunk allowed vsan 2001

  no shutdown

Firmware version i am running is, 5.0(3)N2(2a)

[Kris Vandecruys]

Hi Niranjan,

I would not recommend trying to remove trunking, in 5.x the ports are automatically TF ports.

The trick here is that you need to enable the VSAN-VLAN on the trunk of the Ethernet interface, so:

(config)# interface eth1/1
(config-if)# switchport trunk native vlan 1 

(config-if)# switchport trunk allowed vlan 2001

Assuming that you have configured it like this:

vlan 2001

  fcoe vsan 2001

The other thing that can have gone wrong is that there's something wrong with the QoS config for FCoE.  Can you provide me with the output of show policy-map if the fix above doesn't do it?

HTH,

Kris

[niranjandas]

Thanks Kris for your valuable reply. I have not tried to make the VLAN change on the ethernet int like you have shown above. In my previous experiences i got similar issue resolved by changing it on the vfc interface alone. On VLAN VSAN association i have similar config that you have shown on the example. On a educational based, i want to know what made cisco to choose TF port for vfc? In reality only one VN_port can get attached to it.

What ever i tried the issue did not go away.

Finally i resorted to borrow a switch config from another C5548p switch we have on the lab and apply it to the not working switch. That tricks seemed to work. I had to change the mgmt ip address after applying the config. I saved the earlier not working config. Still have to scan through it to find out what was missing or wrong. Traditionally cisco Nexus switches are found to be more stable on FCoE than other vendor. This time only this was stuck at the corner.

[erikpsmith]

It sounds like you have an issue with FIP Keep Alive.  If the 5548 does not recieve a keep alive within 2.5 times the Keep Alive Timeout value, it will log the end device out. Since you are saying it happens after a few minutes, this could mean the VN_Port keep alive is not being received by the switch and since FKA_VN_PERIOD = 90 seconds by default, the CNA would be logged out after 2.5 * 90 or 225 seconds.. 

I think I've seen this issue before but it was NX-OS or CNA driver specific.  Since you are running a recent version of NX-OS, Im inclined to think the problem could be related to the end device you are using.

Regards, Erik   

[niranjandas]

On the logical analysis front you are exactly right. The initiator did not send FKA for more than 2.5 times the FKA period. This resulted sewitch to send CVL. But when i move the same initiator to ther working sewitch it satyed logged in for long. Interestingly, Netapp target never logged into the switch. If i move the same target to another switch it logged in. I have not seen a issue like this before at cisco nexus switch.

Thanks, Erik for your response.