vFC is down Nexus 5020 Nexus 4001i

[snewar]

Hi All,

I am very new to Nexus 4001 and facing a problem related to its configuartion.

The topology involves IBM HS22 blade server with Qlogic dual port CNA, Nexus 4001, Nexus 5020, MDS 9513 and EMC storage.

I have done the configuration related to fip snooping in Nexus 4k and FCoE configurations in Nexus 5k.

The problem is that i am not able to get the enode mac address when i am running the command "show fip-snooping vlan-discovery".

also i am not able to make the vFC up in Nexus 5k.

I can upload the configuartions if needed.

Kindly suggest your comments or any solution.

FYI: All licenses are installed in Nexus 4k and 5k and FCoE and CNA drivers are installed in IBM server.

Thanks in advance,

Sanjeev

[ababakha]

Hi Sanjeev,

Just give you some understanding,  when VFC is down it means the Server CNA has not logged into the fabric using its FIP  (FCOE Initialisation)  protocol. You may still need to troubleshoot the ethernet segment. ie. spanning-tree.

So from the ethernet interface level i assumed you have configured it as a trunk  and specified a FCOE vlan so here are the things i would check:

1.   Make sure you have a native vlan defined  for the   FIP Vlan discovery that will get sent towards the N5k.  Also run "show mac-address-table) on the native vlan to ensure it is learning the servers mac.

2.   Check spanning-tree for FcOE vlan and ensure it is forwarding.

3.   Ensure the native vlan and fcoe vlan are forwarding on the FCF uplink trunk from N4k  connecting to N5k.

4.  Check Fip-snooping is configured on the N4k FCOE vlan only and not any other vlan.

5 . On the N5k check  to see the VFC is binding to the mac-adress and not an interface.. this is  enabled N4k / N5k to provide multi hop fcoe.

If all the above is correct then the issue most likely could be on the CNA as thats where FIP is actually programmed on. Whether it is Qlogic or Emulex i would suggest you contact the vendor to ensure you are on the correct firmware and that DCB is enabled and functioning.

I hope this helps.

Andrew

[snewar]

Hi Andrew,

First of all thanks for your reply.

Now, i have checked the chekclist that you have given.

Here the first checklist fails to retrieve the server's mac address when i run the command "Show mac-address-table" on native vlan.

Kindly suggest any solution or comment.

Also i have attached the output of "show fip-snooping fcf" which shows that active enode is not discovered.

Thanks,

Sanjeev

[ababakha]

If the show mac-address is not showing any src mac being learnt in native vlan indicates server is not sending FIP messages.  The output you provided does confirm this  theory, see the marked output in bold which has value of zero.:

Interface Name    Counter Desciption                                Value
-------------------------------------------------------------------------------
Ethernet1/1       Unicast packets received                          0
                  Multicast packets received                        0
                  Vlan discovery packets received                   0
                  Vlan discovery response packets received          0
                  flogi (fabric login) packets received             0
                  flogi lsacc (acknowledgment) packets received     0
                  flogi lsrjt (reject) packes received              0
                  fdisc packets received                            0
                  fdisc lsacc received                              0
                  fdisc lsrjt received                              0
                  logo (logout) packets received                    0
                  logo lsacc received                               0
                  logo lsrjt received                               0
                  Enode Keepalive received                          0
                  Enode Discovery received                          0
                  FCF Discovery Solicit received                    0
                  FCF Keepalive received                            0
                  Clear Virtual Link received                       0
                  Clear Virtual Link generated                      0
                  Unicast packets transmitted                       0
                  Packets dropped due to invalid length             0
                  Packets dropped due to tlv parse error            0
                  Packets dropped due to unsupported tlv            0
                  Packets dropped due to fcmap mismatch             0

Based on the output you provided it looks to be a CNA issue, the  next step would be working with vendor to ensure Datacenter Bridging is working correctly.  Do you have OS loaded on local disk  or trying to boot from san?

[snewar]

Hi Andrew,

I have attached the screenshot from management tool Sansurfer.

From that i can see that the DCBX is working fine.

Also i have raised the case regarding the issue in Qlogic.

Shall post the reply from them if any..

Thanks,

Sanjeev

[ababakha]

Thanks for the screen shot. DCBX is definitely enabled. DCBX is used to program the CNA symmetrically but it is not responsible for the FCOE initialisation which needs to happen before DCBX.

Are you booting from SAN ?

What OS are you using?

[snewar]

Hi Andrew,

Thanks for your information.

Currently the OS is booting from local disk only.

And the OS used is Microsoft Windows Server 2008 Enterprise 6.0.6002 Service Pack 2 Build 6002.

I have attached the detailed system info that contains information regarding the IBM server.

Thanks,

Sanjeev

[ababakha]

Happy to help here.

- Does Windows server see the HBA in device manager?

- If you are able to see the HBA and cannot scan for LUNS once again it will come back to the adapter not starting the FCOE initialisation.

[snewar]

Hi Andrew,

In Device manager, i can see the 2 ports Qlogic ethernet ports and 2 ports Qlogic FCoE Adaptor.

Attached is the screenshot of device manager.

Other than that, does the Qlogic hba shows explicitly here?

[ababakha]

Hi Sanjeev,

Just checking up to see how you went with the CNA vendor on this issue?

[snewar]

Hi Andrew,

Yes i confirmed with the CNA vendor. They told that the CNA is supported.

The issue was solved by re-applying the setup script and explicitly mentioning to enable FCoE and fip-snooping during the setup wizard.

Doing that the service policies and QOS are applied and the fip-snooping started working.

I would also like to mention that the same service policies i had applied manually but it didnt work at that time.

Thanks,

Sanjeev

[j.muozmellado]

Hello Sanjeev and Andrew:

I had the same issue with a CNA card and Cisco Nexus 4001i and I followed everything that you posted on the discussion board. The last thing mentioned by Sanjeev worked out perfectly.I re-applied the setup scrip and explicitly enabled FCoE and fip-snooping during the setup wizard..I could see activity on the fiber ports after 2 or 3 minutes    

Thanks for your help

[Surya ARBY]

QoS and MTU have to be configured and enabled before applying FIP snooping on the FCOE VLANs, maybe the issue comes from that point.