Configuring Nexus 5k for SAN switching between UCS 6100s and NetApp storage

[JUSTIN LOUCKS]

We are trying to get our UCS environment setup to do SAN boot to LUNs on our NetApp storage. We have a pair of Nexus 5Ks with the Enterprise / SAN license on them and the 6-port FC module installed. What needs to be done to configure the Nexus 5000 to operate as the SAN switch between the storage target and the UCS environment?

[Jeremy Waldrop]

The FC ports on the 61xx-A explansion module need to be connected to one and then the FC ports on the 61xx-B need to be connected to the other. Create a vsan on one N5K, I usually use something like vsan 11 and not the default vsan 1. On the other N5K create vsan 12.

The NetApp SAN FC ports then get connected to the N5Ks and then you can zone.

Also make sure the npiv feature is enabled on the N5Ks.

[JUSTIN LOUCKS]

Thanks Jeremy. I got that done now and I'm a little bit closer. I've got my NetApp connected to the VSANs in the 5K now and I can do a fcping from the 5K to the pwwn on the NetApp port for thta VSAN. I have created a service profile in UCS with a vHBA on the fabric and VSAN which match that of the NetApp, assigned it to a blade and zoned the NetApp WWPN and the vHBA WWPN together but I don't see the vHBA show up anywhere in the 5K when the Service Profile is powered on.

Does anything else need done in the 5K beside just enabling the NPIV feature for the interaction between the 6120 and 5010s?

Any further assistance or suggestions would be appreciated.

[Jeremy Waldrop · Accepted solution]

Are you seeing the 6120 WWPNs on the N5Ks? What is the output of "show flogi database" and "show int br"

Did you map the FC ports on the 6120 to the correct VSAN?

Is npiv enabled?

[JUSTIN LOUCKS]

It was the VSAN mapping of the FC ports in the 6120. When I made my way out to the Equipment tab and looked at the uplinks, they were red. I changed them to the new VSAN I created on the 5K they uplink to and they came live. The FCS database showed the pwwn of the vHBA right away too. Hopefully this is the last thing I was missing...

Thanks for the prompt response and assistance.

[JUSTIN LOUCKS]

I'm still not seeing the LUN on the NetApp device from my service profile. Here are the outputs from the two

commands you referenced here along with a few other commands if they help at all.

Port fc2/1 is the conneciton to the UCS 6100 with FCID 0x640004 being the vHBA in my Server profile.

Port fc2/5 is the NetApp target. I have the LUN masked to the vHBA port name 20:00:00:25:b5:01:00:ff

I have just the wwpn from the vHBA in my server profile and the wwpn of the NetApp target port zoned together. I'm not seeing any FC traffic at the NetApp though from looking at the statistics. Do I need to include something else in my zoning?

Again, any assistance would be appreciated. This is obviously our first venture into FC...

[Jeremy Waldrop]

What are you seeing on the NetApp side, do you see the host initiators login? What is the output of "show zoneset active" , you should see asterisks * next to each active pwwn.

[JUSTIN LOUCKS]

We don't see anything at the NetApp device. I just booted the Service Profile that I am working with (has two vHBAs - one on each fabric) but I never see either of it's vHBAs show up in the active set on the Nexus 5Ks. The active pwwn in each below is the NetApp device.port. The inactive one is the vHBA from the Service Profile.

N5010A# show zoneset active
zoneset name ZONESET10 vsan 10
  zone name UCS-SERVER1-NETAPP-R200-11A vsan 10
  * fcid 0x840004 [pwwn 50:0a:09:81:83:d1:3b:b3]
    pwwn 20:00:00:25:bf:01:00:ef

N5010B# show zoneset active
zoneset name ZONESET11 vsan 11
  zone name UCS-SERVER1-NETAPP-R200-11B vsan 11
  * fcid 0x640000 [pwwn 50:0a:09:82:83:d1:3b:b3]
    pwwn 20:00:00:25:bf:01:00:ff

[Jeremy Waldrop]

Is the boot policy configured for SAN boot?

[JUSTIN LOUCKS]

Yes. I even did the vHBA name enforcement to ensure that it was matching up with what

is in the Service Profile.  Screenshots attached if you see anything that jumps out.

[JUSTIN LOUCKS]

Well, I'm learning a lot about troubleshooting this stuff but in the end I think it was a typo that caused this last issue. I've got my pWWN off by one character in the zoning and of-course I copied and pasted with a quick edit for the zoning on the 2nd Nexus 5K.

Just been staring at it too long!  Thanks for helping me go through things to check. Also the 'show zoneset active' command is definitely good to know. Any changes to the zoneset do not show up until you run the activate command on it again I guess.

I'm booting it back up now so we'll see if it logs in correctly this time...

[Lucien Avramov]

you need to enable feature fcoe.

A few links:

http://www.cisco.com/en/US/partner/products/ps10281/products_configuration_example09186a0080afd130.shtml

http://www.cisco.com/en/US/partner/docs/switches/datacenter/nexus5000/sw/fcoe/421_n1_1/Cisco_n5k_fcoe_config_gd_re_421_n1_1_chapter3.html

[Jeremy Waldrop]

Good catch Lucien, I forgot all about enabling fcoe.

[JUSTIN LOUCKS]

Thanks for offering this up. I did have it enabled and found the issue elsewhere.

I cannot look these up though as they are in the Partners site, which us lowly end users cannot get to with our Cisco IDs...HA!

[Jeremy Waldrop]

If you take the partner out of the URLs you can access them

http://www.cisco.com/en/US/products/ps10281/products_configuration_example09186a0080afd130.shtml

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/fcoe/421_n1_1/Cisco_n5k_fcoe_config_gd_re_421_n1_1_chapter3.html