I'm setting up a UCS environment with a pair of MDS 9148's for the first time. The fabric interconnects are 6120XP's with the 6 port 8Gb fc expansion modules. Here is what I've done: I enabled npiv on both 9148's, I've assigned the FC ports on the fabric interconnects to the appropriate VSAN, I've assigned the ports on the 9148's to the appropriate VSAN, I've created a service profile template with vnic's and vhba's (the vhba's also being assigned to the appropriate vsan, and then I created two service profiles from the templates.
I see that the two machines I created have logged into the fabric as I can see the wwn's in fabric manager. I then created the alias and zones and saved the configuration. I went to our storage managment software and allocated a LUN to each server. Each server will be an ESXi server that is booting from SAN. So I mount the ESXi iso to the machines and boot to the iso. The machines boot to the iso and I can install ESXi onto the LUN (the machines see the boot LUN). There is no local disk in these machines. ESXi successfully installs and the machines reboot and I start configuring ESXi. I go to lunch and come back and the ESXi servers have lost connectivity to the SAN. I validate that ports on the 9148's are in the appropriate vsan, and the fabric interconnect ports are in the appropriate vsan, and the vhba template is using the appropriate vsan.
In Fabric Manager the connections from UCS to the individual servers I setup have a red X through them and when I click on the red X it takes me the connection of the vhba and says unavailableNPVUpstreamPort and the Port VSAN says 1, which is not the appropriate VSAN. How did this get like this and how to I address it? All the ports on the 9148's and fabric interconnects are in the appropriate vsan.
- how many FC links did you setup per fabric ?
- are the VSAN's id's for fabric a and b different ?
- are they all configured identical
- are you using port channel trunking
- did you do copy run start on the MDS
- did you check the up time of MDS and FI (e.g power fail and then reboot)
- are all the vhba's of your SP in the proper VSAN ?
I opened a case with TAC and apparently it is some kind of bug. I use vhba templates in my service profile templates. The vhba templates are configured for the appropriate vsan, but when you attach them to the template they lose the configuration. I usually just add a couple of vhba's using vhba templates into my service profile template. So I screwed around and created a SAN Connectivity Policy and added the same vhba templates into the policy and then assigned the policy to the service profile template (which is still bound to the service profiles) and everything all of a sudden connected without issue.