UCS unable to see shared storage

[sytomas]

I just upgraded UCS to 2.2(1b) and I'm trying to install ESXi 5.1 U1 on a B200 M3 with VIC1240.  I can see the Flogi's on the FC switch, VIC can see the luns using lunlist in CLI, but the ESXi installer cannot see any remote storage. 

The installer I used was the Cisco customized installer from VMware.  It worked fine on my other blades when on version 2.1(1a).  Has someone run into this before?  Was there a change in 2.2 that I need to modify my SP Template to get it to recognize the remote storage? 

[Manuel Velasco]

Hi Sytomas,

Please provide the output of lunlist and lunmap.

[sytomas]

Hi Manvelas,

Here you go:

zoning information on MDS
Fab-A
zone name esx_02 vsan 1010
  pwwn 20:00:00:25:b5:aa:00:03
  pwwn 21:00:00:00:87:b5:39:85
Fab-B
zone name esx_02 vsan 1020
  pwwn 22:00:00:00:87:b5:39:85
  pwwn 20:00:00:25:b5:bb:00:01

UCS LUNLIST
lunlist:
adapter 1/3/1 (fls):1# lunlist
vnic : 16 lifid: 5
  - FLOGI State : flogi est (fc_id 0x010304)
  - PLOGI Sessions
     - WWNN 21:00:00:00:87:b5:39:85 WWPN 21:00:00:00:87:b5:39:85 fc_id 0x0100cc
       - LUN's configured (SCSI Type, Version, Vendor, Serial No.)
           LUN ID : 0x0000000000000000 (0x0, 0x3, NETAPP  , VAXKEJNA)
       - REPORT LUNs Query Response
           LUN ID : 0x0000000000000000
  - Nameserver Query Response
     - WWPN : 21:00:00:00:87:b5:39:85

vnic : 17 lifid: 6
  - FLOGI State : flogi est (fc_id 0x020204)
  - PLOGI Sessions
     - WWNN 22:00:00:00:87:b5:39:85 WWPN 22:00:00:00:87:b5:39:85 fc_id 0x0200cc
       - LUN's configured (SCSI Type, Version, Vendor, Serial No.)
           LUN ID : 0x0000000000000000 (0x0, 0x3, NETAPP  , VAXKEJNA)
       - REPORT LUNs Query Response
           LUN ID : 0x0000000000000000
  - Nameserver Query Response
     - WWPN : 22:00:00:00:87:b5:39:85

UCS LUNMAP
adapter 1/3/1 (fls):2# lunmap
lunmapid: 0  port_cnt: 1
  lif_id: 5
    PORTNAME                 NODENAME                 LUN               PLOGI
    21:00:00:00:87:b5:39:85  00:00:00:00:00:00:00:00  0000000000000000  Y

lunmapid: 0  port_cnt: 1
  lif_id: 6
    PORTNAME                 NODENAME                 LUN               PLOGI
    22:00:00:00:87:b5:39:85  00:00:00:00:00:00:00:00  0000000000000000  Y

adapter 1/3/1 (fls):3#

See Screenshots:

I changed the FW of the VIC back down to 2.1(3a) as I thought it was a FW issue of the VIC on 2.2.  the VIC can see the storage at boot up (with both 2.1(3a) and 2.2(1b))

Even with ESX5.5 Cisco custom drivers, no drives can be seen.

[Walter Dey]

I see one weird thing, why are pwwn and nwwn identical ! please change this asap !

WWNN 21:00:00:00:87:b5:39:85
WWPN 21:00:00:00:87:b5:39:85

WWNN 22:00:00:00:87:b5:39:85
WWPN 22:00:00:00:87:b5:39:85

Can you please post a screenshot of your vhba's !

[Walter Dey]

Questions

1) do you have any other servers, which access successfully over this fc path lun's on this netapp filer ?

2) you use n-port trunking channeling ! has this been verified and is working ok

3) what version of fnic/enic driver do you have on your ESXi

Flogi shows

pwwn 21:00:00:00:87:b5:39:85 nwwn 20:00:00:00:87:b5:39:85

LUNLIST shows

WWNN 21:00:00:00:87:b5:39:85 WWPN 21:00:00:00:87:b5:39:85

LUNMAP shows

21:00:00:00:87:b5:39:85  00:00:00:00:00:00:00

I trust flogi; maybe it's a caveat ofLUNLIST and LUNMAP ?

[Manuel Velasco]

Hi Sytomas,

From the logs you provided we can see that the vHBAs are able to communicate to the storage array as the PLOGI is there, so we know for sure the communication between these two devices exist.

One thing I did noticed that you mentioned that you are installing ESXI5.1 U1 but from the screenshot you sent me I see that the installer version is for ESXi 5.5.  So one question I have is, are you trying to install esxi 5.5 or 5.1? Also can you provide the version of the ESXi cisco customize image that you are using?

How many blades are being affected? Is only the b200 m3 affected by this?

For troubleshooting steps, I would recommend the following:

1. Enable a single path to the storage to isolate the issue a bit more, for example you can leave the path from fabric A to the storage and disable the path from fabric B.

2. If you haven’t tried, please use a different operating like windows, when you do this please make sure to load the proper drivers.

3. Verify the storage masking is correct

adapter 1/3/1 (fls):2# lunmap
lunmapid: 0  port_cnt: 1
  lif_id: 5
    PORTNAME                 NODENAME                 LUN               PLOGI
    21:00:00:00:87:b5:39:85  00:00:00:00:00:00:00:00  0000000000000000  Y    <-- Plogi is there

[sytomas]

All my B200 M3's are affected.  I tested both ESX 5.5 and 5.1 using the Cisco custom ISO and neither worked:

5.1 = ESXi-5.1.0-1065491-custom-Cisco-5.1.1.1

5.5 = ESXi-5.5.0-1331820-custom-Cisco-5.5.0.1

I have some pre-2.2 B200 M3 blades that are already booting from SAN and can see it's LUN and datastore.

I also tried using iSCSI boot as well trying to mount Nimble storage.  The blades see the IQN target during boot, but still get the same problem. Once I'm able to get the OS installed on local disks, it see's the datastore on the SAN just fine.  So I'm thinking it's the ESXi installer. 

I haven't tested another baremetal OS, but will try with Windows next.  I'm just trying to think what has changed that the installer for 5.1 works on 2.1 but when I upgraded to 2.2 no new installations can see the remote drive, but existing ESXi Boot From SAN works just fine.  

[Walter Dey]

Did you ever try the non Cisco OEM iso package from VMware ?

[Walter Dey]

I don't know any caveats, that a migration to 2.2.1b would bring.

If the installer doesn't see your lun's, someting must be wrong either with the fibrechannel zoning, and/or lun masking/mapping.

can you please post the cli of

1) show flogi database vsan ... where we see the pwwn of the initiator

2) the show show zoneset active vsan ... on the fc switch, where we should see a zone with the above initiator and the pwwn of the storage target

[sytomas]

Hi wdey

Here it is:

FAB-A Flogi Database
Flogi Database for VSAN 1010
MDS9134-A# sh flogi data vsan 1010
--------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME
--------------------------------------------------------------------------------
fc1/31           1010  0x0100cc  21:00:00:00:87:b5:39:85 20:00:00:00:87:b5:39:85
                           [DISK-13-SAN-A]
port-channel 160 1010  0x010300  24:a0:00:2a:6a:60:6a:00 23:f2:00:2a:6a:60:6a:01
                           [UCS-FI6248-A]
port-channel 160 1010  0x010304  20:00:00:25:b5:aa:00:03 20:00:00:25:b5:aa:00:02

Total number of flogi = 18.

MDS9134-A#

FAB-B Flogi Database
MDS9134-B# sh flogi data vsan 1020
--------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME
--------------------------------------------------------------------------------
fc1/31           1020  0x0200cc  22:00:00:00:87:b5:39:85 20:00:00:00:87:b5:39:85
                           [DISK-13-SAN-B]
port-channel 161 1020  0x020200  24:a1:00:2a:6a:57:d8:00 23:fc:00:2a:6a:57:d8:01
                           [UCS-FI6248-B]
port-channel 161 1020  0x020204  20:00:00:25:b5:bb:00:01 20:00:00:25:b5:aa:00:02

Total number of flogi = 21.

MDS9134-B#

FAB-A Zoneset
MDS9134-A# sh zoneset active vsan 1010
zoneset name ucs_fab_a vsan 1010
 
  zone name esx_02 vsan 1010
  * fcid 0x010304 [pwwn 20:00:00:25:b5:aa:00:03]
  * fcid 0x0100cc [pwwn 21:00:00:00:87:b5:39:85] [DISK-13-SAN-A]

MDS9134-A#

FAB-B  Zoneset
MDS9134-B# sh zoneset active vsan 1020
zoneset name ucs_fab_b vsan 1020

  zone name esx_02 vsan 1020
  * fcid 0x0200cc [pwwn 22:00:00:00:87:b5:39:85] [DISK-13-SAN-B]
  * fcid 0x020204 [pwwn 20:00:00:25:b5:bb:00:01]

MDS9134-B#

cheers,

Syd