UCS mini - Pod config - FC switch mode - EMC VNX5400 panics

[Charles Morrall]

I am currently deploying a UCS mini with 6324 fabric interconnects. This is converged infrastructure solution where I have connected the VNX FC ports directly to the 6324 interconnects, and I've configured the FI modules in Switch mode, as suggested by the various guides.

I do not have any other FC switches or any other storage, using Switch mode is the obvious configuration option to me.

For the most part it works fine, zoning is working and I get all paths OK to the VNX from the UCS blades.

However, on two occasions it would seem, when rebooting the FI modules during our initial integration or when doing firmware maintenance, the VNX has crashed hard. Both its storage processors have paniced and rebooted, which of course means a complete loss of storage connectivity.

I'm looking for any others who has deployed such a configuration, and if anyone can share their experiences.

Naturally, I have a case with EMC and they are analyzing the crash dumps, but I find it very hard to believe a tried platform such as the VNX should suffer such a catastrophic failure all on its own.

[Walter Dey]

Hi Charles

Which version do you have 3.0.2e ?

Several caveats with FC direct attach have been fixed, see

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/notes/ucs_3_0_rn.html#pgfId-310509

and

https://supportforums.cisco.com/discussion/12750406/ucs-mini-fc-port-remains-down-when-sync-lost

The interop matrix (I don't see any newer one)

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx/Matrix8.html#wp323848 says

Supported Storage Arrays		Rel. Version	Notes
Vendor	Model
EMC	EMC Symmetrix family (VMAX & DMX), EMC CLARiiON (AX & CX) EMC Unified Storage VNX EMC Celerra (Native Block)	1.3(1), 1.4(1), 1.4(2), 1.4(3), 1.4(4), 2.0(1q), 2.0(1s), 2.0(1t)	note 1, 2, 3, 12

Notes:

1) For detailed information on supported Servers, CNAs, and drivers, refer to the Cisco Hardware and Software Interoperability Matrix: http://www.cisco.com/en/US/products/ps10477/prod_technical_reference_list.html

2) Refer to the EMC ESM matrix for support storage firmware versions and specific supported configuration information: http://www.emc.com/products/interoperability/

3) Direct-Attached FC storage to 6120XP/6140XP/6248UP is supported in release 1.4(1) and higher. However, it is restricted to topologies in which the zoning database is provided from an upstream Cisco MDS 9000 switch or Nexus 5000 or 5500 switch.

[Charles Morrall]

We updated the UCS mini to 3.0.2e, and the VNX crashed with a dual SP panic again during the reboot of the Fabric Interconnects.

The interop guide seems to cover the MDS9000 line, which I'm not using in this setup.

I've cross checked the EMC support matrix, nothing really stands out. Possibly the topology listed is "FCoE", but I'm using native FC from the egress point on the Fabric Interconnects. The VNX came with 8 Gbps native Fibre Channel ports, not FCoE adapter cards.

The note 3 I hope does not apply to FI 6324, and I have configured them in FC Switch mode and doing zoning internally on the Fabric Interconnects.

There is some hope, I tried another reboot of the Fabric Interconnects today, and this time the VNX remained stable.

[Walter Dey]

Hi Charles

In the past, EMC required a MDS on a stick, pushing the active zoneset to the FI. The FC switch implementation on the FI is not 100% compliant with MDS SAN-OS, and EMC did all their certifications with MDS.

What happens if you disable a FC port on the FI ? Does the VNC crash as well ?

Do you see any log messages on UCS ?

Walter.