Nested NPV

[anazarenko]

Dear All,

I want to make the following setup

Brocade FC switch <--- FC ---> Nexus 5k  in NPV mode <--- FCoE---> IBM CN4093  Converged switch in NPV Mode <--FCoE--> ESXi in NPV mode <--FCoE --> VM with RDM.

Can you please advise if 3 nested NPV switches are supported? Or it's allowed only 2 times to chain NPV switches?

[Walter Dey]

It has been discussed before:

https://supportforums.cisco.com/discussion/11572501/npv-supported-topology

https://supportforums.cisco.com/discussion/11611656/ucs-n5k-and-npv-mode

https://supportforums.cisco.com/discussion/12349391/npv-feature-needed

Not supported !

[anazarenko]

That's not logical. Brocade switch is under Storage team administration. Nexus 5k is under  Network team and can't be part of Brocade fabric. Blade center is a customer of Nexus. If this configuration not supported, it means each blade center requires a dedicated connection into the brocade FC network.

What is the best practice for such scenarios?

[Walter Dey]

Just to clarify; Cisco talks about NPV and NPIV. Usually a access device is a NPV, connecting to a core which is NPIV. NPV acts as a HBA concentrator, is not a FC switch; NPIV is a full function FC switch. Can you please clarify your diagram accordingly.

What is not supported is: NPV -- NPV -- NPIV.

[anazarenko]

Here's the topology

Brocade FC switch(NPIV) <--- FC ---> Nexus 5k  in NPV mode (maybe additional NPIV mode) <--- FCoE---> IBM CN4093  Converged switch in NPV Mode <- FCoE-> converged adapter on ESXi

[Walter Dey]

NPV and NPIV are mutually exclusive !

Enable FCoE and then enable NPV—This method requires that you enable FCoE first using the feature fcoe command and then you enable NPV by using the feature npv command. When FCoE is enabled, the default mode of operation is FC switching. When you enable NPV, the mode changes to NPV mode. Switching to NPV mode automatically performs a write erase and reloads the system. After the reload, the system comes up in NPV mode. To exit NPV mode and return to FC switching mode, enter the no feature npv command. Exiting NPV mode also triggers a write erase and a device reload. This method requires the Storage Protocols Services Package (FC_FEATURES_PKG) license.

What could work, although you will have the trouble of interoperating Brocade and Cisco; and merges Brocade and Cisco, which you don't allow.

Brocade FC switch(NPIV) <--- FC ---> Nexus 5k  in NPIV mode  <--- FCoE---> IBM CN4093  Converged switch in NPV Mode <- FCoE-> converged adapter on ESXi

or

Brocade FC switch(NPV) <--- FC ---> Nexus 5k  in NPIV mode  <--- FCoE---> IBM CN4093  Converged switch in NPV Mode <- FCoE-> converged adapter on ESXi

[anazarenko]

I think the following setup should work:

Brocade FC switch(NPIV) <--- FC ---> Nexus 5k  in NPV mode  <--- FCoE---> IBM CN4093  Converged switch  FCoE Forwarder <- FCoE-> converged adapter on ESXi

I was wrongly assuming that IBM CN4093  Converged switch must be in some mode as well, but it's just FCoE transit switch.

[Walter Dey]

This is weird: in the drawing it says FCoE Forwarder (FCF) which is a FCoE switch, and in your explanation you say "FCoE transit switch".

In your proposal: nexus 5k acts as a FCoE NPV which implements FIP snooping.

From http://www.redbooks.ibm.com/abstracts/tips0910.html?Open it is not clear, if 4093 supports FCoE NPV

If the IBM 4093 would also be a FCoE snooping bridge, you would have a cascade of 2 such devices and a interop challenge (lossless multihop ethernet)..

[anazarenko]

Hi

First of all Nexus is connected to the Brocade fabric via native fabric-channel in NPV mode.  This is decided and can't be changed.

Brocade FC switch(NPIV) <--- FC ---> Nexus 5k  in NPV mode 

Then there's a multihop FCoE from Nexus 5k towards CNA on Vmware esxi host.

Ideally bladeswitch IBM CN4093  should act as a FCOE NPV node and proxy all FIP messages towards Nexus 5k. Then Nexus 5k will terminate FIP and proxy only FCP messages towards the native FC fabric in FC NPV mode.

I call it nested NPV setup. But as I understood it can't work?

[Walter Dey]

my 2c

- I understand your requirements

- Cisco doesn't support nested NPV

- you have even a more complex situation: nested FCoE with multivendor

- does it work ? Don't know ? is it supported ? ask IBM, Cisco's answer above

- anyway not clear, if CN4093 can be configured as a FCoE NPV ?

- is the CN4093 a active IBM blade component ? and if not, why not bypassing it ?

[bchintak]

Cisco doesn't support nested npv. (UCS is exception)
It is visible if you connect in nested npv mode  and check the command "show npv internal errors"