FCoE Pass Through

[visitor68]

Let's say we have an FCoE pass-through switch sitting between an  Enode's CNA and the FCoE port of an FCF.

Although the FCoE pass through  switch is invisible to the FC stacks on both ends, its existence does  impact the manner in which the FC stack of the FCF will process the FC  ULP traffic. Namely, this topology creates a new paradigm for FC because  FC is a point-to-point based technology, but connecting an FCoE pass  through device to an FCF presents to the FCF a point to multipoint  (p2mp) construct, in which the traffic flows from multiple E ports (lets  say blade servers with CNAs) are aggregated and fed to the FCF FCoE  port. So, the FCF FCoE port sees multiple endpoints (p2mp) that it  cannot present to the FC stack as such.

The  fix for that is to use FIP to create multiple logical point-to-point virtual FC  links, using the concept of virtual FC interfaces. The triplet used to  define these virtual FCoE links is the MAC address of  endpoint A, the  MAC address of endpoint B, and the VLAN that they  communicate on. The  FCF will then present to its FC stack a collection of  virtual  point-to-point links, which the FC stack will understand and be able to  process.

My  question then is this: What is "endpoint A" and "endpoint B" in this  scenario? My gut tells me that endpoint A is the CNA, endpoint B is the  FCF's FCoE interface and they both will belong to the same VLAN. Is that the case? If  so, why do we need a VE to VE link between the FCoE pass through  switch's uplinks and the FCF's downlink? It seems that as long as the  FCoE pass through can support FIP snooping, that logical construct (the  virtual connection triplet) with the endpoints I just described should  be possible.

Thoughts?

[Manish Tandon]

Hi

I believe you will get a conclusive answer to this if you were to check on the Nexus 4000/5000 boards.

I will chime here in here with what I know of -

Both FIP snooping and VE ports are trying to achieve the same thing i.e multi-hop FCoE.

FIP snooping stretches the "link" between the VN_port and the VF_Port by having DCB switches (like the Nexus 4000) in between.

VE_Port leverages FCoE wires between FC switches.

This url talks abt configuring Nexus 4000 to Nexus 5000 with fip snooping on the Nexus 4000. Its basically a port channel between the two as seen from the config.

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps10596/deployment_guide_c07-574724.html

Thanks

--Manish

[visitor68]

Manish:

Good stuff.

Sorry it took me so long to get back to you. Been very busy...

Anyway, let me present some of my thoughts to you and I would love to hear your feedback.

My understanding with regard to the FCoE roadmap, as it pertains to the extension of the FCoE domain, is to extend the lossless Ethernet network past today's FCoE demarcation point, which today is the ToR, to the EoR, and then to the core. Moreover, that path may be part of what we would call a fabric (self-healing, built-in intelligence, horizontal expansion, full bi-sectional bandwidth availability, extended control plane across physical switch boundaries, etc), or simply a patchwork of lossless Ethernet switches connected to each other via the classic 802.1D architecture (STP, blocked uplinks, 50% bi-sectional bandwidth, necessary reconvergence after every fault, etc). Either way, the FCoE traffic will leverage a lossless Ethernet path.

This having been said, what happens to the ToR FCF switch that presently terminates the FCoE domain and does NOT have  FSB (FIP-Snooping Bridge) capability? I don't understand what is to be done with the Brocade B-8000 or Cisco Nexus 5000 when data center architects want to take  the next step and make the EoR switch the new demarcation/termination  point for FCoE traffic - meaning, the point at which FC and Ethernet go  their separate ways. Will the Nexus have to be ripped out? Perhaps I am wrong and the B-8000 and Nexus 5000 ARE indeed FSBs, which  would mean that the FC ports would no longer be used on them, and the 10G CEE  uplinks to the EoR will simply carry the FIP/ FCoE traffic to the EoR, where the new FCF will terminate the FCoE path. This question isn't  vendor specific. Thoughts?

Lastly, I have a question with  regard to the Nexus 5000 supporting a connection to an FSB. I am asking this  because there are discussions being had among data center architects,  and there are proposed reference architectures being submitted, in which  a blade switch acting as an FSB (and a normal port aggregator, like  every other switch, not a 1:1 pass-through) is connected to a single Nexus 5000 10G CEE port. So, to be crytstal clear, I am talking about a blade  chassis with multiple blade servers, each with a CNA, all connected to  an FSB Ethernet blade switch that aggregates all the CNA traffic and is  connected/uplinked as a "trunk" to ONE port on the Nexus 5000.

Is the N5K capable of supporting multiple FLOGI logins on a single port? If not, that means that reference architecture will not work. Yes or no?

Regards

[estine]

Hi Ex-Engineer:

Maybe I can help shed some light on your questions -- you bring up some good ones.

Today in native FC, there are two ways to interconnect FC switches within a fabric: either through E ports, where both switches have FC Services running (equal to an FCF in FCoE) and through NPV/NPIV connections where the NPV device acts more like a login proxy to the existing FC fabric and does not run fabric services.  There is no concept of a FSB or any pass through device in native FC today.

Moving to FCoE, the capability to attach and FCF to another FCF using VE_Ports (analogous to E ports in the FC world) is already shipping on the Nexus 5000 platform.  This means that you can connect an Nexus 5000 to another FCFs today to create an FCoE multihop solution.  Therefore, to answer your question: NO, you will not need to rip out any existing Nexus 5000’s in order to support an end-to-end FCoE solution. You can connect them to FCoE enabled Agg/Core devices using VE_Ports. 

In the near future, we expect that the NPV/NPIV functionality will enter the FCoE world on the Nexus 5000 product line to offer another way to connect multiple FCoE devices.  There are a few benefits that this NPV model in FCoE has over a FSB.  One is that you are no longer relying on the underlying l2 (STP) protocol to carry your FC traffic.  The other is that since an NPV device is actually FC aware, it can make traffic engineering and deterministic decisions with regards to your FCoE traffic where as an FSB only can switch FCoE based on the MAC address.

Also, to your last question: the Nexus 5000 does support multiple FLOGI terminations for the reason you just described.

Thanks