Im wondering if someone can clear up one thing for me….
What exactly does a UCS FEX do besides aggregate the server traffic and push it up to the Fabric Interconnects? What I am wondering is what functionality/technology does it need to possess to do that?
From my understanding, a conventional blade switch needs to be DCB-capable (support at least PFC and ETS) and do FIP snooping to be used as an FCoE pass-through. IS that the case with a UCS FEX?? I dont think so…it seems like nothing more than a “dumb” MUX that has no intelligence, no code to upgrade and simply passes DCB traffic between CNA and FI…
What am I missing, if anything?
Hi, Eric. That much I understand.
What I dont understand is the disparity in requirements from a blade switch and a UCS FEX when it comes to being capable of passing FCoE traffic.
A blade switch has to do FIP snooping, support PFC, ETS and DCBx....a UCS FEX is nothing but a "bump in the wire" - so how does it get away with passing FCoE traffic to the FIs from the CNAs and back??
A blade switch provides a local switch as well as additional management point. The FEX is an extended line card for the parent switch. The FEX is a fabric architecture where the blade switch is an isolated switch.
Attached is a picture which shows the internal architecture of the IOM.
Will not delve into what has been already discussed - chassis mgmt, cabling, mgmt ease etc.
Yes, one of the key functions is mux traffic from HIF's to NIF's and vice versa.
Traffic from HIF's is unconditionally sent up to the FI for switching.
The ASIC is programmed via the CMC . It has the ability to replicate frames as is done with multicast traffic.
If you have2 recievers connected to the same IOM interested in the same multicats group, there is 1 packet which is sent by the FI and the replication
is handled by the ASIC. The ASIC also does tag imposition/removal etc (part of MUX functionality). The ASIC also provides QoS/buffering functionality.
So yes, packet forwarding to the SUP following a line card paradigm is its main purpose in life.
Manish, as usual, great stuff. Thank you..
So, what you are telling me is inline with my thoughts about the UCS FEX. There is no switching intelligence in it and -- more pertinent to this discussion -- no Data Center Bridging (CEE) capability to speak of. Its just a traffic MUX/DeMUX.
That means the UCS FEX HIF ports will NOT act as an 802.1D switch port does when it is DCB-enabled and generate a PFC PAUSE frame and send it back to the server CNA if its input buffers get overwhelmed. I Iimagine then that the UCS FEX leaves it up to the Fabric Interconnect's ingress port to generate the PUASE and the FEX will simply pass it on to the CNA. Correct? That would mean the FEX has no input buffers to speak of. If it did, it would have to be able to PAUSE traffic that is overrrunning them - or so I would think.
The same then can be said for the FEX and ETS. The FEX does not contain the hardware to participate in ETS. There is no egress scheduling algorithm and no recognition of packet prioritization or a mechanism to honor any prioritization that can be leveraged to dynamically assign bandwidth to the confgiured Traffic Class that would be serviced acording to the ETS Transmission Scheduling Algorithm (Algorithim 2, according to IEEE 802.1qaz v2.4).
Lastly, the FEX does not participate in DCBx either - the link initialization semantics occur between the CNA and the Fabric Interconnect port.
Is all this correct, Manish?
I really appreciate your time and help. Thank you!
The IOM is a pass through for switching but it does have buffers and queues.
The DCBX packets between the FI and the adapter are passed as is like a line card would. Same goes for other things like CDP packets etc.
But the FI communicates with the CMC using a diff software path and the ASIC is programmed on the IOM for CoS and bandwidth values.
Think of it as a "qosmgr" process in the system.
So its not DCBX between the FI and IOM but some other way as the IOM is a part of the FI.
The IOM does send PFC when congestion hits to either the adapter or the FI.
When congestion between IOM and FI hits, its too late or away in path for the FI to send PFC to adapter. The IOM has to.
This is how lossless is achieved.