Nexus - FEX performance

[bapatsubodh]

Hi,

If we connect a FEX device to 2 X 6004 Nexus switches with uplinks to each 6004 with VPC as compared to a Cisco L3/L2 switch connected to 2 X 6004 Nexus with trunked-uplinks to each 6004 with VPC, what could be the impact on network speed performance for the hosts connected to FEX or switch ports, especially if the network is to perform in low latency.

Access ports for FEX and Cisco L3/L2 switch is same 1 Gbps-copper.

In both cases the uplink from FEX or switch to 6004 is same (10 G + 10 G ) 20 Gbps.

FEX - Sounds like extending the fabric back-plane of parent 6004 way down to FEX device but still bandwidth is bottle-neck?

Is the protocol with which Nexus 6004 communicates with FEX over VPC- port-channel better than the how it would communicate with VPC-port-channel trunk-port with Cisco switch? (If the combined uplink bandwidth is same in both cases).

Please share the experinece.

Thanks in advance.

Subodh

[Vasilii Mikhailovskii]

Hello.

FEX works in the way, that is can't switch traffic between two ports locally... first packet must walk to parent Nexus,  being processed and then walk back. So, if you have 20x1G interfaces, they might saturate FEX uplink.

Also you might need to take into account FEX uplink binding.

PS: the primary aim of the FEX is to be like TOR switch; not sure if that's what you want.

[bapatsubodh]

Thanks for Reply Vasili.

What I was looking for, if the uplink bandwidth between FEX device two NX 6k is say 20 G.

If we replace FEX by a Switch -uplink bandwidth is 20 G ( 10G + 10G  VPC  - TRUNK) connected to two NX 6K devices.

Will there be any difference in the server access as the bandwidth is same in either case ( 20 G)

We can expect performance difference, in case if the communication between FEX and 6K is better than VPC configured for switch. As bandwidth variable is constant - 20 G combined (10G + 10G)

Any thoughts ... Any stat - numbrs available that can throw some light on two setups..

[Vasilii Mikhailovskii]

Hello.

If you use FEX, then any inter-server communication (attached to same FEX) should run via Nexus (that means you will utilize your 20G for local traffic): traffic received on FEX port, goes to north, being switched on Nexus and then goes south to another FEX (outgoing) port.

If you attach servers to some switch, then the switch will perform traffic forwarding. Surely it would be better to run stack of switches to have single Port-channel.

PS: I assume that FEX would provide you a worse performance if you run a lot of inter-server (within on FEX) traffic and if you have high oversubsription ratio. At the same time, as far as I know, Nexus is the only option for CNA.