cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4436
Views
10
Helpful
7
Replies

RDMA Mellanox support for Cisco UCS.

Kristofer Kawas
Beginner
Beginner

Hi

 

I wonder if anyone can help or answer me if there is support from RDMA Mellanox and Cisco UCS B series or fabric interconnect. It is possible to connect it technically.

I have customers who have Cisco UCS B Series more Windows 2012 R2 HyperV installed, who now want to connect RDMA Mellanox storage support

2 Accepted Solutions

Accepted Solutions

jjmcdani
Beginner
Beginner

Hello Kristofer.

A more important question might be "Why Infiniband?".  With UCS Manager 2.2(4b) firmware Cisco now supports Microsoft SMB Direct with RoCE (RDMA over Converged Ethernet).  Check the requirements and see if this is a better solution for you.  It will require the newer VIC1340 adapter in each blade requiring this feature:

UCSM 2.2 GUI Configuration Guide

Get the Most Complete Solution

You Tube Video from Cisco Joe

View solution in original post

Hi Kristofer

Important health warning: RoCE means RDMA over Converged Ethernet.

If you watch the YouTube mentioned above, you will see, that the SMB Server as well as Client are on UCS blades; therefore no interop Situation; everything happens within a UCS Domain.

In your situation, you have a Extended Converged Ethernet Network (Cisco and Mellanox); your Clients might be on blades, the Servers attached to the Mellanox Switches.

This  is essentially a interop question; I have no clue if it has  been tested or not.

Walter.

View solution in original post

7 Replies 7

Walter Dey
VIP Alumni
VIP Alumni

Cisco UCS Rack Servers with Mellanox ConnectX EN RoE

https://www.cisco.com/web/strategy/docs/finance/mellanox_low_latency_wp.pdf

http://www.cisco.com/c/dam/en/us/products/collateral/switches/nexus-7000-series-switches/le_32804_pb_hpc10ge.pdf

One could connect a rack Mount Server with the proper VIC adaptor to a UCS Fabric interconnect, and have all the benefits of a B-series System (e.g. Service profiles)

 

 

 

Hi

Thanks for your answer,the problem is my customer wants to connect two Mellanox Ethernet switchs to a UCS blade systems by FI and 1340 mezzenin card, I wonder if it will work, these two switches will be used for SAN traffic that I mean that connects the SAN system to Mellanox switches and then to FI and on to all blades,

I have no rack servers all servers in the system are B200 and 5108 Chassis.

 

/Kristofer

I don't think that Infiniband was ever asked for.

Can you please clarify what you mean by

...my customer wants to connect two Mellanox Ethernet switchs to a UCS blade systems by FI and 1340 mezzenin card, I wonder if it will work, these two switches will be used for SAN traffic that I mean that connects the SAN system to Mellanox switches and then to FI and on to all blades...

do you mean IP storage traffic, e.g. SMB (RoCE) ?

As was indicated by jjmcdani, VIC 1340 supports RoCE, and therefore it is not necessary to have any Mellanox adaptor on the blade.

Hi

 

Thanks walter.

What I meant was that I got a question from a customer who wants to change the usual FC storage with SMB RDMA and that they use two Mellanox switches, so I wondered if it is possible to connect the to the UCS system that they already has two pieces 5108 and 8 B200 blades.
And now i got the information about that and the conclusion that it is possible to do so with the right firmware version and the right mezzanine card.

Hi Kristofer

Important health warning: RoCE means RDMA over Converged Ethernet.

If you watch the YouTube mentioned above, you will see, that the SMB Server as well as Client are on UCS blades; therefore no interop Situation; everything happens within a UCS Domain.

In your situation, you have a Extended Converged Ethernet Network (Cisco and Mellanox); your Clients might be on blades, the Servers attached to the Mellanox Switches.

This  is essentially a interop question; I have no clue if it has  been tested or not.

Walter.