Nexus 7K/77K for DCI with FCoE and Synchronous Replication

[rliessi]

Nowadays we have two links (4G each) for synchronous storage replication (FC) and two links (1G each) for ethernet (L2L) using DWDM.

We are planing reduce costs utilizing FCoE and eliminate two links.

Is anyone has experience utilizing Nexus 7K/77K for DCI with FCoE? Or another solutions for DCI with Unified Fabric and Synchronous replication?

Our DC is about 25 km far.

The datasheet says that is possible up to 80KM (http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/datasheet-c78-730273.html).

But I'd prefer real cases :)

Thanks a lot for your help and sorry for my english :)

Regards,

Ricardo Liessi

[Walter Dey]

my 2c

Are your clients (hosts) using FCoE ? meaning that you decide between FCoE end to end or FC end to end, or a hybrid configuration of FCoE and FC ?

Nexus 7K doesn't support native FC interfaces, therefore only a FCoE end to end solution would be possible.

Multihop FCoE is definitely rather new; most of the FCoE installations are single hop in the access layer.

If this is a conservative customer environment, I would go with FC; if you like new technologies and challenges, go with FCoE.
 

[Walter Dey]

Hi Ricardo

- I assume you do storage subsystem based synch. mirroring, not host based ? what is your storage vendor ?

- storage vendors require dedicated links for synch mirroring; therefore no sharing of IP and storage traffic over FCoE; dedicated FC link over FCoE, or native FC

- FCIP; yes, it requires a MDS 92xx type of switch

- you connect FI with FC to what type of switch ? (cannot be N7K); why not use a N5K as a hybrid FC/Ethernet device ? and use FC for your WAN link ?

[rliessi]

Hi wdey

a) - Correct it's for synch. mirroring. It's IBM Storwize v7000

b) - Ok.  But can I connect the storage to FI (FC) and FI to 7000 (FCoE) and forward to other DC? But if there is the distance limitation to use FCoE with other Ethernet traffic, thats isn't a option.

c) - Do you have good experiences using FCIP for synchronous replication?

d) - If FCoE for DCI isn't a option, I'm thinking to use the N5K or maybe 4500-X for WAN aggregation and connect the MDS and the FI on it.

I heard that V7000 has a new feature for IP replication, maybe thats a option and I wouldn't need the MDS.

Whats your consideration?

Thanks again,

RL

[Walter Dey]

b) - Ok.  But can I connect the storage to FI (FC) and FI to 7000 (FCoE) and forward to other DC? But if there is the distance limitation to use FCoE with other Ethernet traffic, thats isn't a option.

I'm still not clear what you mean; do you want to connect your IBM storage with FC directly to FI ? this is not supported ! only Netapp, EMC and HDS arrays.

Is your IBM storage having clasical FC as well as FCoE interfaces ?

c) - Do you have good experiences using FCIP for synchronous replication?

Yes, and there is no problem to do synch. mirroring over 25km

I would not go with a solution, doing FCIP directly on the storage subsystem; it's very expensive, and the management boundaries between storage and networking are unclear !

[rliessi]

- I'm still not clear what you mean; do you want to connect your IBM storage with FC directly to FI ? this is not supported ! only Netapp, EMC and HDS arrays.

Really? I thought that the FI in Switch Mode I'll could attach my Storage with FC.

See: http://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-infrastructure-ucs-manager-software/116082-config-ucs-das-00.html

- Is your IBM storage having clasical FC as well as FCoE interfaces ?

Now I have only FC interfaces, but I can plan an upgrade if necessary.

- Yes, and there is no problem to do synch. mirroring over 25km

Good to know. Do you use MDS for that?

I'll have a meeting next week with Cisco's engineers. Let's see what they think too.

[Walter Dey]

Sorry I have to correct one statement I made: It seems that IBM Storewize is supported in direct attachment; seems to be rather new !

see p 44

http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/interoperability/matrix/r_hcl_B_rel2-21.pdf

and

5 Direct Connect FC/FCoE topologies are supported with this array

and yes, synch mirroring done with MDS !

[richbarb]

Hello Ricardo.

Keep in mind that two FCoE links with "E" ports (fabric between two switches) are dedicated storage protocol links, you can't use those to carry regular ethernet and storage traffic at the same time.

You will not eliminate two links, but you can do is upgrade the bandwidth somehow.

Regards.

Richard

[rliessi]

Hi Richard, I thought that with N77-F348XP-23 or N77-F248XP-23E I will could use FC and Ethernet, differently of N7K-F248XP-25 or N7K-F248XP-25E that says in the DataSheet to use only FCoE. "FCoE traffic is supported up to 70 km, in pure FCOE environment"

So, the same thing applies to N77 module? =/

wdey, I'm planning to use UCS as host and my storage with FC connected to FI.

What do you all think about FCIP? In that case I'll need the MDS 9200, right?

Thanks for the help.

Regards,

RL