cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2716
Views
0
Helpful
12
Replies

Nexus 5k Topology/ Extending FCoE between 2 campuses

mta2891974
Level 1
Level 1

Hello,

I need help to answer the following questions related to interconnecting 2 campuses (Upper and Lower in the attached diagram), using the Nexus 5K FCoE ports:

Kindly check the attached topology.

QUESTIONS:

Multi-hop FCoE:

·         Can the link (10G,SFP+)  between the pairs of N5k, interconnecting the 2  campuses, carry FCoE traffic (both Ethernet and FC traffic over the same link):

·         Can the Servers with CNA adapters in the Lower Campus, access the FC storage in the Upper Campus?

·         Can the Servers with FC HBA adapters in the Lower Campus, access the FC storage in the Upper Campus?

·         Can we extend the 2 FC fabrics between the 2 sites using this FCoE link?

·         Can this Link carry Ethernet traffic between the 2 LANs (between Upper and Lower Campuses)

Thank you.

2 Accepted Solutions

Accepted Solutions

so it's 3km for FCoE traffic. but in case of FC traffic (over the FCoE link), what i mean if we have Servers with FC HBA adapters in the Lower Campus, accessing the FC storage in the Upper Campus), is it always 3km limitation? or even less?

-> should be 3km as well, independent how targets and initiators are connected. (Limitation is for a single FCoe hop)

Concerning using the native FC option to extend this link (of course withouth ethernet), do we have to go for an MDS switch? or simple use the FC module of the 2 Nexus 5k?

-> you can do it directly using FC Module if you are running the N5K in switching mode (not in NPV Mode)

do you have any reference link, mentioning those distances?

-> sorry I also got this infos via Cisco SE

Hubert

View solution in original post

Hi Michel, looking on the cisco-platforms supporting FCoE

MDS: As I know so far FCoE on MDS supports only the transport of FC-frames over Ethernet (no native Ethernet-traffic), so no unified Wire, that means you need a dedicated wire for FCoE tranport

N7K: at FCS N7K will support FCoE only in a seperate VDC. Because assignment to a VDC is Interface (Port-group)-based you have to use dedicated Wire for FCoE transport ( no native Ethernet-traffic)

N5K: with N5K I think you do not have the limitation that you have to seperate FCoE traffic from native Ethernet-traffic, VE Ports can also transport Ethernet-traffic AND FCoE Traffic, like VE-Port can do. So for FCoE transport between N5K you can use unified wire. The only limitation is the distance and the maximum hop-count of 7.

KR

Hubert

View solution in original post

12 Replies 12

rammuthaiya
Level 1
Level 1

Questions 1,4,5

It is possible but we have to use the  long Reach SFP+ module.

Questions 2,3

Need to do zoning accordinly. But it is possible.

Regards,

Ram

Hi,

It doesn't matter if it's Long Reach or short Reach SFP+, this will be defined by the distance between the 2 campuses. IT's not related to any of my previous questions.

HUBERT RESCH
Level 3
Level 3

Hi, as I know so far FCoE multihop is supported between 2 N5Ks , but the problem is the limitation for the distance to 3 kms(independent of the optics).

(Reason for this limitation is bufferlimit of N5K Ports) If this 3 km are sufficent for you, yes aou can use this FCoE-Link for Ethernet as well.

If you need longer distance between the two N5K you should use natice FC which allows you up to 16 km between the two N5K, of course no way to transport ethernet via the same link.

Hubert

Thank you for your answer.

so it's 3km for FCoE traffic. but in case of FC traffic (over the FCoE link), what i mean if we have Servers with FC HBA adapters in the Lower Campus, accessing the FC storage in the Upper Campus), is it always 3km limitation? or even less?

Concerning using the native FC option to extend this link (of course withouth ethernet), do we have to go for an MDS switch? or simple use the FC module of the 2 Nexus 5k?

do you have any reference link, mentioning those distances?

Thank you

Michel

so it's 3km for FCoE traffic. but in case of FC traffic (over the FCoE link), what i mean if we have Servers with FC HBA adapters in the Lower Campus, accessing the FC storage in the Upper Campus), is it always 3km limitation? or even less?

-> should be 3km as well, independent how targets and initiators are connected. (Limitation is for a single FCoe hop)

Concerning using the native FC option to extend this link (of course withouth ethernet), do we have to go for an MDS switch? or simple use the FC module of the 2 Nexus 5k?

-> you can do it directly using FC Module if you are running the N5K in switching mode (not in NPV Mode)

do you have any reference link, mentioning those distances?

-> sorry I also got this infos via Cisco SE

Hubert

thanks a lot

johgill
Level 1
Level 1

vPC and FCoE can not coexist on the same wire beyond any first hop access device.  The VE ports must be dedicated from site to site, separate from the normal ethernet traffic.

Regards,
John Gill

Hi John,

vPC is not spanned between the 2 campuses. There's no vPC across the 2 sites.

However, the link between the 2 sites (connecting the VE ports) is a FCoE link and therefor it carries FC traffic (over Ethernet) AND normal Ethernet traffic.

What do yo mean by  "The VE ports must be dedicated from site to site, separate from the normal ethernet traffic."

thank you

Michel

Hi Michel, looking on the cisco-platforms supporting FCoE

MDS: As I know so far FCoE on MDS supports only the transport of FC-frames over Ethernet (no native Ethernet-traffic), so no unified Wire, that means you need a dedicated wire for FCoE tranport

N7K: at FCS N7K will support FCoE only in a seperate VDC. Because assignment to a VDC is Interface (Port-group)-based you have to use dedicated Wire for FCoE transport ( no native Ethernet-traffic)

N5K: with N5K I think you do not have the limitation that you have to seperate FCoE traffic from native Ethernet-traffic, VE Ports can also transport Ethernet-traffic AND FCoE Traffic, like VE-Port can do. So for FCoE transport between N5K you can use unified wire. The only limitation is the distance and the maximum hop-count of 7.

KR

Hubert

Thanks again

Sorry for being too general, the vPC statement doesn't apply to your setup here so it would be up to you to use shared wire for your traditional LAN VLANs and FCoE VLANs.  You could not transition these links to vPC in the future though, due to the restriction of vPC and FCoE not being allowed to share a wire except for on access nodes.

It is generally preferred to have separate VE links just for FCoE due to the advantage vPC can provide for the traditional ethernet and the storage links can be configured based on fabric isolation requirements.

Food for though, sorry for any confusion.

Regards,

John

mta2891974
Level 1
Level 1

no problem. Thanks a lot for your clarification

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: