Fabric Interconnect 6120XP to Nexus 5548

[darren-carr]

Hi all,

We are in the throws of preparing our Nexus 5548 environment for the unduction of our new Cisco UCS platform. I have a query regarding the interconnectivity between the 6120XP and 5548.

We have two 5548's connected by a 20Gbps vPC Peer Link. We have two 6120XP Fabric Interconnects. I want to create two port-channels, each one made up of 2 x 10Gbps links. Each interconnect will have one connection to each 5548 that will be grouped into a port-channel.

We have a stack of FET-10G's that came as part of a bundle when we purchased the 5548's and 2248 FEX's. I was hoping to use these to connect the UCS to the 5548.

I have got an issue however when I am trying to achieve the above. To enable the FET I have to execute the command switchport mode fex-fabric. However, with this command enabled I can't create the port-channel. Is this a limitation of the SFP and should I be using something else? I'm trying to use the channel-group x mode active/passive/on and neither works as the 5548 is telling me it isn't compatible?

I've searched everywhere for information but can't find the specific answer I am looking for.

Any advice?

Thanks,

Darren

[padramas]

Hello Darren,

FET-10G  SFP is supported only links between FEX and parent switch.

Also, in UCS, it is supported from Gen 2 hardware ( 62xx / 220x ) with 2.0.X software release.

Cisco 10-Gigabit Ethernet Transceiver Modules Compatibility Matrix

http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/compatibility/matrix/OL_6974.html#wp62311

Table 3 NOTE 2

2. Supported for fabric links only, from a Nexus 2000 Series to a Cisco parent switch.

HTH

Padma

[darren-carr]

Hi Padma,

I appreciate you getting back to me. I did see this matrix. The issue I am experiencing is when I try to create a port-channel from the North Bound 'Cisco 5548' to the South Bound 'Cisco 6120XP'.

I am trying to form a Port Channel using e 1/23 on Nexus 5548 (1) and port e 1/23 Nexus 5548 (2).

When I try to apply the configuration 'channel-group 1 mode active' on the e 1/23 port, with the FET-10G SFP installed, the Nexus tells me that it is not supported/compatible.

I was receicing 'SFP validation failed' prior to adding the command 'switchport mode fex-fabric'.

Any idea how I get around this or have a link to a document for creating port-channels to the Cisco 1620XP? I am trying to follow Cisco best pratice by multi-homing the 1620XP to seperate Nexus 5548's but can't seem to form a port-channel.

Am I doing something wrong?

Thanks,

Darren

[padramas]

Hello Darren,

We need to use different SFP to connect Nexus 5548 and Fabric Interconnect ( FI ) 6120 device.

N5K cannot be parent switch to FI and thus we cannot use FET between them.

Do you have different SFP to try with ?

Padma

[darren-carr]

Hi Padma,

I don't at present, but can secure these. I just need to find an official Cisco document that states this as our Solutions Integrator proposed this. If you look on the Cisco 1620XP 'Data Sheet', taken from the Cisco website, it suggests that it supports the use of the FET-10G, beneath 'Product Specifications'.

http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/data_sheet_c78-524724_ps10276_Products_Data_Sheet.html

I see there are many other options.

I was just looking for a Cisco document or reference that states that you must use SFP'x' to connect the 6120XP to the Nexus 5548.

I am assuming therefore that the preferred choice is the 'Cisco SFP-10G-SR'? We have multimode fibre between our data centre racks.

Thanks,

Darren

[padramas]

Hello Darren,

FET can only be used between N5K or N7K to  N2K   OR  FI 62XX - IOM 220X devices.

You can use either Twinax or other SFPs to  connect Nexus switch and FI 6120 device.

Supported SFPs on N5548

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/hw/installation/guide/overview5500.html#wp1259395

Padma

[darren-carr]

Thanks Padma,

Regarding your comment 'FET can only be used between N5K or N7K to N2K OR FI 62XX - IOM 220X devices.' do you have a reference for this?

Any idea why the data sheet for the 6120XP includes a referece to the FET-10G if it isn't supported? Is this for connecting to a different device?

[darren-carr]

Hi Padma,

I just found this link that confirms what you have outlined above.

http://www.cisco.com/en/US/prod/collateral/modules/ps5455/data_sheet_c78-455693.html

Cisco FET-10G

The Cisco FET-10G Fabric Extender Transceiver support link lengths up to 100m on laser-optimized OM3 or OM4 multimode fiber. It is supported on fabric links only from a Nexus 2000 to a Cisco parent switch. Note this product is not orderable individually. For more information refer to Nexus 2000 datasheet: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps10110/data_sheet_c78-507093.html.

Thanks for your help

Darren

[padramas]

Hello Darren,

The FI docs will be updated to state that FET-10G will only be supported  in second generation hardware ( i.e FI 6200 / 2204 / 2208 IOM / FEX )

Padma