8-Gbps Advanced Fibre Channel Switching Module

[Jeremy Waldrop]

Does anyone know if the 9506 supports the 8Gbps line rate to every port advertised with this new card?

We just had a customer purchase one for their 9506 that is running NX-OS 5.2.1. The card works fine but they aren't able to get 8Gpbs dedicated per port.

Did some research and found that the config guide states this in a foot note - "This bandwidth is available with the Fabric 3 module (DS-13SLT-FAB3) in the MDS 9513 switch"

Is this correct?

The data sheet states this - "With Arbitrated Local Switching enabled, this module supports 48-ports of line-rate 8-Gbps and is perfect for deploying dense virtual machine clusters with locally mapped storage"

The 9506 data sheet states that Arbitrated Local Switching is one of the features. Is this a config option or am I missing something?

http://www.cisco.com/en/US/prod/collateral/ps4159/ps6409/ps5990/ps5395/product_data_sheet09186a00801ce948_ps5990_Products_Data_Sheet.html

[maminhas]

Not sure which perticular 8gb LineCard is used on 9506

48-port 8-Gbps Advanced Fibre Channel module (DS-X9248-256K9) or to the Cisco MDS 9000 32-port 8-Gbps Advanced Fibre Channel module (DS-X9232-256K9)  ?

Please check this link

http://www.cisco.com/en/US/docs/switches/datacenter/mds9000/hw/9500series/installation/5.2/installmigratefab3.html#wp1435713

which states

"to use the full bandwidth capability of the 8-Gbps Advanced Fibre Channel modules.fab 3 card is required.

and DS-13SLT-FAB3 card is only available/applicable  for 9513 chassis (not for 9506/9509)

[Filiph Westman]

I assume that you are using the 48-port 8-Gbps Advanced FC module (DS-X9248-256K9) as you referred to the datasheet for that module and that you already reloaded the MDS switch earlier to be in 8G mode:

Switch1-MDS9509# sh hardware fabric-mode

Fabric mode supports 8G FC modules.

The MDS 9506 support up to 96-Gbps backplane bandwidth (MDS 9513 with FAB3 will give you 256-Gbps). However with the Arbitrated Local Switching enabled, this module supports 48-ports of line-rate 8-Gbps (locally).

Local Switching is disabled by default and can be enabled on a per-line card basis. It is recommended that all the ports on the module be shut before moving to and out of local switching.

In addition, if a dedicated port is configured when Local Switching is already enabled, the switching bandwidth of the entire module will drop to the bandwidth supported by the backplane. If the port is reconfigured in shared mode, the switching bandwidth will be restored.

The arbitrated local switching feature is not supported when the 10-Gbps operating mode is enabled.

Hope this helps

[Jeremy Waldrop]

That is promising new, thanks. How do I enable local switching?

[Kris Vandecruys]

Hi Jeremy,

you can enable this on a module using the following commands:

# conf term

(config)# local-switching module [forced]

where is the slot in which your DS-X9248-256K9 is inserted.  you can optionally include force, but in normal circumstances this isn't necessary.  It is recommended that all the ports on this module be shut first as moving to and out of local switching mode may be disruptive.

Be aware that local switching only makes sense on a 48 port linecard, as the 32 port version is not oversubscribed on the backend at 8Gb and the overhead of passing the crossbar is negligable.

Also, Arbitrated Local Switching doesn't work on 10GbE, only on FC.

Best Regards,

Kris

[samet.aksoy]

Hello Kris,

Is there a disadvantage of enabling the local-switching?(e.g can we able to make zones between modules when this feature is enabled?) If there is not, why this feature is disabled by default?

Thanks

[Kris Vandecruys]

Hi Samet,

As Filiph mentioned above, Local switching is disabled by default and can be enabled on a per-line-card basis. However, enabling and disabling local switching may be disruptive. Therefore, you should shut down all ports on the module before moving to and from local switching.

In addition, dedicated mode ports are not supported when local switching is enabled. All ports must be in shared mode when enabling local switching.

In the real world, this means you cannot configure ISLs on a ALS enabled linecard.

[Michael Brown]

When you enable local switching, frames between ports on the same line card do not go through the chassis crossbar so the latency on those frames is somewhat lower than if local switching was not enabled.

Without local switching, there is a very deterministic latency, no matter where devices are attached.  Frames from port 1 on line card 1 to port 10 on line card 1 are switched at the with latency as frames between port 1 on line card 1, and port 20 on line card 4 (examples).

When you enable local swtiching (on module 1 for instance) frames between port 1 and 10 on the line card 1 are not swtiched thorugh the remote crossbar.

The effect with local swtiching enabled is that a device on port 1 line card 1, that is communicating with devices both on port 10 line card 1, and another device on a different line card, that device will see different latencies between the 2 devices.  Disable local swtiching, and the latencies will be the same.

That's why it's not on by default.

  Hope this helps,

MikeB

[samet.aksoy]

Thank you all guys. I have also found below whitepaper. It explains the feature in details.

http://www.cisco.com/en/US/prod/collateral/modules/ps5991/white_paper_c11-545453.pdf