cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
8222
Views
0
Helpful
3
Replies

Flow Control Netap -> Nexus 5548

mmedwid
Level 3
Level 3

Our netapp filer 3140 was getting swamped with requests from our Cisco UCS during storage vmotion of snap deletions.

Still troubleshooting.  But one thing found along the way the netapp had been sending pause frame to the 10Gbps interfaces

on the Nexus and flow control had not been enabled as it turns out this is not done via autonegotiation.  Now that flow control

is in - I can see the RxPause counters incrementing (see below).  What action does the Nexus switch take in response to

receiuving these pause frames?  Flow control is still not enabled at the UCS.  (may require another change window.)  I

am wondering in the meantime if just enabling the flow control at the Nexus might solve the overwhelm situation with

the netapp.  Thank you for any enlightenment.

hrk-c5548-1# sho int ethernet 1/3 flowcontrol

--------------------------------------------------------------------------------

Port         Send FlowControl  Receive FlowControl  RxPause   TxPause 

             admin    oper     admin    oper

--------------------------------------------------------------------------------

Eth1/3       off      off      on       on          1379              0               

hrk-c5548-1# sho int ethernet 1/4 flowcontrol

--------------------------------------------------------------------------------

Port         Send FlowControl  Receive FlowControl  RxPause   TxPause 

             admin    oper     admin    oper

--------------------------------------------------------------------------------

Eth1/4       off      off      on       on          416               0               

hrk-c5548-1#

3 Replies 3

alf
Level 1
Level 1

Hi,

I´ve got a similar discussion. What I found in the NetApp NAS Best Practices:

For network switches it is recommended to set the switch ports connecting to ESX hosts and FAS storage

arrays to either ’Desired’, or if this mode isn’t available, set these ports to ‘send off’ and ‘receive on.’ Note

the switch ports are configured with the opposite settings of the end points, the ESX & FAS systems.

Here are two references:

http://media.netapp.com/documents/tr-3428.pdf, page 39

http://www.seanxwang.com/2011/03/cisco-flow-control-with-netapp-nas.html

So I think you should work with flow control not only on the Switch side but also on the NAS and Server side.

Hope that helps

Amanda Hunt
Cisco Employee
Cisco Employee

Hi Michael,

I'm not sure if your question was answered, but I thought I'd explain a little more about how flowcontrol works with the Nexus so you know where the best practices are coming from.

Flowcontrol Receive (Rx) is when another device sends US pause frames. There are two protocols we support to do this: 802.3x and Priority Flow Control. Flowcontrol Transmit (Tx) is when WE send pause frames out.

A pause frame is a communication sent by a switch to tell the link on the other end to slow down sending traffic, because we can't handle it, or somewhere downstream can't handle it. Its always a good thing to have, because without it we or the other switch will drop frames and in storage environments the retransmits that result can be disastrous.

So on to your question,

What action does the Nexus switch take in response to receiving these pause frames?


The action depends on what we have configured.

If we don't have Flow Control Rx turned on, we will drop the packet and take no action, after ticking the counter to show we received it.
If we do have Flow Control Rx turned on, we will receive pause frames and send traffic to your NIC slower
If we have Flow Control Tx turned on, we will also have the ability to send pause frames to you and inform you if you are sending data too fast for the target to receieve (assuming the target communicates to us that it needs the traffic to be slowed, and assuming we have flowcontrol enabled on all interfaces in the path to the target).

Hi Amanda,

If flow control configuration is not the same (one side on, the other side off) on a link between 2 Nexus devices (N5K and N7K), does it hurt anything? Thanks

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: