cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1597
Views
5
Helpful
4
Replies

Lots of TxPause after QoS Change (Nexus 5596UP)

jordanperks
Level 1
Level 1

We have a Dell Equalogic iSCSI SAN. We have been having latency issues with it and Dell gave us their best practices for Nexus 5k config to implement. Prior to the change we had MTU Jumbo settings set system wide and pause no-drop was not configured. The only traffic on these 5k switches is iSCSI traffic. There is nothing else so I did not feel the need to use any QoS settings of any kind. We had flow control send and receive turned on on each interface as well. Below is the Dell best practices config I implemented. Ever since implemented I have been seeing millions and billions of TxPause frames counting up on many interfaces. Does anyone have any ideas?

##2.10.1 Configure CoS for iSCSI

interface ethernet 1/1-40

untagged cos 4

##2.10.2 Define a QoS map for iSCSI

class-map type qos class-iscsi

match cos 4

exit

##2.10.3 Define a QoS policy map for iSCSI

policy-map type qos policy-qos

class type qos class-iscsi

set qos-group 3

exit

exit

##2.10.4 Define a Network QoS class map

class-map type network-qos class-iscsi

match qos-group 3

exit

##2.10.5 Define a no-drop policy map and enable jumbo frames

policy-map type network-qos policy-nq

class type network-qos class-iscsi

mtu 9216

pause no-drop

exit

exit

##2.10.6 Define a queuing class-map

class-map type queuing class-iscsi

match qos-group 3

##2.10.7 Define a queuing policy-map

policy-map type queuing policy-queuing

class type queuing class-default

bandwidth percent 5

class type queuing class-fcoe

bandwidth percent 0

class type queuing class-iscsi

bandwidth percent 95

exit

exit

##2.11 Apply the Nexus policies

system qos

service-policy type qos input policy-qos

service-policy type queuing output policy-queuing

service-policy type network-qos policy-nq

exit

exit

interface ethernet 1/1-40

no lldp receive

no lldp transmit

4 Replies 4

jordanperks
Level 1
Level 1

After some research I have answered my own question. By enabling pause  no-drop it has sliced my available buffer from 470KB to 90KB. It now  sends TX pause frames out at 37KB and Resume frames at 17KB. It will not  use available buffer space in excess of 90.24KB It appears I will need  to use a command similar to "pause no-drop buffer-size 442000  pause-threshold 300560 resume-threshold 243100”. My problem now is, what  to set all those values to. I wonder what percentage of buffer usage I  should have it send pause/resume frames.

DId you get anywhere?  I am in the exact same spot but the 5500 series documentation indicates max buffer size on nodrop queue to be 150000 which is far less than the 470000 max.

On a Cisco Nexus 5500 Series device, you can configure a maximum buffer size of 152000 bytes.

  • pause-threshold—Specifies the buffer limit at which the port pauses the peer.

  • xoff-size—Buffer limit for pausing, in bytes. Valid values are 0 to 490880.
    Note   

    On a Cisco Nexus 5020 switch, you can configure a maximum pause-threshold value of 58860 bytes.

    On a Cisco Nexus 5500 Series device, you can configure a maximum pause-threshold value of 103360 bytes.

  • resume-threshold—Specifies the buffer limit at which the port resumes the peer.

  • xon-size—Buffer limit at which to resume, in bytes. Valid values are 0 to 490880.
    Note   

    On a Cisco Nexus 5020 switch, you can configure a maximum resume-threshold value of 38400 bytes.

I'm having the same issue.  Anyone ever figure this out?

on our 5596, we worked with Cisco TAC to configure buffers to nearly 400k.  The manual is wrong or ambigous and TAC asked why we had them set so low while troubleshooting the issue.  The PFC on the port-channels was set to off and that seemed to resolve our issues.  PAuse frames are now almost entirely absent.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: