03-21-2013 07:58 AM
We have a Dell Equalogic iSCSI SAN. We have been having latency issues with it and Dell gave us their best practices for Nexus 5k config to implement. Prior to the change we had MTU Jumbo settings set system wide and pause no-drop was not configured. The only traffic on these 5k switches is iSCSI traffic. There is nothing else so I did not feel the need to use any QoS settings of any kind. We had flow control send and receive turned on on each interface as well. Below is the Dell best practices config I implemented. Ever since implemented I have been seeing millions and billions of TxPause frames counting up on many interfaces. Does anyone have any ideas?
##2.10.1 Configure CoS for iSCSI
interface ethernet 1/1-40
untagged cos 4
##2.10.2 Define a QoS map for iSCSI
class-map type qos class-iscsi
match cos 4
exit
##2.10.3 Define a QoS policy map for iSCSI
policy-map type qos policy-qos
class type qos class-iscsi
set qos-group 3
exit
exit
##2.10.4 Define a Network QoS class map
class-map type network-qos class-iscsi
match qos-group 3
exit
##2.10.5 Define a no-drop policy map and enable jumbo frames
policy-map type network-qos policy-nq
class type network-qos class-iscsi
mtu 9216
pause no-drop
exit
exit
##2.10.6 Define a queuing class-map
class-map type queuing class-iscsi
match qos-group 3
##2.10.7 Define a queuing policy-map
policy-map type queuing policy-queuing
class type queuing class-default
bandwidth percent 5
class type queuing class-fcoe
bandwidth percent 0
class type queuing class-iscsi
bandwidth percent 95
exit
exit
##2.11 Apply the Nexus policies
system qos
service-policy type qos input policy-qos
service-policy type queuing output policy-queuing
service-policy type network-qos policy-nq
exit
exit
interface ethernet 1/1-40
no lldp receive
no lldp transmit
03-22-2013 09:41 AM
After some research I have answered my own question. By enabling pause no-drop it has sliced my available buffer from 470KB to 90KB. It now sends TX pause frames out at 37KB and Resume frames at 17KB. It will not use available buffer space in excess of 90.24KB It appears I will need to use a command similar to "pause no-drop buffer-size 442000 pause-threshold 300560 resume-threshold 243100”. My problem now is, what to set all those values to. I wonder what percentage of buffer usage I should have it send pause/resume frames.
03-03-2017 01:04 PM
DId you get anywhere? I am in the exact same spot but the 5500 series documentation indicates max buffer size on nodrop queue to be 150000 which is far less than the 470000 max.
On a Cisco Nexus 5500 Series device, you can configure a maximum buffer size of 152000 bytes.
pause-threshold—Specifies the buffer limit at which the port pauses the peer.
Note |
On a Cisco Nexus 5020 switch, you can configure a maximum pause-threshold value of 58860 bytes. On a Cisco Nexus 5500 Series device, you can configure a maximum pause-threshold value of 103360 bytes. |
resume-threshold—Specifies the buffer limit at which the port resumes the peer.
Note |
On a Cisco Nexus 5020 switch, you can configure a maximum resume-threshold value of 38400 bytes. |
08-08-2017 10:19 AM
I'm having the same issue. Anyone ever figure this out?
08-08-2017 10:50 AM
on our 5596, we worked with Cisco TAC to configure buffers to nearly 400k. The manual is wrong or ambigous and TAC asked why we had them set so low while troubleshooting the issue. The PFC on the port-channels was set to off and that seemed to resolve our issues. PAuse frames are now almost entirely absent.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: