PFC Question

[visitor68]

Hi--

How does PFC decide which traffic class to pause when it's time to send out a PAUSE frame back to the sender? I havent read a single white paper or blogpost anywhere that explains that. All that is explained is that "during congestion" (which is seldom ever defined to boot, but I know now what's meant by that thanks to a Cisco white paper on PFC), a PFC-enabled switch port will send a PAUSE frame back to the sender based on the CoS value of the traffic.

What is never explained is how PFC selects the traffic to pause. Is it that the highest CoS values are paused first? Is it the lowest CoS value thats paused first? Is it the traffic class that is utilizing the most BW during a defined period of congestion that's paused first?

I hope someone knows.

Thanks

[visitor68]

I have to say I am thinkful that I have had so many smart people at Cisco answer my questions. I know I ask a lot of them. Thats what you have to do when you dont have the equipment in front of you to play with.

I am also disappointed that no one at Cisco has been able to answer these questions above. It really takes someone who has actually configured PFC and ETS in a working environment or a lab to answer them,

[habadr]

Hi, Marhaba  Ex-Engineer,

Sorry for late response but I was very busy in the last few days. 

Please find my answers

1- There are two topics we are talking about here the PFC (no-drop in the configuration) and ETS (Bandwidth management).

For PFC be default we allocate hardware ingress buffer of around 76KB for FCOE (COS3) traffic.

Please refer the following link

http://www.cisco.com/en/US/partner/docs/switches/datacenter/nexus5000/sw/qos/503_n1_1/cisco_nexus_5000_qos_config_gd_503_chapter3.html#task_81ABFBE86A57475DA65966D5C9BC24A1

On a Cisco Nexus 5020 switch, you can configure a maximum buffer size of 143680 bytes

On a Cisco Nexus 5548 switch, you can configure a maximum buffer size of 152000 bytes

If you configure other NO-drop queues you will be able to allocate Hardware ingress buffer it will be another 76 KB. Default queue will get all remaining buffer.

This entire configuration is in system QoS

However for Bandwidth management (ETS) by default Ingress and Egress queues will be split 50/50 between FCOE and default class

"When you configure an input queuing policy on an interface or EtherChannel, the switch sends the configuration data to the adapter using the DCBX protocol"

http://www.cisco.com/en/US/partner/docs/switches/datacenter/nexus5000/sw/qos/503_n1_1/cisco_nexus_5000_qos_config_gd_503_chapter3.html#task_1137670523875610861

Please note if less than four Ethernet classes are defined, up to two of these classes can be configured as no-drop classes. If more than three Ethernet classes are defined, only one of these classes can be configured as a no-drop class. The default drop class is counted as an Ethernet class.

Bandwidth management is flexible you can customize per queue however the summation should be 100%.

2- For ETS you should look for Bandwidth management and not PFC. Ingress and Egress Queuing. 

3- With Bandwidth management by default you will have 50/50 however it is flexible as mentioned above.

4- That is true it is based on the queue and not the COS value.

thanks for the invitation however unfortunately I'm not in NY area

Thanks

[mfrase]

The ULP of FC is SCSI. SCSI is a complete acknowledged protocol , end to end. Every SCSI command must be ack'ed. See T10.org

[visitor68]

Thanks, mfrase. Interesting. I forgot about that.

So, can you elaborate further? How does that impact our discussion? If SCSI has command acks/responses, what does the initiator do when it doesnt receive the ack/response? Perhaps it resends the command over again, but FC, which is the transport mechanism for SCSI, does not have a system wherein it will throttle back its rate of traffic, like TCP does. Since FC is lossless, SCSI assumes --  I imagine -- that the end node is having a problem and simply resends the command. Thats not the same as TCP.

I imagine this is what happens, otherwise there wouldnt be a need for PFC. SCSI would simply inform FC that a loss has occurred and FC would throttle back.

Thoughts?