Switching performances questions

[loiccruchade]

Hello everyone,

I recently had some issues with output drops, and i'm planning to re distribute the physical connections per asics cards.

But i was wondering about the switch performance itself.

I use 3750X-48T-E stacks, and i looked on the datasheets.
According to the document, the forwarding rate are up to 101.2 mpps.

Is there a way to know the forwarding rate capabilities per interface ?

Is this calculation correct ? :

101,2mpps / 48 interfaces = 2,11 mpps

And last question, if an interface is experiencing output drops due to bursty traffic, may it be due to forwarding rate capabilities of that interface ?

Thanks for your help !

[Mark Malone]

You will nearly always have issues with 3750s and output drops due to the small buffer size compared to other switches depending whats connected and the traffic been sent , they have always had that problem.

Burst traffic spikes the interface and fills the buffer while traffic is waiting to be sent when its like that traffic that didn't make it into the buffer gets dropped as it has nowhere to go causing output drops

Are you using mls qos or is it just standard out of the box , you can reduce them with qos but you will probably need to tweak the queues to get a drop free interface

did you work out the volume of drops compared to the traffic sent , if its less then 1% wouldn't worry about it as there will always be legitimate drops as well but if its over that you could try and resolve it through qos flow control or other methods

Not sure about that calculation so cant answer that bit

How to identify burst traffic

http://www.cisco.com/c/en/us/support/docs/lan-switching/switched-port-analyzer-span/116260-technote-wireshark-00.html

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

As Mark has already described, interface drops on a 3750 series switch are almost always do to lack of egress buffers, which as he also notes, is often made worst if QoS is enabled with its default settings.

Basically, a 3750X is wirespeed capable on its edge ports.

The 3750 series provides 2 MB RAM buffers per 24 copper edge port and also 2 MB RAM buffers for its uplink ports.

I've found you can often dramatically reduce 3750 interface drops, be enabling QoS and adjusting buffer settings to mimic more of a common pool buffer architecture rather that reserving buffers per port.

[loiccruchade]

Hi guys !

Thanks again for those informations and your help !

I dont know if you remember, but we used to talk a lot about those drops on this topic :

https://supportforums.cisco.com/discussion/12711751/3750-stacks-outdiscards-drops

I was asking those questions thinking about our architecture desing and possible évolutions.

Thanks Mark for the link, i'm gonna take a look about this.

Thanks you too Joseph about the QoS. It's something i'm really not familiar, and i have to do somme research too, for creating a "common pool " as you said. It's still a little complicated for me to understand properly the mls QoS, but i'll give it a try !

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Ah, with the link - I remember.

If you've been using the 3200 3200 1 3200 settings, you're pretty much running a config that's a common pool.

The reason I mentioned, in the prior thread, to make change incrementally, it's possible one congested port will starve other ports (which is why Cisco uses, by default, a reserve buffer model on the 3750s but reservations also tie up buffers, that aren't being used from the ports that need them - it's a balancing act to find the settings that generally work best for you).

[loiccruchade]

Last configuration actually used is this one :

mls qos queue-set output 1 threshold 1 3200 3200 50 3200
mls qos queue-set output 1 threshold 2 3200 3200 50 3200
mls qos queue-set output 1 threshold 3 3200 3200 50 3200
mls qos queue-set output 1 threshold 4 3200 3200 50 3200
mls qos queue-set output 1 buffers 0 50 0 50

Do you have any advice to reach this "common pool architecture" ?

I'm still doing some research on this topic, i'll do some tests and give a feedback.

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Common pool config would be:

mls qos queue-set output 1 threshold 1 3200 3200 1 3200
mls qos queue-set output 1 threshold 2 3200 3200 1 3200
mls qos queue-set output 1 threshold 3 3200 3200 1 3200
mls qos queue-set output 1 threshold 4 3200 3200 1 3200

What your config is doing, is reserving half the buffer resources for interfaces.  The above config, minimizes buffers reserved for interfaces.  Instead, most will now be in the common (sharable) buffer pool.

BTW, I believe the 3750 will accept either of the above in their config, even if QoS isn't actually enabled.  For the above to work, I believe you need to enable QoS.

If QoS hasn't been enabled, and you enable it, also keep in mind, by default, a 3750 will erase the frame's/packet's QoS tags.  If you don't want that to happen, you need to set the ports to trust.

If you don't intend to treat different frame/packet QoS tags differently, you could map all your tags to the same queue and set the buffers parameter to allocate it all to that one queue.

[loiccruchade]

Ok, i'm beginning to understand a little bit more.

I still have two question about my config.

First one: In my config, i have this :

mls qos queue-set output 1 buffers 0 50 0 50

If i'm not wrong, it means that two queue are currently disable. i'm thinking about using this instead, to reach something like a common pool :

mls qos queue-set output 1 buffers 25 25 25 25

What  do you think about this ?

The second question:

I saw that for egress queue, and since i'm having output drops, i saw we can enable priority on interface with :

priority-queue out

Is it required on those interface to enable QoS ?

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

If i'm not wrong, it means that two queue are currently disable. i'm thinking about using this instead, to reach something like a common pool :

mls qos queue-set output 1 buffers 25 25 25 25

What  do you think about this ?

No, by common pool, I mean interfaces draw buffers from a common pool rather than keeping them tied to an interface.  In the 3750 series, allocations are based on percentages, so using the buffers command adjusts the meaning of the other percentages.

For example:

mls qos queue-set output 1 threshold 1 3200 3200 1 3200

The 3200 sets Q1's set all drop values to 3200%.  This seem way too large, but also remember each interface gets a share of the buffers.  So you might have 3200% of 1/48 or about 2/3 of all buffer space.  If you want an interface's queue to have a higher effective limit, then you need to "skew" the buffer allocations.

The second question:

I saw that for egress queue, and since i'm having output drops, i saw we can enable priority on interface with :

priority-queue out

Is it required on those interface to enable QoS ?

No, but QoS needs to be enabled to for PQ to actually be active.