Re: QoS causing slow data transfers

mmillington_nal · ‎08-23-2012

I am a new network admin at my company, and I am investigating a problem that is on a network designed by someone no longer here. So, I can't ask why things were set up the way they were. Anyway, here's the issue:

We have 50Mbps WAN connections between locations. The WAN devices are 3750s, but the port plugged into the providers network is a layer 3 port. Config is below for one of the ports:

interface GigabitEthernet1/0/1

description

no switchport

ip address x.x.x.x x.x.x.x

load-interval 30

srr-queue bandwidth share 10 10 60 20

srr-queue bandwidth shape 10 0 0 0

queue-set 2

mls qos trust dscp

So, these are Gig ports, but our service provider limits to 50Mbps. When we do a data transfer, we can't exceed about 20Mbps total on the circuit. The data transfers at maximum 15-16Mbps.

I'm not familiar with QoS. I've tried to read more about SRR, but don't understand it well enough to know. Are the QoS settings creating this restriction? Our provider swears that the bandwidth is allocated and its not being choked on their end. I have not taken off the QoS config to test it that way because I'm not 100% sure if its needed and serving some purpose. I don't want to break something in order to fix this.

Is the QoS causing this? Do I need to put a bandwidth statement on the port? Should we be using routers and LLQ? Any help would be greatly appreciated.

Joseph W. Doherty · ‎08-23-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

What's the link physical speed gig or 100 Mbps?

If gig find out if the vendor will support a connection at 100.

If 100, try adding the interface command ssr-queue bandwidth limit 50.

The foregoing command will "shape" the interface at 50% of physical bandwidth (to keep within your provider's 50 Mbps).

There's much more that might be required to optimize your QoS behavior, but there's insufficient information to suggest anything else.

mmillington_nal · ‎08-24-2012

Joseph,

Thank you for responding. I apologize for not giving enough information. I was trying to leave it vague so as to not ask the wrong questions, but may have been too vague.

I guess my first question is this:

What is the effect on the interface from the srr-queue commands? The way I understand it, queue 1 (which contains our voice traffic by virtue of DSCP marking) will get 10% of the bandwidth. Since the physical interface is 1Gig, that would be 100M, and the remaining 3 queues will split the rest of the bandwidth at 300M apiece. So, what will the result be since the connection is not truly 1Gig, but rather once it enters our provider's network will be rate-limited by software to 50M? W

What will be the effect on a data transfer that tries to fill up a 1Gig connection with the shaping that we have and the rate limiting that will happen on the provider's side? Could that combination produce the results we are seeing that a single data transfer could only effectively use 20-30% of the connection? Is that because of the shaping policy that is only allocating that much to a single queue?

Secondly, would entering the bandwidth command as you suggest help with this?

Thank you.

Joseph W. Doherty · ‎08-24-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

As configured, Q1 will be limited to 10% of port speed. The other queues will use remaining bandwidth (which could be 100% to 90%) in proportion to their share weights, 10 60 20. Any queue can use more bandwidth then its share if it's available.

Only your SP can describe what happens when you exceed 50 Mbps, but typically they will discard overrate traffic (which indeed can slow effective throughput to even less than 50 Mbps).

The command(s) I suggested don't help, per se, as they should cap your transmission rate to avoid exceeding the SP's provided rate. If this causes congestion (assuming you're indeed exceeding 50 Mbps), you then have the option to manage the congestion.

chris.monk · ‎08-24-2012

Could you post the complete output of "show mls qos interface GigabitEthernet1/0/1 statisitics". This will tell us if you are in fact dropping anything outbound toward the provider.

As Joseph suggested, I would definately limit the bandwidth of the port to your actual purchased bandwidth. If you are connecting to the SP equipment at 100M then limit to 50%, if connecting at 1G then limit to 5%. Giving SRR inacurate information would definately cause problems in times of congestion.

mmillington_nal · ‎08-27-2012

Thanks Chris. So, does the fact that there are no drops tell us anything? Could it be that its just being dropped at the SP level, so what's happening is just retransmissions causing the overall throughput to be limited?

With regard to the bandwidth limit, the values are only between 10 and 90%. I have a Gig port, but only 50M service, so I can't limit to only 5%. Would limiting it to 10%, or 100M, accomplish anything or would it essentially be the same as what it is now?

Thanks,

Matt

Here is the output:

GigabitEthernet1/0/1 (All statistics are in packets)

dscp: incoming

-------------------------------

0 - 4 : 348089929 624 4738554 63 29594

5 - 9 : 16 38 3103 673595 39

10 - 14 : 6661 0 6877 404 63

15 - 19 : 1 10236 25 1046 0

20 - 24 : 194 0 6590 75 9962

25 - 29 : 9 7106 0 5361 0

30 - 34 : 491 0 128440 0 403

35 - 39 : 0 10 13 7546 1

40 - 44 : 164 0 180 27 43

45 - 49 : 5 55655509 8 18044034 0

50 - 54 : 0 0 0 0 0

55 - 59 : 0 21 0 0 0

60 - 64 : 1 0 0 68

dscp: outgoing

-------------------------------

0 - 4 : 3668241245 0 0 0 0

5 - 9 : 0 0 0 0 0

10 - 14 : 0 0 0 0 0

15 - 19 : 0 0 0 0 0

20 - 24 : 0 0 0 0 0

25 - 29 : 0 0 0 0 0

30 - 34 : 0 0 0 0 0

35 - 39 : 0 0 0 0 0

40 - 44 : 0 0 0 0 0

45 - 49 : 0 334108 0 16601063 0

50 - 54 : 0 0 0 0 0

55 - 59 : 0 0 0 0 0

60 - 64 : 0 0 0 0

cos: incoming

-------------------------------

0 - 4 : 428065617 0 0 0 0

5 - 7 : 0 0 0

cos: outgoing

-------------------------------

0 - 4 : 3668257832 0 0 0 0

5 - 7 : 334108 16601063 3173

output queues enqueued:

queue: threshold1 threshold2 threshold3

-----------------------------------------------

queue 0: 2 0 33

queue 1: 0 1826720 19260363

queue 2: 0 0 0

queue 3: 0 0 3667274884

output queues dropped:

queue: threshold1 threshold2 threshold3

-----------------------------------------------

queue 0: 0 0 0

queue 1: 0 0 0

queue 2: 0 0 0

queue 3: 0 0 0

Policer: Inprofile: 0 OutofProfile: 0

chris.monk · ‎08-27-2012

What platform is transitting this data? Is it connected directly to this switch?

Sent from Cisco Technical Support Android App

mmillington_nal · ‎08-27-2012

Sorry, are you asking what is doing the rate-limiting on the SP side, or what is the hardware device we plug into? We plug into a Cisco ME3400E on the provider side. I don't know what they're using to rate-limit, but they said at one point that its software-based. I'm not sure if they're doing it in the ME3400 or not.

chris.monk · ‎08-27-2012

I apologize, what software/server are you using to send the data to your remote site? Is this server/appliance connected directly to your switch, or is it daisey chained off another switch.

Server~~>3750~~>3400-->SP?

Sent from Cisco Technical Support Android App

mmillington_nal · ‎08-27-2012

Ahh. I'm using FTP to test.

And your point is very valid. The devices are not plugged in to the WAN switch on either end. We are plugged in at both locations to a 3750 stack hanging off the WAN 3750s. 1 Gig uplinks between the two.

Sent from Cisco Technical Support iPhone App

chris.monk · ‎08-27-2012

Could you check for drops on the uplink between the stack and your edge at both sites? Also, could you hang a PC off the wan edge and FTP to it to take the SP out of the equation?

Sent from Cisco Technical Support Android App