Cisco LAN with lync poor quality (loss and delay)

[ryancisco01]

Here is the secanrio,

2x Cisco 3750E in a stack (core)

connects to another 2x Cisco 3750E in a stack via 4x 1gig port channel (dist)

and multiple 3560 48 port switches connecting off the dist stack. There are multiple different lans and they are routed on the core switch.

Between two users in different vlans. the Lync calls are intermittent bad. usually one caller canot hear the otherside for a while then they will come back again.

the Lync report shows up to 30% packet loss and upto 4 second round trip time.

I cannot replicate the issue, pinging the hosts I can only see around 0.2% loss max.

The links are well underutilized, the port channel is less than 200Mbps and the 3560 uplink trunks are around 10mbps

CPU's and resources are looking ok on all the devices. QoS is disabled across all the switches

the only thing I see is this on the 3560's output drops, (I cleared these counters 2 hours ago, see there is quite a bit of loss already incrementing) anyway to find out what this is.

FastEthernet0/11 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 127
FastEthernet0/15 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 579
FastEthernet0/16 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 290
FastEthernet0/17 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 418
FastEthernet0/18 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 36538
FastEthernet0/19 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
FastEthernet0/20 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 271
FastEthernet0/31 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 286
FastEthernet0/32 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 351
FastEthernet0/33 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 450
FastEthernet0/34 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1494
FastEthernet0/35 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1746
FastEthernet0/36 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 567
FastEthernet0/37 is down, line protocol is down (notconnect)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
FastEthernet0/38 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 227
FastEthernet0/39 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 736
FastEthernet0/40 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 31662
FastEthernet0/43 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 507
FastEthernet0/44 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 73
FastEthernet0/45 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 78
FastEthernet0/46 is up, line protocol is up (connected)
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 499

[ryancisco01]

yes its definitely disabled:

sh mls qos
QoS is disabled
QoS ip packet dscp rewrite is enabled

You have raised an excellent point In your last statement. My load balancing mechanisms don't even match on the connected switches:

s2# sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-mac

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source MAC address
  IPv4: Source MAC address
  IPv6: Source MAC address

s1#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination IP address
  IPv6: Source XOR Destination IP address

Switch 1 has "port-channel load-balance src-dst-ip" set globally but s2 is let as default.

I will get that changed to match, is that the best algorithm or is there a better option for real time traffic?

I have these options:

  dst-ip       Dst IP Addr
  dst-mac      Dst Mac Addr
  src-dst-ip   Src XOR Dst IP Addr
  src-dst-mac  Src XOR Dst Mac Addr
  src-ip       Src IP Addr
  src-mac      Src Mac Addr

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

The "best" load balancing choice is an "it depends" answer but most of the time src-dst-ip works well.  (BTW, sometimes different directions are fine using different choices.)

Ok, you've confirmed QoS is disabled, so your FE edge ports are likely dropping due to transient bursts.  What you could do is enable QoS and configure to protect your Lync traffic.  You'll still see port drops, hopefully not much higher than what you have now, but the protected Lync traffic shouldn't be dropped or delayed.