Cisco 3560G Slow Performance

[vb543]

I've been having some performance issues with one of my Cisco switches for a while now. I originally posted on Reddit seeking some help. Now that the issue continues, I wanted to see if anyone had some ideas over here.

I'm using a Cisco 3560G to handle some basic vlans/routing for a rack of servers. The switch has a gigabit uplink and I've recently only been getting a few MBps from external tests to various local servers as well as internally between systems across vlans.

My original configuration had a lot of secondary addresses on various vlans, after discovering that this wasn't best practice - I've offloaded a lot of the addressing for each vlan to a small ER-X. Unfortunately, that hasn't fully corrected the issue.

Here's my current config: https://pastebin.com/raw/dYaZ6q22

Any thoughts/advice would be appreciated!

[Joseph W. Doherty]

I see a lot of CPU for IPv6. You're using an IPv4/IPv6 SDM template?

A couple of interfaces show quite a few drops, but insufficient data to see percentage.

[paul driver]

Hello

CPU utilization for five seconds: 38%/13 - relativity low 5 minute % but above normal interrupt % 

Can you post the following please-

sh platform tcam utilization | in  direct <-- check to see if this are maxed out
sh control cpu-interface | in sw <-- check to see if these a increasing very quickly
sh platform ip unicast stat | in  CPUAdj   <--- do this multiple times to see if this increases

res
Paul

[vb543]

As requested:

switch1#sh platform tcam utilization | in  direct
 IPv4 unicast directly-connected routes:       672/5376        519/4050
 IPv4 unicast indirectly-connected routes:     144/1152         16/80
 IPv6 unicast directly-connected routes:       672/5376        519/4050
 IPv6 unicast indirectly-connected routes:     128/1024         13/46
switch1#sh control cpu-interface | in sw
sw forwarding     3281437491 0          0          0          0
sw forwarding     0            0
switch1#sh control cpu-interface | in sw
sw forwarding     3281437619 0          0          0          0
sw forwarding     0            0
switch1#sh control cpu-interface | in sw
sw forwarding     3281437931 0          0          0          0
sw forwarding     0            0
switch1#sh platform ip unicast stat | in  CPUAdj
        EncapFail:0 CPUAdj:2016365044 Null:21786364 Drop:1733
        EncapFail:0 CPUAdj:2016365044 Null:21786364 Drop:1733
switch1#sh platform ip unicast stat | in  CPUAdj
        EncapFail:0 CPUAdj:2016365117 Null:21786377 Drop:1733
        EncapFail:0 CPUAdj:2016365117 Null:21786377 Drop:1733
switch1#sh platform ip unicast stat | in  CPUAdj
        EncapFail:0 CPUAdj:2016365175 Null:21786391 Drop:1733
        EncapFail:0 CPUAdj:2016365175 Null:21786391 Drop:1733

[Joseph W. Doherty]

Could you define what kind of performance issues you believe you have?

If it concerns transfer rates, the 3560/3750 series are a bit infamous for having relatively small RAM for buffers that can often lead to premature egress frame/packet drops. This can be especially a problem if QoS is globally also especially using the default QoS settings.

[vb543]

It is indeed transfer rates. Any way I could check if this known issue is what's causing my problems?

I've run iperf tests between several systems. Some systems on the same vlan aren't able to saturate the gigabit link while one of those same systems talking to a different server can.

IPerf Results:

S3 <-> LG    OKAY
S3 <-> VZ2    236Mbps
S3 <-> KVM    291Mbps
S3 <-> S4     336Mbps    
S3 <-> WSI    61.8Mbps [External via port 1 uplink]
S4 <-> LG     297Mbps

S3 and S4 share a vlan, as do VZ2 and KVM

[Joseph W. Doherty]

Interface stats show any drops?

What IOS version are you running?

[vb543]

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 52    WS-C3560G-48TS     12.2(55)SE11          C3560-IPBASEK9-M

[Joseph W. Doherty]

Ok, that's a very solid version, although if your doing DHCP helper, you'll want to upgrade to SE12.