08-24-2016 05:21 AM - edited 03-05-2019 04:33 AM
Curently my setup is simple.
Cab 1 = primary 3750G switch
Cab 2 = stacked 3750G switch
If I get a quick replacement 3750 for the stacked switch in cab 6, would that one be a simple swap (just re-cabling) or require re-configuration? It’s stacked, keep in mind. I create, edit VLANs via one single login, the primary so my guess it would be a simple replacement swap?
Reason for my replacement is the 3750 in cab 2 (secondary stacked) seems to have major packet loss and latency. It seems the primary in cab 1 is all good across the board. Also, how do I check if there's some sort of flow control / QoS enabled that might be causing this?
08-24-2016 06:06 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
A 3750 stack operates as if it were a single unit with the stack member "appearing" like a chassis line cards. Swapping a single unit, if the replacement isn't configured, should allow the replacement to "pick up" the prior unit's config. (This, though, may require the replacement to be the same model 3750 as the one being replaced. If not, you would need to configure its ports after it joins the stack.)
As to packet loss and latency, if a egress port is dealing with traffic bursts, the 3750 series is a bit infamous for dropping packets, especially if default QoS is active, due to the 3750's small buffer capacity. For transient bursts, sometimes tuning a 3750's config can dramatically decrease egress drops.
08-24-2016 06:10 AM
Thanks for the info. How do I disable QoS, or check if it's even there?
I can’t determine which model is which switch though, and curious what the difference is between WS-C3750G-24T-S and WS-C3750G-24TS-S
Cisco3750#show inventory
NAME: "1", DESCR: "WS-C3750G-24T"
PID: WS-C3750G-24T-S , VID: V05 , SN:
NAME: "2", DESCR: "WS-C3750G-24TS"
PID: WS-C3750G-24TS-S , VID: G0 , SN:
NAME: "GigabitEthernet2/0/28", DESCR: "1000BaseSX SFP"
PID: Unspecified , VID: , SN:
Switch# Role Mac Address Priority Version State
----------------------------------------------------------
*1 Master 001f.6c8a.9380 15 0 Ready
2 Member 0012.01db.f600 14 0 Ready
08-24-2016 09:48 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
How to disable 3750 QoS? In config mode, no mls qos.
How to check if 3750 QoS enabled? Either show mls qos or look for "mls qos" in config.
WS-C3750G-24T-S is a 1 RU
WS-C3750G-24TS-S is a 1.5 RU with 4 SFP ports.
08-24-2016 10:02 AM
Truly appreciate your help.
I'm not sure what the below means.
Cisco3750#show mls qos
QoS is disabled
QoS ip packet dscp rewrite is enabled
I see it says disabled, but what about the 2nd line, dscp rewrite?
08-24-2016 10:19 AM
I believe that has nothing to due with your issue.
08-25-2016 04:49 AM
Ok here's the strange thing.
1. Several Dell 610 servers are crawling when it comes to speed, simple things like wget or cachefly tests. They're set on GigE / Full Duplex on the switches. NIC drivers updated, rebooted, running Centos 6.8
2. Somehow, I found one particular server on 100Mbit speed, but consistently pushing nearly 40MBps every day. How is that even possible?
My Cisco processor is constantly at 99%. See attached. If I issue a reload, it shoots up rather quickly.
http://i.imgur.com/JLSHlrH.png
08-25-2016 04:34 PM
Check the interface of each server. See if there are any "Output Drops".
08-25-2016 05:37 PM
Could you please explain to me explain how I would go about doing that?
08-25-2016 05:55 PM
Could you please explain to me explain how I would go about doing that?
Perform the command "sh interface <BLAH>" and see if there are any "Total Output Drops".
Total Output Drops are an interface-based counters where the interface could no longer "hold on" to the data and drops the packets. In an ideal environment, Total Output Drops should be a ZERO.
Another thing, Cisco Catalyst range of switches were not designed for server duties. They are notorious for very shallow buffers and start dropping packets when high-speed traffic starts pummeling them.
08-25-2016 07:04 PM
For some reason I don't see any line that says "Total Output Drops"
Example:
Cisco3750#sh interface Gi1/0/7
GigabitEthernet1/0/7 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 001f.6c8a.9387 (bia 001f.6c8a.9387)
Description: Server 111
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 10000 bits/sec, 9 packets/sec
5 minute output rate 8000 bits/sec, 10 packets/sec
5091180 packets input, 1915463255 bytes, 0 no buffer
Received 0 broadcasts (0 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
5243439 packets output, 694967769 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
08-25-2016 07:20 PM
GigabitEthernet1/0/7 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 001f.6c8a.9387 (bia 001f.6c8a.9387)
Description: Server 111
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 10000 bits/sec, 9 packets/sec
5 minute output rate 8000 bits/sec, 10 packets/sec
5091180 packets input, 1915463255 bytes, 0 no buffer
Received 0 broadcasts (0 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
5243439 packets output, 694967769 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
In RED.
08-26-2016 05:38 AM
Thanks. Just ran this on another box, the one that claims is having latency issues.
Cisco3750#sh interface Gi1/0/21
GigabitEthernet1/0/21 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 001f.6c8a. (bia 001f.6c8a.9395)
Description: Server 11
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 23:36:47, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 44000 bits/sec, 86 packets/sec
57706331 packets input, 32444097669 bytes, 0 no buffer
Received 43355 broadcasts (43325 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 43325 multicast, 0 pause input
0 input packets with dribble condition detected
67902039 packets output, 34141503953 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
08-26-2016 07:34 AM
Last input 23:36:47, output 00:00:00, output hang never
Not a latency issue. Server issue.
No response from the server for almost 24 hours???? Only a server issue can do this.
Total Output Drops is ZERO. Look at the significant difference between the Packets Input vs Packets Output. `tis a server issue.
08-26-2016 04:15 AM
Could you post switch's CPU stats when it's running at 99%; both normal stats sorted (top 5 processes should be enough) and CPU history.
3750 CPU utilization shouldn't be that high. Wondering whether you're doing software switching.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide