- Cisco Community
- Technology and Support
- Networking
- Switching
- 3560G Output drops
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
3560G Output drops
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-10-2016 06:40 PM - edited 03-08-2019 04:55 AM
Hello everyone,
I'm attempting to understand why a gig interface on our switch is consistently dropping packets outbound towards a server that's requesting a certain set of data off a file server on a 5 minute basis. We have two servers that read the same data off another file server every 5 minutes and one interface is showing dropped frames and the other interface is not.
GigabitEthernet0/38 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0014.6ae8.0aa6 (bia 0014.6ae8.0aa6)
Description: SERVER - S05IA01
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters 1d09h
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 306000 bits/sec, 103 packets/sec
5 minute output rate 491000 bits/sec, 93 packets/sec
17934745 packets input, 6283093363 bytes, 0 no buffer
Received 2044 broadcasts (319 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 319 multicast, 0 pause input
0 input packets with dribble condition detected
18824555 packets output, 8825035553 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
GigabitEthernet0/39 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0014.6ae8.0aa7 (bia 0014.6ae8.0aa7)
Description: SERVER - S05IA02
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters 1d09h
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 37851
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 28000 bits/sec, 36 packets/sec
5 minute output rate 385000 bits/sec, 47 packets/sec
6570859 packets input, 755943945 bytes, 0 no buffer
Received 2002 broadcasts (441 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 441 multicast, 0 pause input
0 input packets with dribble condition detected
8382674 packets output, 5813729117 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
Both switch ports are configured identically
P5_SW_MDF-3560G-48TS#sh run int gig0/38
Building configuration...
Current configuration : 457 bytes
!
interface GigabitEthernet0/38
description SERVER - S05IA01
switchport access vlan 2
switchport mode access
switchport port-security
switchport port-security aging time 2
switchport port-security violation restrict
switchport port-security aging type inactivity
srr-queue bandwidth share 1 25 70 5
srr-queue bandwidth shape 3 0 0 0
priority-queue out
macro description cisco-desktop
spanning-tree portfast
spanning-tree bpduguard enable
end
P5_SW_MDF-3560G-48TS#sh run int gig0/39
Building configuration...
Current configuration : 457 bytes
!
interface GigabitEthernet0/39
description SERVER - S05IA02
switchport access vlan 2
switchport mode access
switchport port-security
switchport port-security aging time 2
switchport port-security violation restrict
switchport port-security aging type inactivity
srr-queue bandwidth share 1 25 70 5
srr-queue bandwidth shape 3 0 0 0
priority-queue out
macro description cisco-desktop
spanning-tree portfast
spanning-tree bpduguard enable
end
P5_SW_MDF-3560G-48TS#show mls qos interface gig0/38 statistics
GigabitEthernet0/38 (All statistics are in packets)
dscp: incoming
-------------------------------
0 - 4 : 17950262 0 0 0 0
5 - 9 : 0 0 0 0 0
10 - 14 : 0 0 0 0 0
15 - 19 : 0 0 0 0 0
20 - 24 : 0 0 0 0 0
25 - 29 : 0 0 0 0 0
30 - 34 : 0 0 0 0 0
35 - 39 : 0 0 0 0 0
40 - 44 : 0 0 0 0 0
45 - 49 : 0 0 0 0 0
50 - 54 : 0 0 0 0 0
55 - 59 : 0 0 0 0 0
60 - 64 : 0 0 0 0
dscp: outgoing
-------------------------------
0 - 4 : 18634184 0 0 0 0
5 - 9 : 0 0 0 0 0
10 - 14 : 0 0 0 0 0
15 - 19 : 0 0 0 0 0
20 - 24 : 0 0 0 0 0
25 - 29 : 0 0 0 0 0
30 - 34 : 0 0 0 0 0
35 - 39 : 0 0 0 0 0
40 - 44 : 0 0 0 0 0
45 - 49 : 0 0 0 0 0
50 - 54 : 0 0 0 0 0
55 - 59 : 0 0 0 0 0
60 - 64 : 0 0 0 0
cos: incoming
-------------------------------
0 - 4 : 17957142 0 0 0 0
5 - 7 : 0 0 0
cos: outgoing
-------------------------------
0 - 4 : 18786263 0 0 0 0
5 - 7 : 0 0 0
output queues enqueued:
queue: threshold1 threshold2 threshold3
-----------------------------------------------
queue 0: 0 0 0
queue 1: 0 7259 62973
queue 2: 0 0 0
queue 3: 0 0 18788143
output queues dropped:
queue: threshold1 threshold2 threshold3
-----------------------------------------------
queue 0: 0 0 0
queue 1: 0 0 0
queue 2: 0 0 0
queue 3: 0 0 0
Policer: Inprofile: 0 OutofProfile: 0
P5_SW_MDF-3560G-48TS#show mls qos interface gig0/39 statistics
GigabitEthernet0/39 (All statistics are in packets)
dscp: incoming
-------------------------------
0 - 4 : 6579229 0 0 0 0
5 - 9 : 0 0 0 0 0
10 - 14 : 0 0 0 0 0
15 - 19 : 0 0 0 0 0
20 - 24 : 0 0 0 0 0
25 - 29 : 0 0 0 0 0
30 - 34 : 0 0 0 0 0
35 - 39 : 0 0 0 0 0
40 - 44 : 0 0 0 0 0
45 - 49 : 0 0 0 0 0
50 - 54 : 0 0 0 0 0
55 - 59 : 0 0 0 0 0
60 - 64 : 0 0 0 0
dscp: outgoing
-------------------------------
0 - 4 : 8190742 0 0 0 0
5 - 9 : 0 0 0 0 0
10 - 14 : 0 0 0 0 0
15 - 19 : 0 0 0 0 0
20 - 24 : 0 0 0 0 0
25 - 29 : 0 0 0 0 0
30 - 34 : 0 0 0 0 0
35 - 39 : 0 0 0 0 0
40 - 44 : 0 0 0 0 0
45 - 49 : 0 0 0 0 0
50 - 54 : 0 0 0 0 0
55 - 59 : 0 0 0 0 0
60 - 64 : 0 0 0 0
cos: incoming
-------------------------------
0 - 4 : 6583571 0 0 0 0
5 - 7 : 0 0 0
cos: outgoing
-------------------------------
0 - 4 : 8337920 0 0 0 0
5 - 7 : 0 0 0
output queues enqueued:
queue: threshold1 threshold2 threshold3
-----------------------------------------------
queue 0: 0 0 0
queue 1: 0 7065 61795
queue 2: 0 0 0
queue 3: 0 0 8339644
output queues dropped:
queue: threshold1 threshold2 threshold3
-----------------------------------------------
queue 0: 0 0 0
queue 1: 0 0 0
queue 2: 0 0 0
queue 3: 0 0 37889
Policer: Inprofile: 0 OutofProfile: 0
As we can see from the output that the drops are happening in queue 3, threshold 3. from the output below
P5_SW_MDF-3560G-48TS#show mls qos maps dscp-output-q
Dscp-outputq-threshold map:
d1 :d2 0 1 2 3 4 5 6 7 8 9
------------------------------------------------------------
0 : 04-03 04-03 04-03 04-03 04-03 04-03 04-03 04-03 04-01 04-02
1 : 04-02 04-02 04-02 04-02 04-02 04-02 03-03 03-03 03-03 03-03
2 : 03-03 03-03 03-03 03-03 02-03 02-03 02-03 02-03 02-03 02-03
3 : 02-03 02-03 03-03 03-03 03-03 03-03 03-03 03-03 03-03 03-03
4 : 01-03 01-03 01-03 01-03 01-03 01-03 01-03 01-03 02-03 02-03
5 : 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03
6 : 02-03 02-03 02-03 02-03
That traffic is dscp 0 thru dscp 7.
what are my next steps to investigating why this switch is dropping that traffic?
- Labels:
-
Other Switching
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-11-2016 05:18 AM
Have you verified the server-side NIC settings; are both of them set up the same ?
From the config above, both servers should be set up to auto-negotiate.
If they ARE the same, then swap ports 38 and 39 and see if the problem remains with the same port, or to the same server.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-14-2016 10:08 AM
we swapped the servers between ports 38 and 39 and the issue followed the server to port 38.
We switched the servers back to their designated ports 38 and 39. We also replaced the patch cables and the output drops are continuing to slowly increase.
Anyone have any other thoughts?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-14-2016 11:53 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Then it sounds like a usage issue.
Your best bet to remediate is probably, as already suggested, some buffer tuning.
However, the 3560G is probably like a 3750G, which provides 2 MB buffer RAM per 24 copper ports and for the uplink ports. Moving your busy server, to another buffer RAM, might help too.
NB: On a 3750, with buffer tuning, I had two SAN hosts go from multiple drops a seconds to a few drops a days.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-11-2016 08:20 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Although configs look alike, if we compare output stats:
18,824,555 packets output, 8,825,035,553 bytes, 0 underruns
8,382,674 packets output, 5,813,729,117 bytes, 0 underruns
We see for the same period, 1d09h, egress volume was more for one port than the other.
Surprisingly, it's the busier port, that doesn't show drops.
However, the less busy port shows more queued packets, for Q3.
Traffic behavior might be different enough that one port enqueues and drops more packets, or perhaps the busier port obtains buffers before the other port, and so the other port runs short.
I.e. buffer tuning might make a difference.
Keep in mind, that 3K switches are not really designed for really busy usage, they're most suitable for user edge devices. Again, though, sometimes buffer tuning can help.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide
