(7G) Links Port-channel output Drop on Cisco 3560 --- Cisco 4500 Ether-channel

[Dr.X]

Hello Folks ,

We have an L2 trunk between cisco 3560 and cisco 4500.

we have 7 ports bundled in that channel.

we have both sides flow control disabled in/out

real traffic can reach to 5.5 G, but we still see output Drops only on 3560.

At 4500 we don't see any Drops at all in Etherchannel.

Both sides have( ip-src-dst) hashing algorithms for load balancing.

I searched almost all posts in the forum but not able to find something can help me.

i tested PPS and switch limitations, all is normal and under thresholds.

I'm just wondering does ether channel port like 7 G with real traffic 4-5.5 G loaded require different settings?

could be a bug? why is it only on one side of ether channel not on both of them ?!!

###############################

below show commands :

1- Cisco 3560 :

3560G#sh etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator

M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port

Number of channel-groups in use: 5
Number of aggregators: 5

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
45 Po45(SU) - Gi0/6(P) Gi0/7(P) Gi0/24(P)
Gi0/32(P) Gi0/34(P) Gi0/45(P)
Gi0/48(P)

3560G#sh run int port-channel 45
Building configuration...

Current configuration : 124 bytes
!
interface Port-channel45
description 3560--4500 Bundle
switchport trunk encapsulation dot1q
switchport mode trunk
end

3560G#sh int port-channel 45
Port-channel45 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 0023.ea72.ba06 (bia 0023.ea72.ba06)
Description: 3560--4500 Bundle
MTU 1500 bytes, BW 7000000 Kbit, DLY 10 usec,
reliability 255/255, txload 123/255, rxload 196/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, link type is auto, media type is unknown
input flow-control is off, output flow-control is unsupported
Members in this channel: Gi0/6 Gi0/7 Gi0/24 Gi0/32 Gi0/34 Gi0/45 Gi0/48
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 01:45:41
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 71866200
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 5405805000 bits/sec, 673407 packets/sec
5 minute output rate 3398399000 bits/sec, 520771 packets/sec
4072242825 packets input, 4102195296979 bytes, 0 no buffer
Received 842862805 broadcasts (150897 multicasts)
0 runts, 0 giants, 0 throttles
2 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 150897 multicast, 0 pause input
0 input packets with dribble condition detected
3185468252 packets output, 2632031579409 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

3560G#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address

3560G#sh ve
3560G#sh version
Cisco IOS Software, C3560 Software (C3560-IPSERVICESK9-M), Version 12.2(53)SE2, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Wed 21-Apr-10 05:33 by prod_rel_team
Image text-base: 0x01000000, data-base: 0x02E00000

ROM: Bootstrap program is C3560 boot loader
BOOTLDR: C3560 Boot Loader (C3560-HBOOT-M) Version 12.2(44)SE5, RELEASE SOFTWARE (fc1)

3560G uptime is 1 year, 27 weeks, 7 hours, 42 minutes
System returned to ROM by power-on
System image file is "flash:/c3560-ipservicesk9-mz.122-53.SE2.bin"

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3560G-48TS (PowerPC405) processor (revision D0) with 131072K bytes of memory.
Processor board ID FOC1245W1F1
Last reset from power-on
21 Virtual Ethernet interfaces
52 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address : 00:23:EA:72:BA:00
Motherboard assembly number : 73-10214-04
Power supply part number : 341-0107-01
Motherboard serial number : FOC12450UNW
Power supply serial number : AZS124310CD
Model revision number : D0
Motherboard revision number : C0
Model number : WS-C3560G-48TS-S
System serial number : FOC1245W1F1
Top Assembly Part Number : 800-26849-01
Top Assembly Revision Number : C0
Version ID : V03
CLEI Code Number : CNMWY00ARC
Hardware Board Revision Number : 0x09

Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 52 WS-C3560G-48TS 12.2(53)SE2 C3560-IPSERVICESK9-M

Configurations 

2- on 4500 Side :

4500E#sh etherchannel su
4500E#sh etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator

M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port

Number of channel-groups in use: 3
Number of aggregators: 3

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
45 Po45(SU) - Gi1/42(P) Gi1/43(P) Gi1/44(P)
Gi1/45(P) Gi1/46(P) Gi1/47(P)
Gi1/48(P)

4500E#sh int port-channel 45
Port-channel45 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 0007.7d64.0529 (bia 0007.7d64.0529)
Description: 4500---3560 Bundle
MTU 1500 bytes, BW 7000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 197/255, rxload 124/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is N/A
input flow-control is off, output flow-control is unsupported
Members in this channel: Gi1/42 Gi1/43 Gi1/44 Gi1/45 Gi1/46 Gi1/47 Gi1/48
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:01, output never, output hang never
Last clearing of "show interface" counters 02:18:08
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 49122
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 3417084000 bits/sec, 522307 packets/sec
5 minute output rate 5415228000 bits/sec, 674449 packets/sec
4139848421 packets input, 3419495488468 bytes, 0 no buffer
Received 53945 broadcasts (25299 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
5286307877 packets output, 5330349202928 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
4500E#sh int port-channel 45
Port-channel45 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 0007.7d64.0529 (bia 0007.7d64.0529)
Description: 4500---3560 Bundle
MTU 1500 bytes, BW 7000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 197/255, rxload 124/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is N/A
input flow-control is off, output flow-control is unsupported
Members in this channel: Gi1/42 Gi1/43 Gi1/44 Gi1/45 Gi1/46 Gi1/47 Gi1/48
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:01, output never, output hang never
Last clearing of "show interface" counters 02:18:14
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 3414069000 bits/sec, 521786 packets/sec
5 minute output rate 5409966000 bits/sec, 673824 packets/sec
4142893680 packets input, 3421981735262 bytes, 0 no buffer
Received 53990 broadcasts (25320 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
5290299377 packets output, 5334375403783 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

4500E#sh version
Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-IPBASEK9-M), Version 15.2(2)E8, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2018 by Cisco Systems, Inc.
Compiled Mon 22-Jan-18 06:32 by prod_rel_team

ROM: 12.2(44r)SG9
4500E uptime is 24 weeks, 10 hours, 22 minutes
System returned to ROM by power-on
System image file is "bootflash:cat4500e-ipbasek9-mz.152-2.E8.bin"
Hobgoblin Revision 20, Fortooine Revision 1.40

Last reload reason: power-on

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C4948E (MPC8548) processor (revision 5) with 1048576K bytes of memory.
Processor board ID CAT1524S1KJ
MPC8548 CPU at 1GHz, Cisco Catalyst 4948E
Last reset from PowerUp
4 Virtual Ethernet interfaces
48 Gigabit Ethernet interfaces
4 Ten Gigabit Ethernet interfaces
511K bytes of non-volatile configuration memory.

Configuration register is 0x2101

4500E#sh run int port-channel 45
Building configuration...

Current configuration : 99 bytes
!
interface Port-channel45
description 4500---3560 Bundle
switchport
switchport mode trunk
end

4500E#

Hope to find some help .

Thanks in advance .