Etherchannel setup

[PacNWNetTek]

I have a Cisco 2960-X AL and a 3850-XS 10G SFP+. I have added an etherchannel bundle of 2 interfaces on both switches and the port-channel is up. The goal is to increase the throughput capability of the 2960-X. My question is, as the ports say they're up/connected on both switches, on the 3850 itself the lights are flashing amber. Is there some error im missing? Or is this normal? The following is my etherchannel detail out and logs.

Switch#sho etherchannel deta
Channel-group listing:
----------------------

Group: 1
----------
Group state = L2
Ports: 2 Maxports = 16
Port-channels: 1 Max Port-channels = 16
Protocol: LACP
Minimum Links: 0

Ports in the group:
-------------------
Port: Te1/0/23
------------

Port state = Up Mstr Assoc In-Bndl
Channel group = 1 Mode = Active Gcchange = -
Port-channel = Po1 GC = - Pseudo port-channel = Po1
Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.
A - Device is in active mode. P - Device is in passive mode.

Local information:
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Te1/0/23 SA bndl 32768 0x1 0x1 0x118 0x3D

Partner's information:

LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Te1/0/23 SA 32768 00d6.febe.0200 5s 0x0 0x1 0x11D 0x3D

Age of the port in the current state: 0d:00h:29m:24s

Port: Te1/0/24
------------

Port state = Up Mstr Assoc In-Bndl
Channel group = 1 Mode = Active Gcchange = -
Port-channel = Po1 GC = - Pseudo port-channel = Po1
Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.
A - Device is in active mode. P - Device is in passive mode.

Local information:
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Te1/0/24 SA bndl 32768 0x1 0x1 0x119 0x3D

Partner's information:

LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Te1/0/24 SA 32768 00d6.febe.0200 12s 0x0 0x1 0x11C 0x3D

Age of the port in the current state: 0d:00h:29m:24s

Port-channels in the group:
---------------------------

Port-channel: Po1 (Primary Aggregator)

------------

Age of the Port-channel = 0d:01h:15m:33s
Logical slot/port = 12/1 Number of ports = 2
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
Port security = Disabled

Ports in the Port-channel:

Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 00 Te1/0/23 Active 0
0 00 Te1/0/24 Active 0

Time since last port bundled: 0d:00h:29m:24s Te1/0/23
Time since last port Un-bundled: 0d:00h:30m:00s Te1/0/24

Switch#sho logging
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.

No Inactive Message Discriminator.

Console logging: level debugging, 98 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 98 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Persistent logging: disabled

No active filter modules.

Trap logging: level informational, 100 message lines logged
Logging Source-Interface: VRF Name:

Log Buffer (4096 bytes):
ED: Interface TenGigabitEthernet1/0/24, changed state to administratively down
*Jul 2 17:13:09.737: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to down
*Jul 2 17:13:09.742: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/24, changed state to down
*Jul 2 17:13:33.136: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:13:33.139: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/24, changed state to up
*Jul 2 17:13:34.389: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:13:34.486: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/24, changed state to up
*Jul 2 17:13:35.387: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up
*Jul 2 17:13:36.388: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up
*Jul 2 17:13:57.232: %SYS-5-CONFIG_I: Configured from console by console
*Jul 2 17:17:32.556: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to down
*Jul 2 17:17:33.560: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/23, changed state to down
*Jul 2 17:17:41.629: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:17:44.312: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:22:02.261: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down
*Jul 2 17:22:03.253: %LINK-5-CHANGED: Interface TenGigabitEthernet1/0/23, changed state to administratively down
*Jul 2 17:22:03.260: %LINK-5-CHANGED: Interface TenGigabitEthernet1/0/24, changed state to administratively down
*Jul 2 17:22:03.262: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down
*Jul 2 17:22:04.253: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to down
*Jul 2 17:22:04.259: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/24, changed state to down
*Jul 2 17:22:26.646: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:22:26.649: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/24, changed state to up
*Jul 2 17:22:27.962: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/24, changed state to up
*Jul 2 17:22:28.961: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up
*Jul 2 17:22:29.798: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:22:29.961: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up
*Jul 2 17:24:29.918: %SYS-5-CONFIG_I: Configured from console by console
*Jul 2 17:57:45.801: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to down
*Jul 2 17:57:46.791: %LINK-5-CHANGED: Interface TenGigabitEthernet1/0/23, changed state to administratively down
*Jul 2 17:57:46.799: %LINK-5-CHANGED: Interface TenGigabitEthernet1/0/24, changed state to administratively down
*Jul 2 17:57:46.801: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down
*Jul 2 17:57:47.791: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to down
*Jul 2 17:57:47.799: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/24, changed state to down
*Jul 2 17:58:20.233: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:58:20.238: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/24, changed state to up
*Jul 2 17:58:23.200: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/23, changed state to up
*Jul 2 17:58:23.232: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/24, changed state to up
*Jul 2 17:58:24.198: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up
*Jul 2 17:58:25.198: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up
*Jul 2 18:08:58.526: %SYS-5-CONFIG_I: Configured from console by console

[Reza Sharifi]

Is the configuration the same on both switches?

Can you post the following outputs from both switches?

sh run int po1

sh run int te1/0/23

sh run int te1/0/24

sh ether-ch summary

HTH

[PacNWNetTek]

Yes, they are the same 

2960X

interface Port-channel1
switchport trunk native vlan 999
switchport trunk allowed vlan 92,100-107,128,132,133,136,140,144,148,152,156
switchport trunk allowed vlan add 160,164,168,172
switchport mode trunk
switchport nonegotiate
spanning-tree bpdufilter disable
spanning-tree bpduguard disable
end

sho etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator

M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port

Number of channel-groups in use: 2
Number of aggregators: 2

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
1 Po1(SU) LACP Gi0/27(P) Gi0/28(P)

interface GigabitEthernet0/27
switchport trunk native vlan 999
switchport trunk allowed vlan 92,100-107,128,132,133,136,140,144,148,152,156
switchport trunk allowed vlan add 160,164,168,172
switchport mode trunk
switchport nonegotiate
srr-queue bandwidth share 1 30 35 5
priority-queue out
mls qos trust dscp
auto qos trust dscp
spanning-tree bpdufilter disable
spanning-tree bpduguard disable
channel-group 1 mode active

interface GigabitEthernet0/28
description DL092-MDF-1
switchport trunk native vlan 999
switchport trunk allowed vlan 92,100-107,128,132,133,136,140,144,148,152,156
switchport trunk allowed vlan add 160,164,168,172
switchport mode trunk
switchport nonegotiate
srr-queue bandwidth share 1 30 35 5
priority-queue out
mls qos trust dscp
auto qos trust dscp
spanning-tree bpdufilter disable
spanning-tree bpduguard disable
channel-group 1 mode active
end

3850 XS

interface TenGigabitEthernet1/0/23
switchport trunk native vlan 999
switchport trunk allowed vlan 92,100-107,128,132,133,136,140,144,148,152,156
switchport trunk allowed vlan add 160,164,168,172
switchport mode trunk
switchport nonegotiate
auto qos trust dscp
channel-group 1 mode active
spanning-tree bpdufilter disable
spanning-tree bpduguard disable
service-policy input AutoQos-4.0-Trust-Dscp-Input-Policy
service-policy output AutoQos-4.0-Output-Policy
end

interface TenGigabitEthernet1/0/24

switchport trunk native vlan 999
switchport trunk allowed vlan 92,100-107,128,132,133,136,140,144,148,152,156
switchport trunk allowed vlan add 160,164,168,172
switchport mode trunk
switchport nonegotiate
auto qos trust dscp
channel-group 1 mode active
spanning-tree bpdufilter disable
spanning-tree bpduguard disable
service-policy input AutoQos-4.0-Trust-Dscp-Input-Policy
service-policy output AutoQos-4.0-Output-Policy
end

switchport trunk native vlan 999
switchport trunk allowed vlan 92,100-107,128,132,133,136,140,144,148,152,156
switchport trunk allowed vlan add 160,164,168,172
switchport mode trunk
switchport nonegotiate
spanning-tree bpdufilter disable
spanning-tree bpduguard disable

Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator

M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port

A - formed by Auto LAG

Number of channel-groups in use: 1
Number of aggregators: 1

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
1 Po1(SU) LACP Te1/0/23(P) Te1/0/24(P)

[Reza Sharifi]

There seems to be some inconsistency here as you are using 10Gig ports on one switch and Gig ports on the other switch. In order for a Portchannel to work correctly the speed and duplex on all the ports must be the same. HTH

1 Po1(SU) LACP Gi0/27(P) Gi0/28(P)

1 Po1(SU) LACP Te1/0/23(P) Te1/0/24(P

[PacNWNetTek]

Yes, that is what I remember but would the Te port not just throttle down to 1000/mbs? Im just confused as to why the ports are showing connected with speed set at that. If it is not going to work shouldnt 1 port be suspended to prevent the loop? 

TenGigabitEthernet1/0/24 is up, line protocol is up (connected)
Hardware is Ten Gigabit Ethernet, address is b4a8.b9f9.c998 (bia b4a8.b9f9.c998)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:13, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
43101 packets input, 3923384 bytes, 0 no buffer

TenGigabitEthernet1/0/23 is up, line protocol is up (connected)
Hardware is Ten Gigabit Ethernet, address is b4a8.b9f9.c997 (bia b4a8.b9f9.c997)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:11, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
5 minute input rate 6000 bits/sec, 10 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
200145 packets input, 15239924 bytes, 0 no buffer
Received 200145 broadcasts (200145 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 200145 multicast, 0 pause input
0 input packets with dribble condition detected
40423 packets output, 3256071 bytes, 0 underruns
0 output errors, 0 collisions, 5 interface resets
641 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
Received 43100 broadcasts (42700 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 42700 multicast, 0 pause input
0 input packets with dribble condition detected
40397 packets output, 3248730 bytes, 0 underruns
0 output errors, 0 collisions, 5 interface resets
636 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

GigabitEthernet0/27 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 00d6.febe.021b (bia 00d6.febe.021b)
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:08, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
43824 packets input, 3726022 bytes, 0 no buffer
Received 43824 broadcasts (43555 multicasts)
1 runts, 0 giants, 0 throttles
1 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 43555 multicast, 0 pause input
0 input packets with dribble condition detected
44714 packets output, 4226487 bytes, 0 underruns
0 output errors, 0 collisions, 3 interface resets
2324 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

GigabitEthernet0/28 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 00d6.febe.021c (bia 00d6.febe.021c)
Description: DL092-MDF-1
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:14, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 6000 bits/sec, 10 packets/sec
149142 packets input, 63735369 bytes, 0 no buffer
Received 97207 broadcasts (94279 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 94279 multicast, 0 pause input
0 input packets with dribble condition detected
279343 packets output, 21225189 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
2240 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

[PacNWNetTek]

Added note: I just swapped all the sfp's for sfp+ and went to a cat 3560-CX and the 3850 is displaying the same yet all ports are showing connected.

TenGigabitEthernet1/0/24 is up, line protocol is up (connected)
Hardware is Ten Gigabit Ethernet, address is b4a8.b9f9.c998 (bia b4a8.b9f9.c998)
MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 10Gb/s, link type is auto, media type is SFP-10GBase-SR
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:07, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
5 minute input rate 9000 bits/sec, 8 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
45188 packets input, 4202735 bytes, 0 no buffer
Received 44201 broadcasts (43456 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 43456 multicast, 0 pause input
0 input packets with dribble condition detected
40463 packets output, 3264495 bytes, 0 underruns
0 output errors, 0 collisions, 5 interface resets
668 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

[Reza Sharifi]

Your setup seems to be correct. You are using SFPs in the 10Gig ports to lower the speed to 1Gig and the 10Gig interfaces show as 1Gig.

TenGigabitEthernet1/0/24 is up, line protocol is up (connected)
Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP

Try deleting the Portchannel completely and remove the config from under the physical interfaces and rebuild them in this sequence.

create the PO by adding the config under the physical interfaces

now under the po do a "no sh" to bring up the po and the physical interfaces and test.

HTH

[PacNWNetTek]

Did as you stated, same result. I do see that no ports are being blocked from spanning tree so I think i may be dealing with a bug from what @Leo Laohoo stated.

[paul driver]

Hello

sh spanning-tree
sh spanning-tree blocked ports

[PacNWNetTek]

Switch#sho spanning-tree blockedports

Name Blocked Interfaces List
-------------------- ------------------------------------

Number of blocked ports (segments) in the system : 0

[Leo Laohoo]

---

@PacNWNetTek wrote:

on the 3850 itself the lights are flashing amber.

---

And the port(s) LED flashes amber ONLY on the 3850 (and never on the 2960G), am I correct? 
If yes, then what is the exact firmware of the 3850?

It could either be CSCvj16691 or CSCvd19192.

[PacNWNetTek]

Im thinking you may be right. I am on 16.06.04 and that version is not stated in CSCvj16691 but it could just not have been reported and it also says its not 100%. As far as I can tell stp is not blocking any ports, and its showing correct speeds. My switches arent fully configured so i guess i wont know for sure until I test in the field.

Thanks

[Leo Laohoo]

---

@PacNWNetTek wrote:

I am on 16.06.04 and that version is not stated in CSCvj16691

---

Do not associate the behaviour with the firmware version of the switch or router.  In my personal opinion and observation, Cisco bug IDs are getting more and more unreliable because they are not being updated.  

To get confirmation if you're hitting this bug or not, raise a TAC Case.

[MHM Cisco World]

your PO is SU meaning it "L2 in used" so PO is ok.
the issue if you can help me to be more sure is STP,

check if two LED is flashing or only one.

flashing meaning the STP is in BLK status.

[PacNWNetTek]

Both lights are flashing. STP show 0 blocked ports.