VSS LAG C6880X

[CliveG]

I have 2 x 1gbps upstream LAG to Provider and it is heavily utilising the following on the ports:

Port 1 - 100% with dropped packets

Port 2 - 2% Utilisation

Cisco's default hash algorithm for Port-Channels appears to be IP based and therefore will utilise the same port out of the 2 each time. The LAG has been configured default.

Is there a way to change the configuration so I can load balance correctly over the LAG (utilising both ports with the same loading) without actually dropping the LAG (I cannot afford for the LAG to be dropped as it is utilised 24/7/365 for streaming)?

Many thanks

[balaji.bandi]

Can you post the below information : (is this port channel configured LACP ?)

show version

show etherchannel x summary

show etherchannel x port-channel

show etherchannel load-balance

show interface gix/x and x/y part of port-channel.

is this l2 port-channel or l3 port-channel, where is the Routing take place in this switch ?

[CliveG]

There is no Etherchannel option on the switch.

This is a layer 3 port-channel. The config is the usual on the physical interfaces of the following:

channel-group xx mode active

And then on the port-channel is the IP Address and the "no switchport" command.

Not sure what other information I can supply.

[balaji.bandi]

with out seeing any input or what configured, its hard to tell what was the issue and guide to right direction

other than suggest below link to troubleshoot :

https://www.cisco.com/c/en/us/support/docs/lan-switching/etherchannel/12023-4.html

[CliveG]

Configuration is very basic:

Physical interfaces:

no switchport, no ip address, no lldp transmit, no lldp receive, no cdp enable, channel-group 22 mode active

Port-channel 22:

no switchport, <Mac-Address>, ip address xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx, ipv6 address xxx, ipv6 enable, ipv6 nd ra suppress, no mop enabled

That is the configuration of the two physical ports and the Port-Channel.

As I mentioned, there is no "show etherchannel" available.

The VSS is comprised of 2 x C6880X Switches. The IOS version is:

(c6880x-ADVENTERPRISEK9-M) Version 15.1(2)SY5, Release software (fc7)

[balaji.bandi]

its strange , never seen that command missing in Cat 6800 switches, i had few of them it works that command ?

#show etherchannel 22  detail  ( will give you information how the load-sharing working)

#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-mixed-ip-port enhanced

#show etherchannel 22 port-channel

[Rich R]

Agreed the command should be there as per https://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/217873-troubleshoot-packet-forwarding-for-6500.html

[CliveG]

Strange. When working from home yesterday the command "show etherchannel" was not available, yet today, I am in the office and it is available. So, here is the output of the commands:

show etherchannel 22 port-channel

Port-channel: Po22 (Primary Aggregator)

------------

Age of the Port-channel = 2498d:09h:50m:08s
Logical slot/port = 46/5 Number of ports = 2
HotStandBy port = null
Passive port list = Te1/5/7 Te2/5/7
Port state = Port-channel L3-Ag Ag-Inuse
Protocol = LACP
Port security = Disabled
Fast-switchover = disabled
Fast-switchover Dampening = disabled
Load share deferral = disabled
Is fex host PO = FALSE

Ports in the Port-channel:

Index Load Port EC state No of bits
------+------+------------+------------------+-----------
0 FF Te1/5/7 Active 8
1 FF Te2/5/7 Active 8

Time since last port bundled: 33d:09h:33m:50s Te2/5/7
Time since last port Un-bundled: 33d:09h:37m:27s Te1/5/7

Last applied Hash Distribution Algorithm: Adaptive

show etherchannel 22 summary:

Number of channel-groups in use: 10
Number of aggregators: 10

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
22 Po22(RU) LACP Te1/5/7(P) Te2/5/7(P)

Last applied Hash Distribution Algorithm: Adaptive

show etherchannel load-balance:

EtherChannel Load-Balancing Configuration:
src-dst-ip enhanced
mpls label-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address
MPLS: Label or IP

show interface te1/5/7:

TenGigabitEthernet1/5/7 is up, line protocol is up (connected)
Hardware is C6k 10000Mb 802.3, address is 0008.e3ff.fc78 (bia 0008.e3ff.fc78)
Description: LINX_TIC-07271_TFM6J02_1
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 192/255, rxload 28/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 1000BaseLH
input flow-control is off, output flow-control is off
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters 5w4d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 74953640
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 110818000 bits/sec, 19528 packets/sec
5 minute output rate 755152000 bits/sec, 67251 packets/sec
L2 Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes
L3 in Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes
L3 out Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes
55738985133 packets input, 38659943742770 bytes, 0 no buffer
Received 1667978394 broadcasts (1106696094 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
159573936223 packets output, 217076676439953 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets

show interface te2/5/7:

TenGigabitEthernet2/5/7 is up, line protocol is up (connected)
Hardware is C6k 10000Mb 802.3, address is 0008.e3ff.fc78 (bia 0008.e3ff.fc78)
Description: LINX_TIC-07271_TFM6J02_2
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 2/255, rxload 13/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 1000BaseLH
input flow-control is off, output flow-control is off
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters 21w6d
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 52644000 bits/sec, 14208 packets/sec
5 minute output rate 11071000 bits/sec, 1483 packets/sec
L2 Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes
L3 in Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes
L3 out Switched: ucast: 0 pkt, 0 bytes - mcast: 0 pkt, 0 bytes
153656591222 packets input, 110702624331932 bytes, 0 no buffer
Received 5116681355 broadcasts (3525897846 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
19775055057 packets output, 19043748528536 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets

You will note the transmit loads is completely out of synch. One is being utilised heavily and is dropping packets and the other is hardly utilised. This is the problem and I need to try and load-balance this across both links without dropping the LAG.

[Rich R]

Your load balancing settings: src-dst-ip enhanced & mpls label-ip

So what is the nature of the traffic? 
For example if most/all of your traffic is in a VPN tunnel with a single src/dst IP pair that would explain why it's doing that.
Ideally you want a good mix of src/dst addresses to achieve a good balance of traffic.

[CliveG]

I will need to check exactly what is going out, but as well as VPN we have Multicast, Data etc.

This connection is an eBGP peering to multiple other peers within the LINX LON1 LAN. All I know for sure, curerntly, is that one link is heavily utilised and the other is not and it is in a LAG. This should be load-balanced correctly so that both links are utilised equally. I fully understand why IP Address would look to be utilised purely from a TCP stream perspective, but surely that defeats the purpose of a LAG?

[Rich R]

That's why you need to understand exactly what your traffic is and which load balancing algorithm is most appropriate for you.
The link @balaji.bandi provided has the options.  If none of those will work for you then you'll need to look at other ways to spread the traffic.  Note the warning there that changing the hash algorithm is disruptive and is recommended to be done out of production hours.  Note there's a test command you can use to determine which link a specific flow will use.