Port channels and delicate traffic - Streaming

joshhboss · ‎01-13-2022

So im doing an event and I have a streaming trailer connected to my 2960 that is set up to port channel to a MIKROTIK which apparently all interfaces on it are just set to passive LACP so by just connecting them. A Port channel forms. but i was wondering on traffic like a stream (from 9 encoders) would the hopping from link to link cause issues. Sorry i know this might seem like a rookie question. Just trying to learn.

Thank you.

Seb Rupik · ‎01-13-2022

Hi there,

Despite a port-channel having multiple bundled links, the load balancing algorithm will select a single link for a stream to use. It is per-stream load-balancing, not per-packet load-balancing. To that end a single traffic stream cannot enjoy the full aggregate bandwidth of a port-channel, instead the available for a single stream bandwidth is that of a single link in the bundle.

Of course, an end device with multiple streams would hopefully have its traffic balanced across the port-channel, but it is not round-robin, and the hash algorithm may not spread the load across the bundle.

cheers,

Seb.

joshhboss · ‎01-13-2022

so strange.. an engineer that i know told me that is was round robin. the reason i asked him was because one of the links was getting an error.

Jan 12 08:03:30.861 EST: %SFF8472-5-THRESHOLD_VIOLATION: Gi1/0/50: Rx power low warning; Operating value: -22.9 dBm, Threshold value: -19.0 dBm.

so i asked him if that would effect the link the port channel or cause me any kind of problems between the 2 switches

and his words to me exactly were..

"Nope a switch uses round robbin to send information though the physical links so I would have someone look at that"

since then i changed the sfps and cleaned the fiber cables and there has been improvement and the errors went away..

but ive had a problem since then where the streams went down for 3 minutes twice in one hour.. i did not notice anything in the logs so i didnt know it maybe it was it going from one link to the other.

Joseph W. Doherty · ‎01-13-2022

Possibly non-Cisco switches might round robin L2 frames on LACP bundles, but to my knowledge, Cisco switches work as @Seb Rupik describes. (BTW, many Cisco switches, such as your 2960, offer various load balancing choices. Worst choice, for your traffic, might result in all traffic, using bundle, just use one-link.) Also possibly your "engineer" has other "bundles" in mind, like multiple L3 links between L3 devices. On those, some Cisco technologies will round-robin flows and/or packets. Lastly, possibly, your "engineer" is just a tad lacking in Cisco knowledge.

Further, the message you posted has nothing to do with port loading, but with quality of the signal being received. Which is why changing SFP and (especially) cleaning the fiber (end) resulted in the issue being resolved.

joshhboss · ‎01-13-2022

Is there anyway to confirm that my settings are set properly. i assume not using round robin . both my ports are configured like this.. and ill add the po1 config as well

interface GigabitEthernet1/0/49
 description PortChanneltoMain
 switchport mode trunk
 channel-group 1 mode active
 ip dhcp snooping trust

interface GigabitEthernet1/0/50
 description PortChanneltoMain
 switchport mode trunk
 channel-group 1 mode active
 ip dhcp snooping trust

interface Port-channel1
description PC-to-MainFiber
switchport mode trunk
ip dhcp snooping trust

SiteOpsOfficeSw204#show etherc load-balance 
EtherChannel Load-Balancing Configuration:
        src-mac

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source MAC address
  IPv4: Source MAC address
  IPv6: Source MAC address

And im surprise my guy was so off.. he really came off to me as extremely knowledgable

Thanks again for all the help.. I really enjoy learning this stuff.

Seb Rupik · ‎01-13-2022

Wow, your switch is *just* using src-mac for hashing, that is going to give fairly poor entropy and guarantee that a devices traffic flows always take the same link in the bundle.

Take a look at the possible options here:

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2960/software/release/12-2_53_se/configuration/guide/2960scg/swethchl.html#33657

Topology matters when choosing the algorithm, eg, if this is a port-channel between two Layer3 devices, such as a /30 subnet between two routers, then using any MAC based hashing would be the wrong choice. Picking src-dst-ip will probably be the best choice, just make sure you have the same algorithm type (L2 or L3) configured on both the switches at either end of the port-channel.

cheers,

Seb.

joshhboss · ‎01-13-2022

This is just going between two switches, but unfortunately im using a Mikrotik CRS328-4C-20S-4S+RM running switch os ( using it because of port density- has 20 sfp ports and 4 sfp+) i do have an older 3750G-12 that has all 12 sfps. but i dont know if swapping it out will help me. and either way i need the ports.

wouldnt the connections using the same link be what i want tho.. so that the streaming encoders and the cloud hosted server they are using are all on the same link.

out of curiosity would configuring the port channel for this situation if i had 2 like switches be something like

port-channel load-balance src-des-ip

Really appreciate the responses..

Edit - maybe src-dst-mac actually? its two switches

Joseph W. Doherty · ‎01-13-2022

Often, as @Seb Rupik suggests, src-dst-ip is the "best" LB choice.

Regarding src-dst-mac that might be okay too, if the all the hosts, both src and dst, are in the same L2 domain. Otherwise, LB just "sees" one or two gateway MACs and then all your traffic will use just single link (either unidirectional or bidirectional).

BTW, the 3750G-12S has SDM templates unique to just that model.

joshhboss · ‎01-13-2022

All of this traffic is destined for the internet there is maybe just 2 printers on site. And my main concern at the moment are these encoders at the end of this port channel link (server for encoder is in the cloud). And it’s actually be working just fine for the past 4 hours but I was wondering if I should something different that would work better.

Seb Rupik · ‎01-14-2022

Internet bound traffic will benefit more greatly from using 'src-dst-ip' .

Hopefully it is becoming clear that using a port-channel does not necessarily grant a device the benefit of using all the links and therefore all of the aggregate throughput. If maximum throughput is of primary concern then the real solution is to increase the link speed from 1Gbps to 10Gbps or higher through use of SFP+ or QSFP's.... so long as your platform supports it. That said if the destination is web based, then you constraint will be the link between you and your cloud provider. Unless you are using something like ExpressRoute/ DirectConnect all the discussion about port-channels is academic as you are going to be limited by your internet edge router.

cheers,

Seb.

joshhboss · ‎01-14-2022

My concern is actually bandwidth in crunches in past events when new locations needed internet we started daisy chaining off existing switches that were 1 gig and about 3 events ago the Uplink switch to the first switches of the daisy chain became saturated. So what I've been trying to do now is do 2 gig links between all my switch incase we are ever in these situations where we have to daisy chain. Its pretty difficult to keep best practice (especially when im learning best practice as i go lol) when the network is changing so rapidly and the demands are needed to be met almost instantly. so I'm just trying to be prepare for things that happen in the future.

Goal are basically

1 - Increase bandwidth so I have a lesser chance of a port getting saturated by either daisy chaining of switch or for any other reason

2 - Not port channel in a way that doesnt actually benefit me. I need to protect things like the streams, and VOIP calls from using the wrong links.

Again thank you guys for the responses. I really enjoy this stuff and appreciate your insight.

Seb Rupik · ‎01-14-2022

You should take a look at three-tier network topologies or 'collapsed core' and understand why different types of devices/ platforms are used at the different levels which complement their required function. Daisy-chaining loads of edge switches sounds like a short-term kludge which you should want to design out.

Can you not get structured cabling to connect these new switches closer to the core?

An edge switch is designed with a certain amount of over-subscription, but it is designed to process traffic from devices directly connected to its edge ports, not 48+ other devices connected to another downstream switch. That is the role of a distribution switch.

You first goal is commendable. As for the second, you should still want port-channels, if not for performance they should be used to provide resilience to your topology.

cheers,

Seb.

joshhboss · ‎01-14-2022

Im looking for a ASR920 with i think 24 SFP ports and some SFP+ ports for a distribution switch. i found one for $2,000. Then maybe with that purchase ill be able to get cisco support.. I cant get it now because of the age of the switches I have. But you guys have been great. Really appreciate it.

Seb Rupik · ‎01-14-2022

I don't think an ASR is the correct device for that position even if it comes with SFP/SFP+ ports. Take a look at a 3850-12XS, 3850-24XS or maybe a 9300X-12Y or 9300X-24Y. Keep in mind that the 3850 has been EoL'd and the 9300 being the replacement platform are new and expensive.

cheers,

Seb.

joshhboss · ‎01-16-2022

For budgetary reasons lol. i started looking around and I was wondering what your thoughts would be on a switch like this.

Cisco WS-C4500X-16SFP+ Catalyst 16 Port SFP+ IP BASE Switch w/ 750W AC. now this one i could probably pick up a few so i have back ups.. At this moment in time it might be hard for me to get (1) 3850 much less. 2. Im still working on getting more funds together.

or Cisco Catalyst WS-C4500X-24X-ES 24 TenGigabit Eth. Aggregation Switch Dual Power?

orrrr lol WS-C4928-10GE..

As you might see.. budget does mean something lol.. These larger events.. normally have about 3,000 clients but id want to at least be confident to handle 10,000 or more.. Im looking for a router also and i have more money I can allocate to that. But im going with a different brand for that.. but for layer 2, i only trust Cisco.