Automate changing the route based on bandwidth utilization in SR

[dundient]

Good day, community.

I have a question about dynamical/automate changing routes based on bandwidth utilization in Segment Routing environment.

For example, from R1's perspective we have 2 links to R2. On link#1 bandwidth usage is limited to 50mbps and link#2 is 100mbps. The main route is through link#1 and after some time, lets say, the link#1 is fully utilized. In this case, we want to automate routes through link#2. Is there any solution to resolve this kind of dynamic route in Segment Routing environment?

Thank you in advance  

[Joseph W. Doherty]

I believe Cisco's PfR has that capability but don't know if it will work with segment routing.

[MHM Cisco World]

Instead I think you can use multi path. 

And router will send packets depends on BW 

I. E. It will send 2 packets via 100 and one packet via 50.

MHM

[Joseph W. Doherty]

@MHM Cisco World using what?

What comes to my mind is EIGRP, which can proportionally LB, although that's not quite what OP asked for.  What technology are you suggesting?

BTW, I recall (?) PfR can either proportionally LB or shift traffic based on link usage cost.  (The latter for cases that the alternative path charges for its usage, so you only want to direct traffic to it when the preferred path is fully utilized.). Also, BTW, PfR does dynamic LB, not just proportional flow LB.  I.e., it monitors actual utilization.

Again, as suitable PfR might be fit this purpose, have no idea whether it can be used with SR.  (I suspect much of PfR technology went into SD-WAN, which is what we're all now supposed to be using. [Right?  Laugh.])

[MHM Cisco World]

https://www.cisco.com/c/en/us/support/docs/ip/enhanced-interior-gateway-routing-protocol-eigrp/13677-19.html

Traffic Sharing

EIGRP not only provides unequal cost path load balancing, but also intelligent load balancing, such as traffic sharing. In order to control how traffic is distributed among routes when there are multiple routes for the same destination network that have different costs, use the traffic-share balanced command. With the keyword balanced, the router distributes traffic proportionately to the ratios of the metrics that are associated with different routes. This is the default setting:

router eigrp 1 
network x.x.x.x 
variance 2 
traffic-share balanced

[Joseph W. Doherty]

Ah, you did have EIGRP in mind; fine.

Anyway, again, don't believe EIGRP actively monitors actual load usage, it just proportionally routes, for flows, or if doing packet by packet routing, for packet.  Overtime, you often see actual LB in the expected proportions, but short term, they can be totally out of expected proportions.

BTW, although something like PfR does much better LB, it's not perfect because there's an intentional (for good reasons) delay in its response.

Decades ago, I had a case of a T3 and T1, side by side.  PfR, was configured to maintain equal usage agross the two paths, also by default, the T3 was the better path, to the routing protocol.

Honestly, there was such a disparity in bandwidths, one could well argue whether to even allow PfR to direct any traffic to it.  However, it was useful for learning how to work with the technology.

[Richard Burts]

@Joseph W. Doherty I had an experience similar to what you describe. I was working with a customer who had a server at a remote site. Every night a process on that server transmitted records of its activity to a backup server at HQ. There were 2 links between the sites, one much higher capacity than the other. It had been set up as primary and failover. But the process was running long time each night and they wanted to improve performance. So they requested that we change the configuration and use load sharing. We made the change. That night the backup did not run quicker, but ran much more slowly. When we investigated we found that there were so many out of order packets and retransmissions that it had negative impact on performance.

[Joseph W. Doherty]

@Richard Burts actually, that's not the issue I had with the combo of a T3 and T1, although what you describe is the classical problem of per-packet LB regardless of ECMP or unequal cost multi path (although per packet issue is generally even worse on the UCMP).

BTW, there are ways to mitigate the impact of packet-by-packet routing, but they often require changes on every source host and might not be configurable except for TCP.

Anyway, PfR basically injects routes, often specific destination routes, so if per flow routing is in operation, a particular flow uses a particular interface.

In my case of T3 and T1, say you have a FTP and VoIP come up.  Assuming "normal" routing, both will use the T3.  PfR, noticing the T1 is unused and also noticing (the very likely disparity) in bandwidth usage, of the FTP vs. VoIP flows, will redirect the VoIP flow to the T1.  Which is all fine and great.

However, say you have just two FTPs, each all ready using more than a T1's bandwidth.  Neither flow (I believe) will be redirected to the T1.

But, if say you have a 60 FTP flows, each using about 1/60 of the T3 bandwidth, a couple of those might be redirected to the T1.  So, you would obtain the advantage of the additional bandwidth of the T1, but in such a case, is it really an advantage?

Well, consider, on the T3, as FTP flows come and go, vary in their bandwidth needs, even with 60 active flows, although a flow should, on average, be able to obtain 1/60 of the bandwidth, it may be able to obtain more, when other flows aren't using their fair share.  The additional "burst" bandwidth, might exceed what a T1 can provide at all.  So, by having any of those FTP flows, redirected to the T1, you might actually be limiting those individual flow's bandwidth.

However, if you have a mixture of flow kinds, like my initial example of FTP and VoIP, getting VoIP flows on the T1 might be ideal.

One of the differences I recall between PfR and its initial variant, OER, the former could use QoS information for part of its decision making,  I.e. VoIP flows might be allowed to use both links, but FTP couldn't use the T1 except if the T3 was unavailable.  I.e. the "best" of both worlds (if you wanted to manage the additional PfR complexity).

Laugh - PfR has some "interesting" capabilities, so much so, I like to tell, when we started using it to control our WAN traffic (between our sites), who should "complain", but out network performance monitoring engineer.  Why?  Because, all of a sudden,  no longer were there any WAN performance issues!

Actually, there were still issues, but another feature of this technology, it can monitor end-to-end performance, and reroute (if there's an alternative) around a network performance issue.  PfR was doing this, so quickly, our network monitoring (and our users) stop "seeing" WAN performance issues.

This caused us, a bit of work, how to "see" WAN issues being avoided by PfR.  Some of which, required, "watching" PfR logging entries, i.e. when it rerouted something, "why" was it rerouted.  Also, getting some networking monitoring traffic to NOT be rerouted by PfR.

Oh, and if you believe I believe PfR is great technology, yes I do, although truly learning it can be, ah, "interesting", like the day I managed to configure (unintentionally) where a WAN performance issue in the US, between Philadelphia and Boston, to opt to take an alternate path via Toyoko (no joke - it did eliminate packet loss, but sort of added a tad bit of latency - I did figure out, what I did wrong, and avoided similar future PfR "corrections").