EIGRP Unequal Cost Load Balancing - 'Variance' Command

[Rob M]

Hi All,

As I'm doing an EIGRP lab, I noticed a strange behavior with the 'variance' command. Its very possible that I'm missing something, which is why I wanted to ask here so if someone can point out the cause of this behavior or if otherwise the documentation of this command is not accurate.

The issue is around the fact that for a route to be affected by the variance command it needs to pass the feasibility condition, i.e it is a feasible successor in the EIGRP topology table.

As for the topology (its based on INE EIGRP labs), Think of R6 as connected directly to two routers, R7 via Ethernet0/1.67 and R1 via Ethernet0/1.146. And frankly, the topology here does not matter that much, the end goal is that I'm trying to manipulate Delay on these two interfaces to unequally load balance to 155.1.9.0/24

Focusing on R6 and before changing anything:

R6(config-subif)#do sh ip eigrp top all | sec 155.1.9.0/24
P 155.1.9.0/24, 1 successors, FD is 76800, serno 662
via 155.1.67.7 (76800/51200), Ethernet0/1.67

---

R6(config-subif)#do sh ip eigrp top 155.1.9.0 255.255.255.0
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 76800
Descriptor Blocks:
155.1.67.7 (Ethernet0/1.67), from 155.1.67.7, Send flag is 0x0
Composite metric is (76800/51200), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 3000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2
Originating router is 150.1.9.9

Now, I'm going to change the delay on VLAN67 to 300 and VLAN146 to 2100.

Here is where things get interesting:

R6(config-subif)#do sh ip eigrp top all | sec 155.1.9.0/24
P 155.1.9.0/24, 2 successors, FD is 76800, serno 701
via 155.1.67.7 (128000/51200), Ethernet0/1.67
via 155.1.146.1 (640000/102400), Ethernet0/1.146

---

R6(config-subif)#do sh ip eigrp top | sec 155.1.9.0/24
P 155.1.9.0/24, 2 successors, FD is 76800
via 155.1.67.7 (128000/51200), Ethernet0/1.67

---

R6(config-subif)#do sh ip eigrp top 155.1.9.0 255.255.255.0
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 2 Successor(s), FD is 76800
Descriptor Blocks:
155.1.67.7 (Ethernet0/1.67), from 155.1.67.7, Send flag is 0x0
Composite metric is (128000/51200), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 5000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2
Originating router is 150.1.9.9
155.1.146.1 (Ethernet0/1.146), from 155.1.146.1, Send flag is 0x0
Composite metric is (640000/102400), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 25000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 4
Originating router is 150.1.9.9

---

R6(config-router)#do sh ip rou 155.1.9.0
Routing entry for 155.1.9.0/24
Known via "eigrp 100", distance 90, metric 128000, type internal
Redistributing via eigrp 100
Last update from 155.1.67.7 on Ethernet0/1.67, 00:00:08 ago
Routing Descriptor Blocks:
* 155.1.67.7, from 155.1.67.7, 00:00:08 ago, via Ethernet0/1.67
Route metric is 128000, traffic share count is 1
Total delay is 5000 microseconds, minimum bandwidth is 10000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 2

Now according to the above, the route that is being learned from 155.1.146.1 has an RD of 102400 which is currently higher than the FD 76800 which means it did not satisfy the FC. All the show output above also confirms that, as it does not show in the topology table without using all-links keyword or with the exact prefix.

So, no matter how much the value of the variance command is, this route should NEVER be considered by EIGRP as it cannot guarantee that its loop free - correct? well, not in this case...

I'm going now to configure variance of 5.

R6(config-router)#do sh ip rou 155.1.9.0
Routing entry for 155.1.9.0/24
Known via "eigrp 100", distance 90, metric 128000, type internal
Redistributing via eigrp 100
Last update from 155.1.146.1 on Ethernet0/1.146, 00:00:07 ago
Routing Descriptor Blocks:
155.1.146.1, from 155.1.146.1, 00:00:07 ago, via Ethernet0/1.146
Route metric is 640000, traffic share count is 1
Total delay is 25000 microseconds, minimum bandwidth is 10000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 4
155.1.67.7, from 155.1.67.7, 00:00:07 ago, via Ethernet0/1.67
Route metric is 128000, traffic share count is 5
Total delay is 5000 microseconds, minimum bandwidth is 10000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 2

---

R6(config-router)#do sh ip rou | sec 155.1.9.0
D 155.1.9.0/24 [90/640000] via 155.1.146.1, 00:00:17, Ethernet0/1.146
[90/128000] via 155.1.67.7, 00:00:17, Ethernet0/1.67

---

R6(config-router)#do sh ip eigrp top 155.1.9.0 255.255.255.0
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 2 Successor(s), FD is 76800
Descriptor Blocks:
155.1.67.7 (Ethernet0/1.67), from 155.1.67.7, Send flag is 0x0
Composite metric is (128000/51200), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 5000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2
Originating router is 150.1.9.9
155.1.146.1 (Ethernet0/1.146), from 155.1.146.1, Send flag is 0x0
Composite metric is (640000/102400), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 25000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 4

So as you can see, even though that route does not satisfy the FC, EIGRP still decided to use it and install it in the routing table, Even unequally load balancing (ratio is 5 to 1)

Anyone can shed some light on why this is happening, that would be really great.

Thanks in advance.

[Rob M]

BTW - I know you're probably going to laugh, but I can assure you that Kevin Wallace is wrong. There are many resources and authors got this wrong. FD is one of the most poorly/wrongly documented topic in EIGRP. I just hope someone else would chime in to confirm what I'm saying....

[Meheretab Mengistu]

Hi Rob,

Please allow me to jump in and share my point of view. 

I used the following link:- https://www.cisco.com/c/en/us/support/docs/ip/enhanced-interior-gateway-routing-protocol-eigrp/16406-eigrp-toc.html#anc7

According to the above link, I humbly disagree with your understanding of "Feasible Distance (FD)". Here are some of the notes from the link:

Feasible Distance, Reported Distance, and Feasible Successor

Feasible distance is the best metric along a path to a destination network, including the metric to the neighbor advertising that path. Reported distance is the total metric along a path to a destination network as advertised by an upstream neighbor. A feasible successor is a path whose reported distance is less than the feasible distance (current best path).

As you can see on the note below from the same document, the FD is changed:

When the link between Routers One and Three goes down, Router One examines each path it knows to Network A and finds that it has a feasible successor through Router Four. Router One uses this route, using the metric through Router Four as the new feasible distance. The network converges instantly, and updates to downstream neighbors are the only traffic from the routing protocol.

Let me ask you a question:- In your example, R6 learnt about 155.1.9.0/24 through interface Gi1.67 and Gi1.146. The best path was through Gi1.67 and the path through Gi1.146 was not a feasible successor. Now, if you shutdown interface Gi1.67, you will learn a route through Gi1.146 as the best path. Do not you think that will conflict with your understanding of FD?

HTH,

Meheretab

[Rob M]

Hi,
Thanks for jumping in! the more people participate here the better for sure, we're all trying to learn here.
I've seen that link and many many others from Cisco docs and others. . You're saying that FD is the best metric for a route. I'm going to copy a portion of the show commands that I shared previously on this thread:

R6(config-subif)#do sh ip eigrp top 155.1.9.0 255.255.255.0
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 2 Successor(s), FD is 76800
Descriptor Blocks:
155.1.67.7 (Ethernet0/1.67), from 155.1.67.7, Send flag is 0x0
Composite metric is (128000/51200), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 5000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2
Originating router is 150.1.9.9
155.1.146.1 (Ethernet0/1.146), from 155.1.146.1, Send flag is 0x0
Composite metric is (640000/102400), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 25000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 4
Originating router is 150.1.9.9

 Can you use EIGRP terminology to name these values?
76800 = ?
128000/51200=?
640000/102400=?

Let me ask you a question:- In your example, R6 learnt about 155.1.9.0/24 through interface Gi1.67 and Gi1.146. The best path was through Gi1.67 and the path through Gi1.146 was not a feasible successor. Now, if you shutdown interface Gi1.67, you will learn a route through Gi1.146 as the best path. Do not you think that will conflict with your understanding of FD?

I'm guessing that you are referring to this:

R6(config-subif)#do sh ip eigrp top all | sec 155.1.9.0/24
P 155.1.9.0/24, 2 successors, FD is 76800, serno 701
via 155.1.67.7 (128000/51200), Ethernet0/1.67
via 155.1.146.1 (640000/102400), Ethernet0/1.146

I can explain exactly what happens here, we have one route that is installed in the routing table via 155.1.67.7 since it has the best metric and satisfies the FC. Second route does not satisfy the FC. If I shutdown Ethernet0/1.67, R6 will have no other valid routes to use, it will go active and send queries to its neighbors. Once all queries are in, the router will install the route via 155.1.146.1 and will update the FD to 640000. I even labbed this for you here:

DUAL: Destination 155.1.9.0/24 for tid 0
EIGRP-IPv4(100): Find FS for dest 155.1.9.0/24. FD is 76800, RD is 128000 on tid 0
EIGRP-IPv4(100): 155.1.67.7 metric 72057594037927935/72057594037927935
EIGRP-IPv4(100): 155.1.146.1 metric 640000/102400 not found Dmin is 640000
DUAL: AS(100) Peer total 1 stub 0 template 1 for tid 0
DUAL: AS(100) Dest 155.1.9.0/24 entering active state for tid 0.
EIGRP-IPv4(100): Set reply-status table. Count is 1.
EIGRP-IPv4(100): Not doing split horizon
DUAL: Destination 150.1.4.4/32 for tid 0
EIGRP-IPv4(100): rcvreply: 155.1.9.0/24 via 155.1.146.1 metric 640000/102400 for tid 0
EIGRP-IPv4(100): reply count is 1
DUAL: AS(100) Clearing handle 1, count now 0
DUAL: AS(100) Freeing reply status table
EIGRP-IPv4(100): Find FS for dest 155.1.9.0/24. FD is 72057594037927935, RD is 72057594037927935 on tid 0found
DUAL: AS(100) Removing dest 155.1.9.0/24, nexthop 155.1.67.7
DUAL: AS(100) RT installed 155.1.9.0/24 via 155.1.146.1
DUAL: AS(100) Send update about 155.1.9.0/24. Reason: metric chg on tid 0
DUAL: AS(100) Send update about 155.1.9.0/24. Reason: new if on tid 0
EIGRP-IPv4(100): rcvreply: 155.1.37.0/24 via 155.1.146.1 metric 588800/51200 for tid 0
EIGRP-IPv4(100): reply count is 1
DUAL: AS(100) Clearing handle 1, count now 0
DUAL: AS(100) Freeing reply status table
EIGRP-IPv4(100): Find FS for dest 155.1.9.0/24. FD is 72057594037927935, RD is 72057594037927935 on tid 0found
DUAL: AS(100) Removing dest 155.1.9.0/24, nexthop 155.1.67.7
DUAL: AS(100) RT installed 155.1.9.0/24 via 155.1.146.1
DUAL: AS(100) Send update about 155.1.9.0/24. Reason: metric chg on tid 0
DUAL: AS(100) Send update about 155.1.9.0/24. Reason: new if on tid 0


R6(config-subif)#do sh ip eigrp top 155.1.9.0/24
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 640000
Descriptor Blocks:
155.1.146.1 (Ethernet0/1.146), from 155.1.146.1, Send flag is 0x0
Composite metric is (640000/102400), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 25000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 4
Originating router is 150.1.9.9

I dont think this conflicting what I've been saying so far...

[Deepak Kumar]

Hi,

Thanks for making correct me. I was studying his book and it seemed wrong. After a long argument with you I tried to check some official guide or Book then I found this series issue. Today I have purchased new Book for my CCIE study and found some guide as:

Yes, FD will one for a route and another will be CD. CD will never call an FD.

Unlike most internal routing protocols, EIGRP has a feature that allows you to distribute a load of your traffic across multiple unequal-cost paths and not just over paths providing the least distance to a destination. This feature is amply named unequal-cost load balancing.
The key to unequal-cost load balancing is the presence of Feasible Successors. These routers provide a guaranteed loop-free path to the destination, although not necessarily the shortest one. Precisely this fact can be leveraged by EIGRP: Paths through Feasible Successors can be installed to the routing table and used along with the best available
path even when the route is in the Passive state. Unequal-cost load balancing is enabled through the variance multiplier command. In named mode, the variance is configured in the topology base section. The multiplier value essentially defines how many times worse than the best path a route through a Feasible Successor can be to be still used by EIGRP for unequal-cost load balancing. More precisely, if the variance is set to the value V, for each destination individually, the router checks whether any path over a Feasible Successor meets the following condition
(CD stands for Computed Distance):
CD via Successor < CD via Feasible Successor in question < V × CD via Successor
If it does, it will be installed into the routing table through the corresponding Feasible Successor.
A multiplier of 1, which is the default, implies that no unequal-cost load balancing is being performed. The current value of the variance multiplier can always be verified in the show ip protocols command output. If multiple unequal-cost paths to a destination are installed into the routing table, the router will forward proportionally less traffic over the worse paths, and vice versa. The amount of traffic flowing over a particular path can be computed as this ratio:
Highest Installed Path Metric / Path Metric As an example, if there are four paths over Successors and Feasible Successors to a destination with metrics 1100, 1100, 2000, and 4000, the amounts of traffic over these paths
would be 4000/1100 = 3, 4000/1100 = 3, 4000/2000 = 2, and 4000/4000 = 1, so the true traffic share ratio would be 3:3:2:1 (recall that IOS routers perform integer division).


It is once again important to realize that the key to performing unequal-cost load balancing is first to have Feasible Successors toward a destination identified in the topology table. Routers that do not meet the Feasibility Condition and thus are not considered Feasible Successors are not considered in the unequal-cost load balancing, either. To utilize several neighbors as Feasible Successors, you might need to perform judicious metric tweaking so that the neighbors pass the Feasibility Condition check. Keep in mind that the unequal-cost paths installed into the routing table also count toward the maximum number of parallel paths to a destination configured using the maximum-paths command. Depending on your network topology and requirements, it might be necessary to modify this setting.

Now coming on your point, 

Yes, If there will no valid route and EIFRP route will go in active mode and installed new route but there the last FD will change and new FD will calculate new route details.

If you remember, we asked you about any changes in the routing table/interface/metric etc.

Regards,

Deepak Kumar

[Rob M]

Hey Deepak,

I'm glad that we are finally in agreement! I know that FD/CD/RD are very poorly documented even on cisco's docs but now you have the accurate understanding of these components that are the core of EIGRP.

And yes, I'm changing the Delay on purpose as I said in my first post, this is a lab environment and I'm just testing things.
Now that we both agree on FD vs CD. Let's go back to the initial question. How did changing the variance, cause installation of a route that did not meet FC (please check the very first post):

R6(config-router)#do sh ip eigrp top 155.1.9.0 255.255.255.0
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 2 Successor(s), FD is 76800
Descriptor Blocks:
155.1.67.7 (Ethernet0/1.67), from 155.1.67.7, Send flag is 0x0
Composite metric is (128000/51200), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 5000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2
Originating router is 150.1.9.9
155.1.146.1 (Ethernet0/1.146), from 155.1.146.1, Send flag is 0x0
Composite metric is (640000/102400), route is Internal
Vector metric:
Minimum bandwidth is 10000 Kbit
Total delay is 25000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 4

Question is, how is the second route (with RD of 102400) got installed in the routing table while it clearly does not satisfy the FC (since 102400 is larger than76800). I understand the part where variance of 5 (5x128000=640000) but the very definition of variance, it says it should only work on Feasible successors...

[Rob M]

okay I think i finally got what is going on. I labbed the exact same topology with a different code version, and it worked just fine. Here is the one that is having issues Version 15.4(2)T: 

R4(config-if)#int eth 0/2
R4(config-if)#del 200


*Mar 18 15:03:38.802: EIGRP-IPv4(100): rcvupdate: 192.168.10.0/24 via 1.1.13.3 metric 56320/5120 on tid 0
*Mar 18 15:03:38.802: EIGRP-IPv4(100): Find FS for dest 192.168.10.0/24. FD is 7680, RD is 7680 on tid 0
*Mar 18 15:03:38.802: EIGRP-IPv4(100): 1.1.13.3 metric 56320/5120
*Mar 18 15:03:38.802: EIGRP-IPv4(100): 1.1.13.2 metric 81920/5120
*Mar 18 15:03:38.802: EIGRP-IPv4(100): 1.1.14.4 metric 84736/7936 found Dmin is 56320


P 192.168.10.0/24, 1 successors, FD is 7680, serno 198
via 1.1.13.3 (56320/5120), Ethernet0/2
via 1.1.12.2 (81920/5120), Ethernet0/1
via 1.1.14.4 (85760/8960), Ethernet0/3

Versus  Version 15.2(4)S4

R4(config-if)#int FastEthernet1/0
R4(config-if)#del 200
R4(config-if)#
*Mar 18 12:40:40.399: EIGRP-IPv4(100): rcvupdate: 192.168.10.0/24 via 10.1.42.2 metric 56320/5120 on tid 0
*Mar 18 12:40:40.399: EIGRP-IPv4(100): Find FS for dest 192.168.10.0/24. FD is 7680, RD is 7680 on tid 0
*Mar 18 12:40:40.399: EIGRP-IPv4(100): 10.1.42.2 metric 56320/5120
*Mar 18 12:40:40.399: EIGRP-IPv4(100): 10.1.41.1 metric 81920/5120
*Mar 18 12:40:40.399: EIGRP-IPv4(100): 10.1.43.3 metric 85760/8960 found Dmin is 56320
*Mar 18 12:40:40.399: DUAL: AS(100) RT installed 192.168.10.0/24 via 10.1.42.2
*Mar 18 12:40:40.399: DUAL: AS(100) Send update about 192.168.10.0/24. Reason: metric chg on tid 0

P 192.168.10.0/24, 1 successors, FD is 56320
via 10.1.42.2 (56320/5120), FastEthernet1/0
via 10.1.41.1 (81920/5120), FastEthernet0/0
via 10.1.43.3 (85760/8960), FastEthernet1/1

Ignore the different IP addresses of the neighbors in both topologies, everything else is the same. As you can see both debug outputs are identical, they both say the FD was 7680, and the new one should be 56320 given the new metric calculation. The main difference (and here the main issue) is that the first one did not actually update the FD to the new one, but still shows the old one. The second one did. So that explains why variance of 2 caused both routes to be installed because in reality, both second and third route do indeed satisfy the DC 5120 and 8960 are both less than 56320 and NOT 7680.

[Rob M]

One more update. I ran the same toplogy that I used initially in this post bu with a different code (same one I used in the previous post) and the results again, explain what was happening. Check this out:

EIGRP-IPv4(100): rcvupdate: 155.1.9.0/24 via 155.1.146.1 metric 64000/10240 on tid 0
EIGRP-IPv4(100): Find FS for dest 155.1.9.0/24. FD is 12800, RD is 12800 on tid 0
EIGRP-IPv4(100): 155.1.67.7 metric 12800/5120
EIGRP-IPv4(100): 155.1.146.1 metric 64000/10240 found Dmin is 12800
DUAL: AS(100) RT installed 155.1.9.0/24 via 155.1.67.7


R6(config-subif)#do sh ip eigrp top 155.1.9.0/24
EIGRP-IPv4 Topology Entry for AS(100)/ID(150.1.6.6) for 155.1.9.0/24
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 12800
Descriptor Blocks:
155.1.67.7 (FastEthernet0/0.67), from 155.1.67.7, Send flag is 0x0
Composite metric is (12800/5120), route is Internal
Vector metric:
Minimum bandwidth is 100000 Kbit
Total delay is 500 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2
Originating router is 150.1.9.9
155.1.146.1 (FastEthernet0/0.146), from 155.1.146.1, Send flag is 0x0
Composite metric is (64000/10240), route is Internal
Vector metric:
Minimum bandwidth is 100000 Kbit
Total delay is 2500 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 4

FD is now 12800, Both RDs are now (5120 and 10240) are less than FD and both satisfy the DC. Now variance of 2 should install the second route and it makes perfect sense now.....