cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1439
Views
15
Helpful
14
Replies

Load balancing over 5xECMP ASR903

Peyman Zadmehr
Spotlight
Spotlight

Hi Everybody

I am using an ASR903- RSP3_400. This router has five TenG connections to another router and formed OSPF Adjacency over These five links. Prefixes are advertised from upstream router to ASR900 via these five links. so for each destination, there will be five entries in Routing table...

When the this router wants to send traffic toward destination learnt from upstream router, It only uses four links out of five and the last link is not used.

I am using following IOS version:

show version | include IOS
Cisco IOS XE Software, Version 03.18.01.S - Extended Support Release
Cisco IOS Software, ASR900 Software (PPC_LINUX_IOSD-UNIVERSALK9_NPE-M), Version 15.6(2)S1, RELEASE SOFTWARE (fc1)
Cisco IOS-XE software, Copyright (c) 2005-2016 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are

And this is my OSPF configuration:

router ospf 1
router-id 10.234.100.103
ispf
area 0 authentication message-digest
redistribute connected metric-type 1 subnets route-map Connected--->OSPF1
redistribute static metric-type 1 subnets route-map STATIC--->OSPF1
passive-interface default
no passive-interface TenGigabitEthernet0/1/0
no passive-interface TenGigabitEthernet0/1/1
no passive-interface TenGigabitEthernet0/2/0
no passive-interface TenGigabitEthernet0/2/1
no passive-interface TenGigabitEthernet0/3/0
network 10.234.100.103 0.0.0.0 area 0
network 10.234.196.76 0.0.0.3 area 0
network 10.234.196.80 0.0.0.3 area 0
network 10.234.196.84 0.0.0.3 area 0
network 10.234.196.88 0.0.0.3 area 0
network 10.234.196.92 0.0.0.3 area 0
maximum-paths 8
!

I also enable this command on the box and reloaded it:


show running-config | section platform
platform loadbalance max-paths 8

show ip cef details also shows all five links are used in CEF table but interface TenGigabitEthernet0/3/0 is not used at all and only 0/1/0,0/2/0,0/1/1,0/2/1 are utilized.

What is the root cause of this issue?

Would you please help me out?

Thank you

1 Accepted Solution

Accepted Solutions

Hello,

 

I found the bug below:

 

Max of ECMP path to 8 for RSP3C
CSCve16000
Description
Symptom:
RSP3 platform supports only for 4 paths for ECMP load-balancing in hardware.

Conditions:
Only happens when more that 4 ECMP paths exist and IGP (OSPF/ISIS) maximum-paths value is configured to be greater than 4.

Workaround:
Configure IGP (OSPF/ISIS) maximum-paths value to be 4 or less.

Further Problem Description:
None

 

Product:

Cisco ASR 900 Series Aggregation Services Routers
Cisco Network Convergence System 4200 Series

 

Known Affected Releases: (1)
15.6(2)SP

 

Known Fixed Releases: (3)
Fuji-16.8.1
16.8.1
16.8(0.23)

View solution in original post

14 Replies 14

Hello,

 

are all 5 routes installed in the routing table (sh ip route) ?

Dear Georg

Yes, For example:

show ip cef 10.2.0.8/30 detail
10.2.0.8/30, epoch 2, per-destination sharing
nexthop 10.234.196.77 TenGigabitEthernet0/1/0
nexthop 10.234.196.81 TenGigabitEthernet0/2/0
nexthop 10.234.196.85 TenGigabitEthernet0/1/1
nexthop 10.234.196.89 TenGigabitEthernet0/2/1
nexthop 10.234.196.93 TenGigabitEthernet0/3/0

This entry is not used

show ip route 10.2.0.8
Routing entry for 10.2.0.8/30
Known via "ospf 1", distance 110, metric 100058, type extern 1
Last update from 10.234.196.93 on TenGigabitEthernet0/3/0, 01:34:28 ago
Routing Descriptor Blocks:
10.234.196.93, from 10.199.233.7, 01:34:28 ago, via TenGigabitEthernet0/3/0
Route metric is 100058, traffic share count is 1
10.234.196.89, from 10.199.233.7, 01:34:28 ago, via TenGigabitEthernet0/2/1
Route metric is 100058, traffic share count is 1
10.234.196.85, from 10.199.233.7, 01:34:28 ago, via TenGigabitEthernet0/1/1
Route metric is 100058, traffic share count is 1
10.234.196.81, from 10.199.233.7, 01:34:28 ago, via TenGigabitEthernet0/2/0
Route metric is 100058, traffic share count is 1
* 10.234.196.77, from 10.199.233.7, 01:34:28 ago, via TenGigabitEthernet0/1/0
Route metric is 100058, traffic share count is 1

 

I wonder is this related to number of the link. I mean Do we need power of two number of the links here?

Thank you

Peyman

 

Hi 

Yes I sent the info before(Start of the thread)

One question here:

I am going to get The mentioned version, Will I encounter Smart Licensing Here? Or Can I disable it and making the device continue working....

Thank you

Hello,

 

smart licensing is mandatory starting with 16.9.1, if you upgrade to 16.8.1, you should be fine without.

Thank you for the information

I am going to test the IOS.. I hope I wont encounter any other bugs. I say this based on experience..

show ip cef interface 0/3/0
check if the per-packet it enable ?

Hi

Show ip cef interface Ten0/3/0 does not contain this information. it just shows the route which are reachable via this interface.

The output is same for all five interfaces.

Thank you

 

show cef interface x

please try this and send me the output.

Hi

I attached the requested log. The "Not-Utilized" Interface is 0/3/0

Thank you

Hi 

I also included the show commands which show information about IP addresses and Utilization

Thanks

Hello,

 

I found the bug below:

 

Max of ECMP path to 8 for RSP3C
CSCve16000
Description
Symptom:
RSP3 platform supports only for 4 paths for ECMP load-balancing in hardware.

Conditions:
Only happens when more that 4 ECMP paths exist and IGP (OSPF/ISIS) maximum-paths value is configured to be greater than 4.

Workaround:
Configure IGP (OSPF/ISIS) maximum-paths value to be 4 or less.

Further Problem Description:
None

 

Product:

Cisco ASR 900 Series Aggregation Services Routers
Cisco Network Convergence System 4200 Series

 

Known Affected Releases: (1)
15.6(2)SP

 

Known Fixed Releases: (3)
Fuji-16.8.1
16.8.1
16.8(0.23)

Thanks a lot, can you check other interface with 
show cef interface "x"?
I see that this is bug but let be sure.

Hi this is the requested output on "not using" Interface

show cef interface tenGigabitEthernet 0/3/0
TenGigabitEthernet0/3/0 is up (if_number 23)
Corresponding hwidb fast_if_number 23
Corresponding hwidb firstsw->if_number 23
Internet address is 10.234.196.94/30
ICMP redirects are always sent
IP unicast RPF check is disabled
IP policy routing is disabled
BGP based policy accounting on input is disabled
BGP based policy accounting on output is disabled
Hardware idb is TenGigabitEthernet0/3/0
Fast switching type 1, interface type 89
IP CEF switching enabled
IP CEF switching turbo vector
IP Null turbo vector
IP prefix lookup IPv4 mtrie 8-8-8-8 optimized
Input fast flags 0x0, Output fast flags 0x0
ifindex 22(22)
Slot 0 (0) Slot unit 192 VC -1
IP MTU 1500

 

This is the output of "using" Interface:

 

show cef interface tenGigabitEthernet 0/2/0
TenGigabitEthernet0/2/0 is up (if_number 15)
Corresponding hwidb fast_if_number 15
Corresponding hwidb firstsw->if_number 15
Internet address is 10.234.196.82/30
ICMP redirects are always sent
IP unicast RPF check is disabled
IP policy routing is disabled
BGP based policy accounting on input is disabled
BGP based policy accounting on output is disabled
Hardware idb is TenGigabitEthernet0/2/0
Fast switching type 1, interface type 89
IP CEF switching enabled
IP CEF switching turbo vector
IP Null turbo vector
IP prefix lookup IPv4 mtrie 8-8-8-8 optimized
Input fast flags 0x0, Output fast flags 0x0
ifindex 14(14)
Slot 0 (0) Slot unit 128 VC -1
IP MTU 1500

 

Apart IP address and IfID they are seemed identical to me. What output line must show the problem?

Thank you

This is a sample of RT:

O E1 10.2.0.0/30
[110/100058] via 10.234.196.93, 1d00h, TenGigabitEthernet0/3/0
[110/100058] via 10.234.196.89, 23:55:26, TenGigabitEthernet0/2/1
[110/100058] via 10.234.196.85, 1d00h, TenGigabitEthernet0/1/1
[110/100058] via 10.234.196.81, 1d00h, TenGigabitEthernet0/2/0
[110/100058] via 10.234.196.77, 21:34:05, TenGigabitEthernet0/1/0
O E1 10.2.0.4/30
[110/100058] via 10.234.196.93, 1d00h, TenGigabitEthernet0/3/0
[110/100058] via 10.234.196.89, 23:55:26, TenGigabitEthernet0/2/1
[110/100058] via 10.234.196.85, 1d00h, TenGigabitEthernet0/1/1
[110/100058] via 10.234.196.81, 1d00h, TenGigabitEthernet0/2/0
[110/100058] via 10.234.196.77, 21:34:05, TenGigabitEthernet0/1/0

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco