Cisco ASR 903 routing issue

[Joe De Lange]

Hi

I have a very interresting routing issue with the Cisco ASR 903 Metro Services router.

I have iBGP setup between 2 internal routers and eBGP between 2 providers.

Provider 1 only sends a default route to me via eBGP.

Provider 2 sends me the full internet routing table.

Provider 2 is the preffered provider for all traffic and Provider 1 is the backup.

When I try to reach 134.213.23.170 from within the organization, the traffic routes to the primary router ( established via HSRP), but it gets routed between the two ibgp neighbors.

When doing a trace to the same address from the Provider facing interface ( Provider 1 or 2) the traffic routes to the desired destination.

When configuring a static route to this address, it does not take affect.

The route is installed on the Primary router:

show ip bgp 134.213.23.170
BGP routing table entry for 134.213.0.0/16, version 946980
Paths: (2 available, best #1, table default, not advertised to EBGP peer)
Multipath: eBGP
Advertised to update-groups:
3
Refresh Epoch 1
36994 1273 1299 12200 15395
41.21.228.219 from 41.21.228.219 (41.0.0.145)
Origin IGP, localpref 100, valid, external, best
Community: no-export
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
36994 1273 1299 12200 15395, (received-only)
41.21.228.219 from 41.21.228.219 (41.0.0.145)
Origin IGP, localpref 100, valid, external
Community: 1273:22000 1299:30000 36994:14 36994:52 36994:150 36994:151 36994:156
rx pathid: 0, tx pathid: 0

Current Image: Cisco IOS XE Software, Version 03.16.00.S - Extended Support Release
Cisco IOS Software, ASR900 Software (PPC_LINUX_IOSD-UNIVERSALK9_NPE-M), Version 15.5(3)S, RELEASE SOFTWARE (fc6)

Any ideas?

[Masoud Pourshabanian]

Hello,

Could you please share the result "show IP BGP 134.213.23.170" and "show IP route 134.213.23.170" on both routers on the edge.

What is the purpose of "Multipath: eBGP". Are trying to share the load across two links?

And a quick IP scheme from the edge your network.

[Joe De Lange]

Hi

Please see below

show IP BGP 134.213.23.170
BGP routing table entry for 134.213.0.0/16, version 1868968
Paths: (2 available, best #1, table default, not advertised to EBGP peer)
Multipath: eBGP
Advertised to update-groups:
3
Refresh Epoch 1
36994 3356 12200 15395
41.21.228.219 from 41.21.228.219 (41.0.0.145)
Origin IGP, localpref 100, valid, external, best
Community: no-export
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
36994 3356 12200 15395, (received-only)
41.21.228.219 from 41.21.228.219 (41.0.0.145)
Origin IGP, localpref 100, valid, external
Community: 3356:2 3356:22 3356:100 3356:123 3356:500 3356:2106 36994:10 36994:52 36994:150 36994:151 36994:156 36994:4246 36994:62317
rx pathid: 0, tx pathid: 0

show IP route 134.213.23.170
Routing entry for 134.213.0.0/16
Known via "bgp 327787", distance 20, metric 0
Tag 36994, type external
Last update from 41.21.228.219 2d03h ago
Routing Descriptor Blocks:
* 41.21.228.219, from 41.21.228.219, 2d03h ago
Route metric is 0, traffic share count is 1
AS Hops 4
Route tag 36994
MPLS label: none

[Amit Goyal]

Hi Joe,

you can get the output of "sh ip cef exact-route <src ip> <134.213.23.170>" on HSRP active ASR and check where it is pointing to?

If it is pointing to Provider 2 but still the traffic going to another ASR, then you might need to check with Cisco TAC for any Hardware miss-progarmming.

By the way is it happening for all destination or only for this specific destination?

HTH

-Amit

[Joe De Lange]

Hi Amit

Apologies for only responging now. I have been off sick for a couple of days.

The ip cef output indicates that the traffic is indeed pointing to the correct provider, but the traffic is still flowing between the two routers.

This is the only subnet where I have picked up the issue, but if there is one, there must be others that I am not aware of.

We have - for a temp solution - changed the routing to only receive a default route from both providers, and this indeed made a huge difference. The traffic is flowing to the prefered provider, with the default route tagged with a higher local pref.

What we did pick up in doing this, that when the route-map to tag the traffic is implemented with an additional seq number, it does not take affect, but when we set the local pref in the same seq number where we set the community it took affect.

This one works:

route-map BGP-XXXX-IN permit 10
set local-preference 200
set community no-export

This one doesn't work:

route-map BGP-XXXX-IN permit 10
set community no-export

route-map BGP-XXXX-IN permit 20
set local-preference 200

I am sure that you are correct in suugesting that we get TAC involved, but now we first need to get our primary provider to make adjustments to the advertised routes and send us the major subnets instead of only a default route or the full routing table.

TAC was involved a couple of issues, but cannot run debug commands for the routing, because they are aware the should the bgp table be flushed for some reason, the debugging will bring our router to a stand still.

I will keep this post up-to-date as and when I have more feedback.

Thank you for your assistance.

[Amit Goyal]

Hi Joe,

Thanks for the information.

I believe there are chances of Hardware Miss-programming which got fixed after receiving only default route.

Please rate if my assistance helped.

HTH

-Amit

[Joe De Lange]

Hi

The purpose for the multipath is to load balance, but 1 of the providers has a design issue,meaning that they cannot provide us with the full routing table at this stage.

[Joe De Lange]

Apologies, didnt read properly.

Below is the output for the other router ( Secondary link) that only receives a  default route from provider 1.

show IP BGP 134.213.23.170
BGP routing table entry for 134.213.0.0/16, version 1517516
Paths: (1 available, best #1, table default, not advertised to EBGP peer)
Multipath: eBGP
Not advertised to any peer
Refresh Epoch 1
36994 3356 12200 15395
196.13.169.21 from 196.13.169.21 (196.13.169.254)
Origin IGP, metric 0, localpref 100, valid, internal, best
Community: no-export
rx pathid: 0, tx pathid: 0x0

show IP route 134.213.23.170
Routing entry for 134.213.0.0/16
Known via "bgp 327787", distance 200, metric 0
Tag 36994, type internal
Last update from 196.13.169.21 2d04h ago
Routing Descriptor Blocks:
* 196.13.169.21, from 196.13.169.21, 2d04h ago
Route metric is 0, traffic share count is 1
AS Hops 4
Route tag 36994
MPLS label: none

[Carlos A. Silva]

I apologize if I misunderstood.

At the beginning of this post, you mention receiving a full internet table. I'm curious if you ever got this to work. The numbers that I have for an ASR903 with RSP400 is that they can support a maximum of 200k IPv4 routes. I happened to find this post because I was looking for confirmation.

Hope you can shed some light into this issue.

Thanks!

c.

[Georg Pauwen]

Hello,

on a side note, according to Table 2 of the attached datasheet, the RSP3C-400-S and RSP3C-400-W support a maximum of 128K IPv4 routes...

https://www.cisco.com/c/en/us/products/collateral/routers/asr-903-series-aggregation-services-routers/datasheet-c78-738339.html

[Carlos A. Silva]

That’s what the data sheet says. I have information that says otherwise. Maybe they went with the “safe number” in the DS.