topic Re: ASR1001-X BGP soft-reconfiguration inbound question in Routing and SD-WAN

ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 13:49:58 GMT

Hello everyone,

I have a big doubt and I hope you can help me, thanks in advance.

Question: Why is the memory consumed by adj-rib-in not greater than or equal to the memory consumed by local-rib?

Context:

I was reviewing the configuration of my Internet Edge routers and in the BGP configuration with the ISP we have applied the following command:

neighbor x.x.x.x.x soft-reconfiguration inbound

The ISP sends me the full BGP routing table and I am aware that the command 'soft-reconfiguration inboud' should be deleted.

But I don't understand why the memory used by the Adj-rib-in is not the same as the local-rib?

show ip bgp neighbors x.x.x.x.x
*Output omitted for brevity*
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 7 818518 (Consumes 111316272 bytes)
Prefixes Total: 7 4120075
Implicit Withdraw: 0 3039791
Explicit Withdraw: 0 261766
Used as bestpath: n/a 777318
Used as multipath: n/a 0
Used as secondary: n/a 0
Saved (soft-reconfig): n/a 1 (Consumes 136 bytes)

show ip bgp summary
*Output omitted for brevity*
1 received paths for inbound soft reconfiguration

PID: ASR1001-X Version 16.9.5

show ip bgp neighbors x.x.x.x policy
Neighbor: x.x.x.x, Address-Family: IPv4 Unicast
Locally configured policies:
prefix-list ISPtoME in
prefix-list ANNONCE_ME_ISP out
filter-list 7 out
soft-reconfiguration inbound
remove-private-as

Re: ASR1001-X BGP soft-reconfiguration inbound question

MHM Cisco World — Sat, 06 Feb 2021 03:38:05 GMT

any update form one peer will be save in
Adj-RIB-IN
after that it will filter with INBOUND policy then it will be add to
Loc-RIB

at end any other routes are discard from Adj-RIB-IN

with
soft-reconfig inbound

the routes are copy from Adj-RIB-IN,
when we do
clear ip bgp soft in

the router only refilter the route again with new INBOUND policy if found.

Re: ASR1001-X BGP soft-reconfiguration inbound question

paul driver — Sat, 06 Feb 2021 06:23:55 GMT

Hello
I would suggest to remove soft-reconfiguration as it’s an old way to refresh your rtrs bgp policy after a change against all your neighbours advertised prefixes they send to you.

Using soft-reconfiguration the bgp rtr has to store each neighbouring bgp peers unmodified routes so they can be checked against your new policy before they are actually placed the bgp route table (soft reset) and this takes up rtr resource such as memory and if you had multiple bgp peerings will large rib tables that’s a lot or resource being used.

Most rtrs should support route-refresh which dynamically requests updated route information from it peers (dynamic soft-reset) thus negates the need to store any peer information locally.

To check if you bgp rtr and it peers support route-refresh

show ip bgp neighbours | in refresh

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 06:34:36 GMT

Hello Paul.

Thank you very much for your reply.

I edited the question a bit to be clearer.

I am aware that the command should be suppressed, what I don't understand is why the memory used by the adj-rib-in is only a few bytes since I receive the whole internet table.

Because as I see in my router, having this command configured, does not affect the router's memory... which is normally not true.

Regards,

Andrés

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 06:38:11 GMT

Hello,

Thank you very much for taking the time to answer my question.

What I am looking to understand is why the memory consumed by local-rib is not the same as adj-rib-in.

Normally with this command, I should store twice the prefixes announced by my peer, but according to my router, this is not the case.

Prefixes Current: 7 818518 (Consumes 111316272 bytes)
....

Saved (soft-reconfig): n/a 1 (Consumes 136 bytes)

Re: ASR1001-X BGP soft-reconfiguration inbound question

Georg Pauwen — Sat, 06 Feb 2021 08:12:03 GMT

Hello,

I don't really have a direct answer, but actually, that is how I always understood it as well: soft reconfiguration actually doubles the memory used. As far as I recall, there are two parts to the adj-RIB, the adj-RIB-in, and the adj-RIB-out. The 'in' contains all the unedited information, while the 'out' contains the information after outbound polciies are applied, Not sure if your specific router has those policies configured ? Either way, only the adj-RIB-in table is stored in memory for each neighbor, the adj-RIB-out table not.

So I wonder what the output:

--> Saved (soft-reconfig): n/a 1 (Consumes 136 bytes)

is actually referring to.

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 08:21:31 GMT

Hello Georg,

Thank you for your response.

I don't understand the behavior on these routers (ASR1K)

On another router, a catalyst 68XX, the situation is different and the behavior is as expected.

 Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 58 187 (Consumes 117768 bytes)
Prefixes Total: 489 24837
Implicit Withdraw: 405 12267
Explicit Withdraw: 26 12382
Used as bestpath: n/a 187
Used as multipath: n/a 0
Used as secondary: n/a 0
Saved (soft-reconfig): n/a 1215 (Consumes 102060 bytes)

In other words, I accept 187 prefixes but receive 1215 prefixes.
I don't know why in Internet Edge routers I don't have the same behavior.

I don't want to be left with that doubt 😞

Regards,

Andrés

Re: ASR1001-X BGP soft-reconfiguration inbound question

paul driver — Sat, 06 Feb 2021 19:12:09 GMT

Hello

Thank you for the clarification

@TelesEC wrote:

Question: Why is the memory consumed by adj-rib-in not greater than or equal to the memory consumed by local-rib?

My understanding here is that the local rib prefixes are those that have been processed against you inbound bgp filter policy and are deemed viable to be entered into the bgp table any other prefixes left the adj-rib-in are left until they become resolvable or they are withdrawn ,However when using soft-inbound configuration a copy of that adj-rib-in is also separately stored.

Re: ASR1001-X BGP soft-reconfiguration inbound question

MHM Cisco World — Sat, 06 Feb 2021 13:18:29 GMT

If i am right this is edge router receive few or only one route which is defualt, but it adverise many route toward SP.

Keep in mind Loc-RIB have the IN and OUT route.

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 13:47:09 GMT

No, my router receives the full BGP routing table

You can see it in the following output, 818528 prefixes:

show ip bgp neighbors x.x.x.x.x
*Output omitted for brevity*
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 7 818518 (Consumes 111316272 bytes)
....
Saved (soft-reconfig): n/a 1 (Consumes 136 bytes)

Section "Saved (soft-reconfig):" should have about the same number of routes.

I don't understand why the difference.
When I use the command.. show ip bgp neig x.x.x.x.x received-routes... I get the whole internet table.

Andrés

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 13:51:26 GMT

From what I understand, when configuring the 'soft-reconfiguration inboud' command.
A memory space is dedicated to store the prefixes before applying any policy.

In this case the ISP announces me the full BGP table... so my router, inefficiently, should store twice the entire Internet table, once in the local-rib and the other in the adj-rib-in... but according to the output this is not happening.

Prefixes Current: 7 818518 (Consumes 111316272 bytes)
....

Saved (soft-reconfig): n/a 1 (Consumes 136 bytes)

For example on another router, a C6XXX, I get this behavior. I receive 1215 prefixes and only accept 187.

 Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 58 187 (Consumes 117768 bytes)
...
Saved (soft-reconfig): n/a 1215 (Consumes 102060 bytes)

Re: ASR1001-X BGP soft-reconfiguration inbound question

MHM Cisco World — Sat, 06 Feb 2021 15:15:02 GMT

sh ip bgp neighbors x.x.x.x | s Neighbor capabilities

please can you show the output for both ASR1000 and other C6xxx

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sat, 06 Feb 2021 16:23:06 GMT

Hello,

Here you have the requested information.
For your information, the ASR is the Internet Edge, the C8XX is a PE.

For ASR1001-X

 Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Graceful Restart Capability: received
Remote Restart timer is 120 seconds
Address families advertised by peer:
IPv4 Unicast (was preserved, Peer capable of Enhanced GR)
Long-lived Graceful Restart Capability:
Enhanced Refresh Capability: advertised
Multisession Capability: 
Stateful switchover support enabled: NO for session 1

A few lines after this section we have.

Inbound soft reconfiguration allowed

For C68XX

 Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability: 
Stateful switchover support enabled: NO for session 1
....
Inbound soft reconfiguration allowed

Thanks

Re: ASR1001-X BGP soft-reconfiguration inbound question

MHM Cisco World — Sun, 07 Feb 2021 18:06:09 GMT

Can you give some time,
this interest,
What I suspect ?
there is
A-soft-reconfig "save a route in DB and re process it when do clear ip bgp soft"
B-route refresh "automatically ask the neighbor a route"

BUT there is
Soft Reset Enhance <- I know it new for me also

this reduce the memory need for save a route in DB but the full technology I don't get until now so I will do more search this week and I will update you.

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sun, 07 Feb 2021 18:34:07 GMT

Hello,

Thank you very much for your time and investigations.... I still can't find a logical explanation.

On the other hand, and as far as I know... If I have configured the feature soft-reconfiguration, the BGP route refresh capability is not used, even though the capability is negotiated.

Andrés

Re: ASR1001-X BGP soft-reconfiguration inbound question

paul driver — Sun, 07 Feb 2021 19:22:02 GMT

Hello
But I don't understand why the memory used by the Adj-rib-in is not the same as the local-rib?

Not necessarily the case - in fact BGP does mention its not by protocol design to specifically maintain 3 separate rib tables even those 3 rib tables are stated it could be just the one rib table is used with pointers towards the specific prefixes.

In any case like i suggested initially if your rtrs support route fresh then utilise that feature and remove soft-inbound from the bgp stanzas

Re: ASR1001-X BGP soft-reconfiguration inbound question

paul driver — Sun, 07 Feb 2021 19:26:42 GMT

FYI - I managed to find the rfc -

rfc4271 - page 10

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Sun, 07 Feb 2021 19:50:48 GMT

Thank you for your reply Paul.

Yes, I will delete this functionality. I just wanted to understand why the behavior was not as expected on these routers.

Normally the soft-reconfiguration functionality should keep in memory all the prefixes before applying the policy... The use of this functionality becomes meaningless with the advent of Route refresh.

Re: ASR1001-X BGP soft-reconfiguration inbound question

Peter Paluch — Mon, 08 Feb 2021 09:20:50 GMT

Hello Paul, hello Andrés,

Please allow me to join.

Andrés, I think that the behavior you see - having only 1 route in the soft-reconfig database instead of all those 800K routes - could be the result of configuring the soft-reconfig only after the BGP peering already came up. Are you sure it has been always configured for that neighbor, particularly before the peering came up and the neighbor sent us all the routes?

The soft reconfiguration can only store the routes it newly receives from a neighbor, and so if it is activated after the peering has come up and the routes were already advertised, it has nothing to remember. It can only store aside routes that are advertised later.

Could this be the case?

Best regards,
Peter

Re: ASR1001-X BGP soft-reconfiguration inbound question

TelesEC — Mon, 08 Feb 2021 09:42:18 GMT

Hello Peter,

Thank you very much for your response.
What you say makes a lot of sense, on the other hand, I have had several problems on this link so the BGP session has been re-established several times with the configuration of the feature (soft-reconfiguration inbound) already applied.

If I have the feature configured and if I lose the BGP session... when it is established again, I should have the 800K in both tables, right?

Thanks,

Andrés