ASR9000 route scale

[ez9]

Hello,

I have an ASR9006 with two RSP440-SE RSP and I am getting errors like:

%ROUTING-BGP-2-ERR_MEM_ALLOC : [23] : Failed to allocate memory for attribute entry : pkg/bin/bgp : (PID=12845322) : -Traceback= 11d022e8 11d0907b 11d2e793 11d2f320 11d30641 11d31bb9 11d32749 4311583 4312300 ece6d15 ece74b2 ece7de8 ed3b48b ed3b570 a1f053d a1ee573

This is the output of the show bgp scale:

sh bgp scale
VRF: default
Neighbors Configured: 32 Established: 30

Address-Family Prefixes Paths PathElem Prefix Path PathElem
Memory Memory Memory
IPv4 Unicast 941307 3620546 2745915 82.59MB 220.98MB 170.22MB
IPv6 Unicast 194061 638330 507458 19.25MB 38.96MB 31.46MB
IPv4 Flowspec 3 3 3 12.26KB 192.00 189.00
------------------------------------------------------------------------------
Total 1135371 4258879 3253376 101.85MB 259.94MB 201.67MB

Total VRFs Configured: 0

sh processes memory detail
Tue Jul 9 18:10:40.248 EEST
JID Text Data Stack Dynamic Dyn-Limit Shm-Tot Phy-Tot Process
------ ---------- ---------- ---------- ---------- ---------- ---------- ---------- -------
1087 1M 10M 612K 1166M 1629M 290M 1177M bgp

As the RSP has about 6gb free is it possible to allocate more memory to the BGP process?

[Joseph W. Doherty]

BTW, sometimes being unable to allocate memory isn't from lack of free memory but due to memory fragmentation.  Cannot say that's the issue here.

[Giuseppe Larosa]

Hello @ez9 ,

>> %ROUTING-BGP-2-ERR_MEM_ALLOC : [23] : Failed to allocate memory for attribute entry : pkg/bin/bgp : (PID=12845322) : -Traceback= 11d022e8 11d0907b 11d2e793 11d2f320 11d30641 11d31bb9 11d32749 4311583 4312300 ece6d15 ece74b2 ece7de8 ed3b48b ed3b570 a1f053d a1ee573

The router is complaining of a BGP attribute in a BGP advertisement received that is too big, this is not a problem of lack of memory as you have noted having 6 GB of free RAM.

Or it might be a SW bug.

Hope to help

Giuseppe

[MHM Cisco World]

https://bst.cisco.com/quickview/bug/CSCvp12914

This is bug I think check link 

MHM

[Giuseppe Larosa]

Hello @MHM Cisco World ,

I don't think this bug applies to this case as the error messages reported in the bug are quite different.

I tried to make a search but I didn't find any other possible bug using the error message as the key.

Hope to help

Giuseppe

[ez9]

Hello,

The version is the 6.9.2. Known fixed release is the 6.8.2. I suppose that I am not hit this bug.

[ez9]

Some other logs are the following:

bgp[1087]: %ROUTING-BGP-3-NOMEM_RESET : [23] : Failed to allocate memory for net in comm library, resetting neighbor : pkg/bin/bgp : (PID=13336842) : -Traceback= 11d2f84c 11d3045b 11d31bb9 11d32749 4311583 4312300 ece6d15 ece74b2 ece7de8 ed3b48b ed3b570 a1f053d a1ee573 10bd8a1e 430ac3c a280780
bgp[1087]: %ROUTING-BGP-3-DLL_ERROR : [23] : Failed to Attribute Decode bgp comm library DLL: Attribute is not found or could not be created
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)
bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)

[Giuseppe Larosa]

Hello @ez9 ,

what kind of device is the BGP neighbor ?

It may be an interoperability issue

>> Failed to Attribute Decode bgp comm library DLL: Attribute is not found or could not be created

>> bgp[1087]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x.x on standby RP due to Peer closing down the session (VRF: default)

This means the peer has just teared down the session and the NSR feature is disabled for this neighbor on the standby route processor.

Hope to help

Giuseppe

[ez9]

The BGP neighbors are Internet exchanges members and ISPs (Cogent, Lumen, GTT etc). The BGP sessions aren't going down. Some of the neighbors are UP for weeks. The messages referring to the standby RP, so I don't know if the problem is with the standby RSP or with a specific configuration. 

[Richard Burts]

In the OP I am focusing on "-Traceback= 11d022e8 11d0907b 11d2e793". A traceback is always an indication of a software problem. I can not interpret the traceback (only Cisco TAC has the ability to intepret it). If you have a support contract for this device I suggest opening a case with Cisco TAC.