cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
configure & troubleshoot anyconnect

BNG deployment scale guidelines on ASR9000

5772
Views
5
Helpful
32
Comments

Update (March 2018)

This document was written at the time when 5.3.4 was about to be released. Hence the SW recommendation for BNG deployment has changed. Please refer to https://supportforums.cisco.com/t5/service-providers-documents/ios-xr-release-strategy-and-deployment-recommendation/ta-p/3165422.

The supported BNG scale specified in this document stands for all 32-bit (aka Classic) IOS XR releases, regardless of the line card generation (Typhoon, Tomahawk) or RSP generation (RSP440-SE, RSP880-SE) because the limit is imposed by the max memory that a process can address. Higher BNG scale requires the 64-bit (aka Enhanced) IOS XR, which also implies Tomahawk and RSP880-SE.


Introduction

The purpose of this document is to share the information about the BNG scale tested within the System Integration Testing (SIT) cycle of IOS XR releases 5.2.4 to 5.3.4 on Cisco ASR9000 platform. This is not the only profile in which BNG was tested at Cisco.

The scalability maximums for individual features may be higher than what is specified in this profile. The BNG SIT profile is provides for a multi-dimensional view of how features perform and scale together. The results in this document closely tie with the setup utilised for this profile. The results may vary depending upon the deployment and optimizations tuned for the convergence.

IOS XR release 5.2.4 is the suggested release for deploying BNG on the ASR9000 platform (until the XR release 5.3.3 is available).

IOS XR release 5.3.4 was posted on October 7th 2016. The XR release 5.3.4 is the Cisco suggested release for all BNG deployments. If your deployment requires features not supported yet in 5.3.4, please reach out to your account  team for recommendation. The scale numbers for 5.3.4 are identical to those from 5.2.4. The difference between these releases is in the supported features and level of BNG hardening cycles that we at Cisco have completed.

Note: starting with XR release 5.3.4 you can use RSP880 in BNG deployments on asr9k. The scale remains the same because the scale is dictated by line card capabilities. The supported BNG scale will increase with the introduction of BNG support on Tomahawk generation line cards.


Feature And Scale Under Test

 

BNG With MSE (RP based subscribers)

Scale parameter Scale
Access ports bundle GigE/TenGigE
Number of members in a bundle 2
uplink ports TenGige
Scale - PPP session PTA 64k (Without any Other BNG Sessions)
Scale - PPP Sessions L2TP 64k (Without any Other BNG Sessions)
Scale - IPv4 Sessions 64k (Without any Other BNG Sessions)
Scale - PPP v6 session 64k (Without any Other BNG Sessions)
Scale - IP v6 Session 64k (Without any Other BNG Sessions)
Scale - Dual-Stack Session 32k (Without any Other BNG Sessions)
Sessions Per Port 8k/32k
Sessions Per NPU 32k
Sessions Per Line Card 64k
Number of classes per QoS policy 8
Total number of unique QoS policies 50
Number of sub-interfaces 2k
Number LI taps 100
Number CPLs 50
Session QoS Ingress: Policing
Egress: Hierarchical QoS; Model F with 3 queues per session
Number IPv4 ACLs 50
Number ACEs per IPv4 ACL 100
Number IPv6 ACLs 50
Number ACEs per IPv6 ACL 100
Number IGP routes 10k
Number VRFs for subscribers 100
Number BGP routes 30k
Number VPLS 3000
Number VPWS 2000
Number MAC addresses 2000
Number of sessions with multicast receivers 8000
AAA Accounting Start-Stop every 10min
Interim
Service Accounting 64k + 2 services
BNG over Satellite 32k
CPS rate 80 calls per second
CoA 40 requests per second
DHCP (proxy or server) 60 requests per second

 

BNG Only (RP based subscribers)

Scale parameter Scale
Access ports bundle GigE/TenGigE
Number of members in a bundle 2
Scale - PPP session PTA 128k (Without any Other BNG Sessions)
Scale - PPP Sessions L2TP 64k (Without any Other BNG Sessions)
Scale - IPv4 Sessions 128k (Without any Other BNG Sessions)
Scale - PPP v6 session 64k (Without any Other BNG Sessions)
Scale - IP v6 Session 64k (Without any Other BNG Sessions)
DS PPPoE 64k (Without any Other BNG Sessions)
Static Session 8k (Without any Other BNG Sessions)
Routed Subscribers 32k (Without any Other BNG Sessions)
Service Accounting 64K with 2 Services , 24 with 3 Services
Routed Pkt Triggered subscriber IPv4/IPv6 64k

 

CPU Utilisation (RP based subscribers; 128k IPoEv4)

HW Type A9K-RSP-440-SE MOD80-SE
Role RSP Subscriber facing line card
Major features Total of 128k RP based IPoEv4 or PPPoE subscribers Data plane
CPU steady state 14% 10-14%
CPU peak 60-70% 50-60%
Top processes lpts_pa prm_server_ty
iedged qos_ma_ea
ppp_ma/dhcpd prm_ssmh
ifmgr netio

 

CPU Utilisation (RP based subscribers; 64k Dual-Stack)

HW Type A9K-RSP-440-SE MOD80-SE
Role RSP Subscriber facing line card
Major features Total of 64k RP based IPoE or PPPoE subscribers Data plane
CPU steady state 14% 10-14%
CPU peak 40-50% 30-40%
Top processes lpts_pa prm_server_ty
iedged qos_ma_ea
ppp_ma/dhcpd prm_ssmh/ipv6_nd
ifmgr/ipv6_nd netio

 

BNG Only (LC based subscribers)

Scale parameter Scale
Access ports GigE/TenGigE
Uplink ports GigE/TenGigE
Scale - PPP session PTA 256k (Without any Other BNG Sessions)
Scale - IPv4 Sessions 256k (Without any Other BNG Sessions)
Scale - PPP v6 session 128k (Without any Other BNG Sessions)
Scale - IP v6 Session 128k (Without any Other BNG Sessions)
DS PPPoE 128k (Without any Other BNG Sessions)
Static Session 8k (Without any Other BNG Sessions)

 

CPU Utilisation (LC based subscribers; 256k IPv4 Sessions)

HW Type A9K-RSP-440-SE MOD80-SE
Role RSP Subscriber facing line card
Major features LC based IPoE or PPPoE Subscribers (256k) LC based Subscribers, Control and Data plane (64K per LC)
CPU steady state 10% 10-14%
CPU peak 10% 80-90%
Top processes dhcpv6/dhcp prm_server_ty
iedged/ipv4_rib dhcpv6/dhcp
ppp_ma/dhcpd iedged
  ppp_ma
  qos_ma_ea

 

PWHE With BNG Only (RP based subscribers)

Scale parameter Scale
Generic interafce list members Bundle, GigE/TenGigE
Number of members in a bundle 2
Scale - PPP session PTA 128k (Without any Other BNG Sessions)
Scale - IPv4 Sessions 128k (Without any Other BNG Sessions)
PW Scale 1k

 

References

Comments

Thanks you for this summary Aleksandar, really appreciated. 

What numbers should we look at for the ASR9001? You're hinting that 5.3.3 will be the next recommended BNG release, is that so? Are there any BNG scale improvements committed for 5.3.3 that would change these numbers?

Cisco Employee

Hi Fredrik,

I'm glad you found the document useful.

5.3.3 offers more BNG features and stability, but it doesn't offer significant scale or performance improvements. That will come with BNG support on the latest generation of Ethernet linecards (Tomahawk), planned for later this year.

Regards,

Aleksandar

Beginner

Hi Aleksandar,

I was wondering if you coud tell me the BNG deployment guidelines on ASR9000 with release 6.0.1.

We are thinking of evolving BNG to migrate from bundle RP based subscribers to PWHE RP based subscriber. Remaining PPP session PTA model.

Best regards

Javier

Cisco Employee

Hi Javier,

scale numbers have remained the same. We have been focusing on introducing many great new features. Higher scale per LC and per system will come with BNG support on Tomahawk line cards and RSP880.

regards,

Aleksandar

Beginner

Dears,

thank you for he helpful guide,

one question if using A9K-36X10G-AIP-SE Line card  which contains 6 NPU's(not sure).

would it mean that it can support 256k sessions ? with or without QOS ?

Regards

Sami

Cisco Employee

hi Sami,

there is a limit 64k of sessions per LC, irrespective of the number of NPs. This scale number will be higher on Tomahawk line cards. On Typhoon it will remain as is.

regards,

Aleksandar

Beginner

thank you Aleksandar.

Beginner

Hi Alex,.. I wonder if you could explain a bit more this parameter you showed in the table: 

Number of sub-interfaces = 2K

I meant in a bundle with two members only 2 thousand L3 subs can be configured?

I look forward to hearing from you asap

Best regards

Javier

Cisco Employee

hi Javier,

that table is not showing the maximum uni-dimensional scale. The table shows what have we tested in this multi-dimensional scale scenario. If your deployment is within the scale limits shown in this table, you should be good. I your deployment exceeds some scale parameters, you need to pay more attention to negative testing in your scenario (link flaps, etc.).

I hope this clarifies.

/Aleksandar

Beginner

Alex thanks for you reply..

I will try to commit to this limits.

Two things are for me a bit weird, you know, that is 2,000 subinterfaces L3 and only 50 unique QoS policies. For what I understand under this scenario you recommend not to have on a bundle more than those figures.

Best regards,

Javier

Cisco Employee

hi Javier,

these are 2000 BNG access sub-interfaces. On each of those you can have multiple BNG subscriber sub-interfaces, up to the max scale per NP (which is 32k/NP).

What is your deployment scenario? I would like to understand why 2000 BNG access sub-interfaces is not sufficient. Even in that case the 2000 is not a hard limit.

The same stands for the QoS policy. These are configred 50 unique policies, but each had multiple instances applied.

regards,

Aleksandar

Cisco Employee

Thanks Aleks

Beginner

Thanks Ale, is clear for us the BNG scale up to the max scale per NP (which is 32k/NP).

Let me try to explain in some words how our network is working according issus of design.

There are mpls rings in the access connecting two or three uPE, the rings end on the MSE ASR9010 that has the BNG and PE function, for as multiservice bng. To separate those service we use differents pseudowires. The termination for HSI's pw is at the level 2 side of the hairpnning, at the other side of hairpinning we have the bba group in  bundle interface as our design is to protect pppoe session on the RPS. Now we are testing 6.0.1 to deploy pw-he and remove all the hairpinnings. A matter of design on this will be the amount of session the int pw-ether can support. So far we have 25.000 sessions for hairpinning to squeeze it at 80% more or less. 

As well, we have some frontiers with a L2 Lanswitch which has many conections to PTN transport network where you can see some uPE at the other end. So at the ASR9K we have a 20G eth bundle (10G active+10G stand by)  as you can see it on the draw, with many subinterfaces, some of them are level 2 and others are level 3 with IP address. So my cuestion was addressed to know how many L3 subinterfaces can support because we could not test it yet.

That's all. Thanks for your answers.

Regards,

Javier

PD: we need xr6.0.1 as it supports LAC

Cisco Employee

The bundle EFP scale (l2transport bundle sub-interfaces) is 64k/LC and 128k/system.

The bundle L3 subinterface hard limit is 3800 per NP.

All of the above are uni-dimensional scale numbers.

If in MSE you are planning to go above the scale documented in the table at the top of the page, please make sure you test the desired scale in the lab.

Community Member

Hi Xander,

This is a great doc, very helpful.

We have ASR9001 BNGs with RP based subs. Are the RP based subs here similar numbers to what we'd expect on the ASR9001, or does the RP tested here perform differently?

We don't have queues on all our subs - maybe 50%, so QoS chunks are only consumed for some customers. I presume the 32k limit that is often stated for the ASR9001 is because of QoS chunks, though I don't have anything to back that up.

We currently have a redundant BNG that has no subscribers on it, but one of our active BNGs is at 31k subs - none of these subs have shapers, so no QoS resources are used. If we can go past 32k unshaped subs on this BNG it would mean we don't have to alter our resiliency model in the interim before adding more BNG capacity.

About 50% of the subs on this BNG that's about at capacity don't even have RADIUS accounting, if that makes much difference.

CreatePlease to create content
Content for Community-Ad
June's Community Spotlight Awards