Limit of BNG subscribers in ASR 9000

[caio.szekely]

 Hi,

I need help to validate the total number of BNG subscribers (PPPoE) supported in a ASR-9010 chassis.

In this network scenario, we have one ASR-9010 with the following configuration:

1x A9K-RSP-440-SE
1x A9K-MOD-160-SE
2x A9K-MPA-8x10GE

NOTE: We will associate subscribers to a MOD.

I know the MOD-160 has four Thyphoon NPUs and that each NPU supports 32K subscribers, totaling 128K. Then, we found out that there is a limitation to 64K subscribers, from the MOD-160 connection to the backplane.

We will be upgrading the MOD-160 module to the maximum number of supported subscribers possible.

Searching for another module, I found out the A9K-MOD400-SE.

I noticed that the MOD-400 has two Tomahawk NPUs, which supports 64K subscribers each. Is it correct? So, is it possible to associate 128K BNG subscribers in this MOD?

In this project, we are estimating that each subscriber will use approximately 1.5Mbps of link consumption. So, 128K subscribers demand a throughput of 192Gbps.

Can I use this amount of subscribers with MOD-400-SE and have the throughput required to connect to the backplane?

100% of subscribers' traffic will be forwarded to the VSM modules, to do CGNAT. We will use four VSM modules, because each module supports only 60Gbps of throughput.

Another question, I want to ask in this post about QoS policies.

I understand that it is advisable to use MPAs with 10GE interfaces, because QoS is applied on physical interfaces. But, do I also need to know if anyone has had any experience using QoS policies on 40GE or 100GE interfaces.

Is there an official Cisco document that indicates best practices for using QoS on 10GE interfaces?

Thank you,

Caio Szekely

[xthuijs · Accepted solution]

Hi Caio,

there are several limits that come into play:

32k/npu (this is a typhoon hard limit since due to the interface idb scale)

tomahawk can do 64k/npu in evolved XR

64k/lc this is a soft testing limit bound by the memoyr limitation of the linecard.

typhoon may not have enough memory to go much beyond 64k, and we're testing the limits

tomahawk can do more then 64k per LC

192k/chassis (LC based subs) this is a test limit only in 621.

xr63 and beyond, evolved XR will see 256k and 512k roadmap.

now one crux is that when we state evolved XR, this means that you need a tomahawk and RSP880 chassis. this is not a sales trick, but evolved XR runs 64bit and virtual machines. the typhoon LC is based on PPC and 32 bit. virtualization on ppc is not commercially available, hence typhoon won't run evolved XR, although we prototyped 32<>64 bit interaction.

If you are having a new deployment I would definitely recommend MOD200/400 (tomahawk based) and RSP880 so that if you ever want to migrate to newer OS's you have that option.

For the QOS, you can apply that to the subscriber, vlan and hpy interface, the media doesn't matter. The only thing you need to be aware of that the tpyhoon NPU has 32k shapers (another reason for the 32k cap per npu there), and these 32k are seggregated in 4 chunks of 8k. the chunk is assigned to a subinterface level. this means that in order to reach 32k subs, you need 4 subinterfaces of 8k each to reach that number of subs.

there is no best practice per-se other than what you would like to achieve and how that best maps to the resources available on that card :)

cheers

xander

[caio.szekely]

Thank you so much for all the information and explanation.

I want to add more information to the scenario.

Currently, we have an ASR-9006 router, with the following configuration:

1x A9K-RSP440-SE
1x A9K-MOD160-SE
2x A9K-MPA-8X10GE

We have a project to upgrade the ASR-9006 chassis to the ASR-9010.

In this upgrade, we need to associate the largest number of PPPoE clients (BNG) in a single slot.

In your explanation, I understood:

- With MOD-160-SE (typhoon) it is possible to associate up to 64K subscribers.

- With MOD-400-SE, (tomahawk), up to the version of ios 6.2.1, it is possible to associate up to 64K subscribers.

- And in this same module, from the version of ios 6.3, it is possible to associate up to 256K subscribers.

Is my understanding right?

About the RSP, it was not clear the compatibility between the MOD-400-SE and the RSP-440-SE.

Can you tell me more details of the compatibility and functionality between the RSP-440 and RSP-880, please? (32 bits / 64 bits)

Do you recommend using a Tomahawk MOD with the RSP-440?

How many subscribers can I associate using a Tomahawk MOD and the RSP-440?

Thanks a lot.

Caio Szekely

[xthuijs]

to have the most subs on a single slot, you will need to:

- lc based subscribers

- realize that typhoon will not go beyond 32k/npu and tomahawk can do 64k/npu in 62+

- a mod 160 with 4 npu's can do no more than 128k/slot (ever, hw bound) and still subject to qualification testing

- a mod200 having a single npu (tomawhawk) can do 64k subs, a mod400 with 2 npu's can do 128k subs in xr62

- a mod 400 pulling 400G through the fabric (per direction) and an RSP440 doing 220G per direction meaning you need 2 RSP's in a 9006/9010 active at all times to serve that bw.

in other words, you can make work what you want, but it doesnt have much redundancy.

it might be best to pair mod200 with an rsp440 or rsp880 ideally.

and a mod 400 with a rsp880.

noting that an rsp880 with mod 200/400 gives you eXR upgrade capability, an rsp440 eliminates that option.

cheers

xander

[Adam Vitkovsky]

Hi Xander,

Have you also tested the connection rate please? How many subs per second can typhoon and tomahawk NPUs terminate please? I think that is also one of the important scaling numbers for a BNG.

adam

netconsultings.com

::carrier-class solutions for the telecommunications industry::

[xthuijs]

hi adam, cps rate is not so much cpu dependent but more feature driven. eg qos and acl (requiring a lot of tcam programming) take down the cps. currently we are at 300 cps per module. that means 300 cps if RP based of x linecards times 300 cps per LC.

xander

[Adam Vitkovsky]

Interesting so the LC’s CPU is only one side of the equation, haven’t ever thought of this from the HW programing perspective,

Thank you very much

adam

[PauloHirakawa]

Hello,

Can I have both RP and LC based subscriber running in the same chassis? I mean, I have two A9K-RSP-440-SE and two A9K-MOD-160-SE, is it possible to have 64k subscribers running in each RSP plus 64k subscribers running in each MOD with a total of 256k subscribers? 

In summary, the client needs to achieve 256k subscribers in this chassis, is it possible or it will be necessary to add more two MODs? If I can distribute the subscribers between RSP and MOD, how would be this configs?

[xthuijs]

hi paolo, ha yeah that would be nice eh :)

the thing is for subs, whether rp or lc based, they still get programmed on the npu. that is where you have a 32k limit per npu (and for up to a total of 64k per lc = mem/soft limit).

so while a bundle sub is controlled by the rsp, it is still programmed on the lc.

regards

xander

[caio.szekely]

I need to validate PPPoE termination information in RSP.

Is it possible to use a TR type line card and associate the pppoe subscriber in RSP?

Caio Szekely

[xthuijs]

yup you can use the TR card, noting that you only have 8 queues available.

so with a standard tripple play policy, you can only establish 2 sessions per phy port.

cheers!

xander

[smailmilak]

Hi,

I see that Xander has clarified your questions. 

I would like to add that you can acquire the A9K-RSP880-RL-SE which is RSP-880 hardware but rate limited to 440G and you need a license to get 880G.

[caio.szekely]

Hi

Exactly, he clarified my questions.

I already know this RSP. In CCW the A9K-RSP-400-SE already has the End-Of-Sales Alert.

Thanks,

Caio Szekely

[caio.szekely]

Hello.

About the VSM module. Is there a loss of performance on a scenario introducing NAT64 to work in VSM in conjunction with NAT44?

With NAT64 will it be possible to support up to what nat traffic capacity?

Tks,

Caio Szekely

[xthuijs]

Hi Caio, 

VSM can do about near 60G (packet size dependent obviously) and about 20M translations.

so there is no loss of performance per-se, we will be routing packets requiring translation to the VSM via the fabric and after xlate back to the fabric and out the egress linecard.

cheers!

xander