Introduction
The support for Line Card based Subscribers in IOS-XR BNG came in Release 5.1.1. Prior releases only had RP based Subscriber (Access interface as Bundle Interfaces) support.
In RP based Subscribers, the entire Control Plane is run on RSP. The important processes like iedged, dhcpd, subdb_svr, ipsub, etc runs on RSP. Also, the feature control plane (<feature>_ma) runs on RSP, while the hardware programming part (<feature>_ea) runs on LC.
The problems associated with RP based subscribers are as follows
Memory usage limits the number of sessions that can be brought up to about 128K IPv4 sessions. Physical/Shared memory resources are limited.
It doesn't use full potential of ASR9k. Even an ASR9k with more LC's will rely on RSP to do processing resulting in problems with Vertical scaling.
CPS is limited as well due to above points.
Poor fault isolation.
LC based Subscriber
Due to above challenges in Bundle based Subscribers, Line Card based Subscriber was introduced in 5.1.1.
The entire BNG control plane is designed to run in a distributed manner, on all the line cards. This allows the BNG control plane to be easily distributed and to terminate any session initiation protocol directly on the line card, as opposed to terminating the protocol on the RSP.
Running the control plane in such a distributed manner allows for higher fault isolation on the router. This allows certain LCs to fail without affecting subscriber sessions on the other LCs in the system. Only subscriber sessions built on that LC is lost.
Here, the subscriber controller plane is replicated to every LC on the system. Every LC will offer 8 Gig of physical memory. Hence, as more LC's are added to the system, horizontal scaling becomes possible. By doing this, memory and CPU on RP is freed up which can be used by Multi-Service Edge (MSE) Profile, allowing the ASR9K to provide significant cost benefits to the service provider. In an MSE environment, not only must the router support BNG functionality, but it must also provide other services, such as high scale BGP and other routing protocols, L3VPN/L2VPN services and other edge/aggregation functionalities.
Feature supported
All features which are supported for Bundle based (RSP) subscribers till release 4.3.1 would be supported for LC based Subscribers too, except for the listed features below.
Per-User QoS
Multicast
Service Accounting.
Local DHCPv4 Server (The support for RP based Subscribers came in 5.1.0. It works for LC based Subscribers too, but is untested as of now)
Hardware Support
Any typhoon linecard such as the A9K-MOD80-SE, A9K-24x10, 36x10 and MOD160.
Note that the linecard should be of the "SE" or Service Edge variant.
Also the ASR9001 will support BNG.
Trident linecards are not supported for subscriber termination, but they can be used as core facing linecard for transport
Memory implications
Unlike RP based subscribers, where each line card holds only data plane related information, in case of LC Based subscribers, the subscriber information is stored primarily on the line cards. LCs on the ASR9K are quite powerful and have nearly the same amount of physical memory as the RSP.
For every LC subscriber, there is a certain amount of data stored on the RSP as well, primarily as:
IM’s G3P database
SADB database in iedge for CoA handling
RIB route
DHCPv4 and v6 store a shadow database on the RSP for every binding on the LC.
IM’s G3P database is used by processes running on the RSP to obtain information about all interfaces, typically by routing protocol processes to be able to run their protocol or by show commands to display interface name to handle mappings.
IM process running on LC will only poll interface which are local to LC. IM process on RP polls interfaces on each nodes of the system. This is done for show commands, converting ifh etc. Protocol on RSP's need to know what there on LC. IM on RP currently keeps a copy of LC. This limits scaling higher.
SADB database in iedge is needed primarily for two reasons.
For managing Change of Authorization (CoA) requests. Only iedge process running on RP has the command handler. iedge looks up the sessions on the RSP using the replicated SADB and uses that to direct the CoA to iedge on the right LC.
LC OIR case. Iedge on RP sends the final stats from the SADB.
RIB route addition is unavoidable and has to go via RSP.
DHCP on RSP maintains shadow bindings. On LC OIR, DHCP on RSP replays bindings back to LC after LC comes back up.
In comparison to the data stored for each subscriber on the RSP for bundle subscribers, the amount of data stored for LC subscribers on the RSP is minimal. This distribution of subscriber data to the LCs where the subscriber is hosted allows the router to scale to much higher number of subscriber sessions.
Scale Numbers
64k Single Stack subscribers per LC
32k Dual Stack subscribers per LC
256K Single Stack subscribers per System
128K Dual Stack subscribers per System
Performance
Calls-per-second (cps) is defined as the number of sessions that can be brought up in a second.
In Bundle subscribers, control plane runs completely on the RSP, while the data plane runs completely on the LC. As a result, the control plane and data plane provisioning can happen on different CPUs and hence can happen in parallel. At the same time, due to this separation, there is a significant number of inter-node IPCs between the RSP and LC.
For subscribers terminating on physical interfaces, both the control plane and the data plane run on the same node, sharing the same CPU resource. As a result, the amount of CPU available to bringup the session is limited. The CPS achieved on a single LC, therefore, is lower than the CPS achieved for Bundle subscribers. At the same time, the two planes don’t have to communicate with each other using inter-node IPCs, as a result that CPU should be available for processes.
A significant gain in performance is achieved for LC subscribers by the distribution of control plane to multiple LCs. This allows us to make use of each LC’s processing power to process more sessions. As a result, even though the LC is less powerful and has to run both the control plane and data plane on the same CPU, in aggregate, the entire chassis can reach much higher scale and performance than RSP based subscribers.
Approximate target CPS is listed in the table below, assuming 64K sessions per LC and adding as many LCs needed to scale to the appropriate level.
Scale/Call Flow
IPoE
PPPoE
256K IPv4
200
200
128K Dual Stack
60
60
High Availability
HA Feature | Plane | Bundle subscribers | Line card subscribers |
Process restart | Control Plane | Subscriber session state maintained. New subscriber bringup delayed by a small time, depending on the component being restarted. | Same as bundle subscribers. |
Data Plane | No impact to traffic | No impact to traffic | |
LC OIR | Control Plane | With multi-member bundles, no impact is seen. With single member bundles, control plane cannot function since no control packet is received. Session state is not lost since that is stored on the RP. | Control plane is down for new sessions and all session state is lost for existing sessions. |
Data plane | With multi-member bundles, no impact is seen. With single member bundles, data traffic is lost. No session state is lost. | All traffic is lost | |
RP FO | Control Plane | Significant quiet time (currently more than 10 minutes) before new sessions can be setup. Existing session state is not lost. | Very small impact (about 30-60 seconds) before new sessions can be setup; the delay is in connecting to RP based servers, like RIB. Existing session state is not lost. |
Data Plane | No impact to traffic | No impact to traffic |
External Interaction
Load balancing – In case of Bundle based subscribers, the user need not worry about load-balancing, since the entire control plane is run on one node (RSP), which will take care of the load-balancing. With LC based subscribers, each LC control plane functions independently, any global configuration of radius and dhcp servers will not result in load balanced usage. It is quite possible that all LCs will end up using the same radius server. As a result, it is upto the user to do some manual load balancing. This can be achieved by creating different aaa groups and method lists using different sets of radius servers and assigning the aaa groups to different service profiles and then assigning the different service profiles to the access interfaces on different LC's. Similarly for dhcp servers, access interfaces on different LCs need to have different profiles, each pointing at different dhcp servers.
Radius - The radius client need to be set up such that the entire BNG router shows up as one BNG to the Radius (NAS IP Address). This is important because currently, the CoAs can only be handled by the iedge on the RSP, which has the command handler module to deal with CoAs. The command handler then looks up the LC associated with a subscriber and pushes the CoA to the iedge on the appropriate LC. The command handler in iedge looks up the sessions on the RSP using the replicated SADB and uses that to direct the CoA to iedge on the right LC.
Address Pools - Since each LC operates independent of each other, when operating in server mode, if the addresses given out by the control plane on those LCs need to be in same pool, then proper coordination is required to ensure that the LCs don’t assign the same address to different subscribers. This is managed by the DAPS server component on the RSP . It is preferable currently to provide different pools to different LCs so that they can work completely independent of each other, without the need to perform significant messaging across nodes.
Configs
(The configs are same as that of Bundle based subscribers)
Radius configuration
radius-server host 8.0.0.172 auth-port 2001 acct-port 2002
key 7 121A0C041104
!
radius-server host 8.0.0.172 auth-port 3001 acct-port 3002
key 7 14141B180F0B
!
radius-server host 8.0.0.172 auth-port 4001 acct-port 4002
key 7 14141B180F0B
!
radius-server host 8.0.0.172 auth-port 5001 acct-port 5002
key 7 070C285F4D06
!
radius-server host 8.0.0.172 auth-port 6001 acct-port 6002
key 7 05080F1C2243
!
radius-server host 8.0.0.172 auth-port 7001 acct-port 7002
key 7 121A0C041104
!
radius-server host 8.0.0.172 auth-port 8001 acct-port 8002
key 7 0822455D0A16
!
radius-server host 8.0.0.172 auth-port 9001 acct-port 9002
key 7 1511021F0725
!
aaa group server radius aaa-group1
server 8.0.0.172 auth-port 2001 acct-port 2002
server 8.0.0.172 auth-port 3001 acct-port 3002
server 8.0.0.172 auth-port 4001 acct-port 4002
server 8.0.0.172 auth-port 5001 acct-port 5002
load-balance method least-outstanding batch-size 1
!
aaa group server radius aaa-group2
server 8.0.0.172 auth-port 6001 acct-port 6002
server 8.0.0.172 auth-port 7001 acct-port 7002
server 8.0.0.172 auth-port 8001 acct-port 8002
server 8.0.0.172 auth-port 9001 acct-port 9002
load-balance method least-outstanding batch-size 1
!
aaa accounting subscriber methodlist1 group aaa-group1
aaa accounting subscriber methodlist2 group aaa-group2
aaa authorization subscriber methodlist1 group aaa-group1
aaa authorization subscriber methodlist2 group aaa-group2
aaa authentication subscriber methodlist1 group aaa-group1
aaa authentication subscriber methodlist2 group aaa-group2
aaa authentication ppp methodlist1 group aaa-group1
aaa authentication ppp methodlist2 group aaa-group2
Access interface config
interface TenGigE0/0/0/0
!
interface TenGigE0/0/0/0.1
service-policy output vlan_policy_egress subscriber-parent resource-id 0
service-policy type control subscriber PM_CNTRL_1
pppoe enable
encapsulation ambiguous dot1q 10 second-dot1q any
!
Class Map and Policy Map
class-map type control subscriber match-any PPP_CM
match protocol ppp
end-class-map
!
policy-map type control subscriber PM_CNTRL_1
event session-start match-first
class type control subscriber PPP_CM do-until-success
10 activate dynamic-template PTA_TEMPLATE_1
!
!
event session-activate match-first
class type control subscriber PPP_CM do-until-success
10 authenticate aaa list methodlist1
!
!
end-policy-map
!
