ASR 9001 SRG 8k SubSessions limit ?

[Serg_tsk]

Dear Colleagues, Can you explain me what is the reason of this situation:

I have 8K+ subscribers in SRGroup. But new session activation is fail when еthe number of subscribers sessions reaches 8227.

sh subscriber manager disconnect-history sum shows:

No                      2018:02:21 19:31:13  Session-Start Failure, 'iEdge'
                                             detected the 'warning' condition
                                             'Start Config Failure', DC: 0 AC:
                                             48 TC: 10
No                      2018:02:21 19:31:13  Session-Start Failure, 'iEdge'
                                             detected the 'warning' condition
                                             'Start Config Failure', DC: 0 AC:
                                             48 TC: 10
No                      2018:02:21 19:31:13  IP Subscriber session create
                                             failure ( Session Activate
                                             Failure ) , DC: 0 AC: 0 TC: 9

sh subscriber manager disconnect-history last shows:

Disconnect Reason:        Session-Start Failure, 'iEdge' detected the 'warning' condition 'Start Config Failure'
Disconnect Cause:         AAA_DISC_CAUSE_DEFAULT (0)
Abort Cause:              AAA_AV_ABORT_CAUSE_SECURITY_FAIL (48)
Terminate Cause:          AAA_AV_TERMINATE_CAUSE_NAS_REQUEST (10)
Time Disconnected:        2018:02:21 19:35:51
Client:                   [iEdge internal]
Subscriber Label:         0x00000e8a
Interface:                No

[ Session Info ]

Interface:                None
Circuit ID:               Unknown
Remote ID:                Unknown
Type:                     IP: Packet-trigger
IPv4 State:               Up Pending, Wed Feb 21 19:35:51 2018
IPv4 Address:             10.227.227.198, VRF: default
Mac Address:              b0b2.dca8.ae2f
Account-Session Id:       0293642N
Nas-Port:                 1527060044
User name:                01302.b0b2dca8ae2f.10.227.227.198
Formatted User name:      01302.b0b2dca8ae2f.10.227.227.198
Client User name:         unknown
Outer VLAN ID:            1302
Subscriber Label:         0x00000e8a
Created:                  Wed Feb 21 19:35:51 2018
State:                    Connected
Authentication:           unauthenticated
Authorization:            authorized
Ifhandle:                 0x00000000
Session History ID:       0
Access-interface:         Bundle-Ether91.1302
SRG Flags:                0x00004000
Policy Executed:

  event Session-Start match-first [at Wed Feb 21 19:35:51 2018]
    class type control subscriber class-default do-until-failure [Succeeded]
      10 set-timer TIMER_UNAUTH 1 [cerr: No error][aaa: Success]
      20 activate dynamic-template DYNTPL_IP_SUB_26 [cerr: No error][aaa: Success]
      30 authorize aaa list default [cerr: No error][aaa: Success]
Session Accounting: disabled
Last COA request received: unavailable
User Profile received from AAA: None
No Services
[Event History]
   Feb 21 19:35:51.424 IPv4 Start
   Feb 21 19:35:51.680 SUBDB produce done(fail)


Disconnect Reason:        IP Subscriber session create failure ( Session Activate Failure )
Disconnect Cause:         AAA_DISC_CAUSE_DEFAULT (0)
Abort Cause:              AAA_AV_ABORT_CAUSE_NO_REASON (0)
Terminate Cause:          AAA_AV_TERMINATE_CAUSE_NAS_ERROR (9)
Time Disconnected:        2018:02:21 19:35:52
Client:                   ipsub_ma
Subscriber Label:         0x000796e2
Interface:                No

[ Session Info ]

Interface:                None
Circuit ID:               Unknown
Remote ID:                Unknown
Type:                     IP: Packet-trigger
IPv4 State:               Up Pending, Wed Feb 21 19:35:51 2018
IPv4 Address:             10.161.155.210, VRF: default
Mac Address:              14da.e923.a067
Account-Session Id:       0258045N
Nas-Port:                 1527060301
User name:                01303.14dae923a067.10.161.155.210
Formatted User name:      01303.14dae923a067.10.161.155.210
Client User name:         unknown
Outer VLAN ID:            1303
Subscriber Label:         0x000796e2
Created:                  Wed Feb 21 19:35:51 2018
State:                    Connected
Authentication:           unauthenticated
Authorization:            authorized
Ifhandle:                 0x00000000
Session History ID:       0
Access-interface:         Bundle-Ether91.1303
SRG Flags:                0x00004000
Policy Executed:

  event Session-Start match-first [at Wed Feb 21 19:35:51 2018]
    class type control subscriber class-default do-until-failure [Succeeded]
      10 set-timer TIMER_UNAUTH 1 [cerr: No error][aaa: Success]
      20 activate dynamic-template DYNTPL_IP_SUB_26 [cerr: No error][aaa: Success]
      30 authorize aaa list default [cerr: No error][aaa: Success]
Session Accounting: disabled
Last COA request received: unavailable
User Profile received from AAA: None
No Services
[Event History]
   Feb 21 19:35:51.552 IPv4 Start
   Feb 21 19:35:51.936 SUBDB produce done(fail)


Disconnect Reason:        IP Subscriber session create failure ( Session Activate Failure )
Disconnect Cause:         AAA_DISC_CAUSE_DEFAULT (0)
Abort Cause:              AAA_AV_ABORT_CAUSE_NO_REASON (0)
Terminate Cause:          AAA_AV_TERMINATE_CAUSE_NAS_ERROR (9)
Time Disconnected:        2018:02:21 19:35:52
Client:                   ipsub_ma
Subscriber Label:         0x0007be0d
Interface:                No

[ Session Info ]

Interface:                None
Circuit ID:               Unknown
Remote ID:                Unknown
Type:                     IP: Packet-trigger
IPv4 State:               Up Pending, Wed Feb 21 19:35:51 2018
IPv4 Address:             10.243.106.5, VRF: default
Mac Address:              001d.9208.eb46
Account-Session Id:       0335492N
Nas-Port:                 1527061070
User name:                01306.001d9208eb46.10.243.106.5
Formatted User name:      01306.001d9208eb46.10.243.106.5
Client User name:         unknown
Outer VLAN ID:            1306
Subscriber Label:         0x0007be0d
Created:                  Wed Feb 21 19:35:51 2018
State:                    Connected
Authentication:           unauthenticated
Authorization:            authorized
Ifhandle:                 0x00000000
Session History ID:       0
Access-interface:         Bundle-Ether91.1306
SRG Flags:                0x00004000
Policy Executed:

  event Session-Start match-first [at Wed Feb 21 19:35:51 2018]
    class type control subscriber class-default do-until-failure [Succeeded]
      10 set-timer TIMER_UNAUTH 1 [cerr: No error][aaa: Success]
      20 activate dynamic-template DYNTPL_IP_SUB_26 [cerr: No error][aaa: Success]
      30 authorize aaa list default [cerr: No error][aaa: Success]
Session Accounting: disabled
Last COA request received: unavailable
User Profile received from AAA: None
No Services
[Event History]
   Feb 21 19:35:51.552 IPv4 Start
   Feb 21 19:35:51.936 SUBDB produce done(fail)

What does it mean ? I see that radius authorized these subscribers and gave access-accept, but ASR did't create sessions.

Subscribers are in different VLAN, have  /24 subnet per VLAN with unnumbered ip address, have one loopback interface with multiple ip subnets on them. I tried to place all 8k+ subscribers into one SRG and i tried to place subscribers into two different SRG on one ASR - the result is - 8277  activated sessions. But if migrate 10% of subscribers to another node in SRG - 8277+ subscribers have a service.

ASR9001 with 4 tengig PHY, XR 6.2.25. Logging have no any error messages. Can you explain this trouble ?

Best regards

Sergey

[Aleksandar Vidakovic]

On how many bundle interfaces are subscribers terminated and how are bundle members distributed across NPs?

[Serg_tsk]

Hi Aleksandar. I'm terminate 57 Vlans on one bundled-ethernet, which consists of one PHY TE interface. Some topics ago , you told me, that SRG doesn't work on LC and this feature will come in 6.5.1 release. I configured the bundle to transfer subscriber sessions to the CP from LC. Is it problem ?

Best Regards

Sergey.

[Aleksandar Vidakovic · Accepted solution]

hi Sergey,

if there is a QoS policy on each subscriber, you are very likely hitting the limit of 8k subscribers per chunk. Traffic manager has 4 chunks, so you have to distribute your SVLANs across 4 chunks to get to the 32k subscribers per NP.

Sample config:

interface Bundle-Ether1.101

 service-policy output SPD subscriber-parent resource-id 1

interface Bundle-Ether1.102

 service-policy output SPD subscriber-parent resource-id 2

interface Bundle-Ether1.103

 service-policy output SPD subscriber-parent resource-id 3

interface Bundle-Ether1.104

 service-policy output SPD subscriber-parent resource-id 4

Try this out and let us know the outcome.

/Aleksandar

[Serg_tsk]

Thank you Aleksandar. We will plane this modification next week. I'll write about result of that modification.

Yes, we have QoS on each subscriber session for input and output direction. Egress - shape policy and Ingress - police mechanism.

Best Regards,

Sergey.

[Serg_tsk]

Hi Aleksandar. I've reconfigured ASRs yesterday and got 8800+ sessions and 40%/60% sessions distribution between chunk0 and chunk1. Thank you for your advice.

I have one more question: How can i put two ore more  subscriber session into the one "QoS pipe"? For example: there is pool of ip addresses /29 and the one subscribers contract for 5 mbit/sec for the entire subnet. There are 4 active subscribers in that network. Is there scheme, which can shape all users into 5 mbit pipe with a fair distribution of available bandwidth ? I have QoS policies and PBR-policies managed by radius-server.

Thank you

Best regards

Sergey

[Aleksandar Vidakovic]

hi Sergey,

I'm glad to hear that we are over the first hurdle. :)

You can make use of the "shared policy instance" for that purpose. See the config guide or the youtube video. 

/Aleksandar

[Serg_tsk]

Thank you for shared-policy docs that are great!

Best regards

Sergey.

[Serg_tsk]

Dear Colleagues, what about ASR9006 with A9K-RSP440-SE and A9K-24X10GE-SE? What is the scheme of QoS chunks distribution on that equipment ? We plans to connect subscribers segments  to the 10GE ports on line card directly. I know, that there is one route processor for 3 10GE ports, but if i'll terminate subscribers on virtual interfaces (Bundle-ethernet) , whether sessions will be created on the CPU of LC or RSP ? Thank you.

Best regards, Sergey.

[Aleksandar Vidakovic]

NP resource utilisation for BNG is the same for LC and RP based sessions. It doesn't matter whether the session is logically terminated on the LC or RP CPU. 

Also, you don't need to remember what is the port/chunk allocation. If you go for QinQ encapsulation, you can distribute your subscriber access VLANs across the 4 chunks using the explicit "resource-id" allocation. 

You can create a dummy flat shaper policy and attach it to the subscriber access interface:

service-policy output <name> subscriber-parent resource-id <0-3>

You can still attach a 2-level HQoS policy to the subscriber either via dynamic template or radius.

[Serg_tsk]

Thank you, Aleksandar. I have one more question about BNG:

What is the limit of subscribers sessions with ipv4 only IPoE sessions on 9k6 with RSP440-SE and A9K-24X10GE-SE ? What is the limit of sessions with dual stack (v4 + v6) ? QoS scheme is: police for input packets and shape for output packets.

Best regards, Sergey.

[Aleksandar Vidakovic · Accepted solution]

hi Sergey,

with RSP440-SE and A9K-24X10GE-SE you have to stick to 32-bit XR. With 32-bit XR the scale limit is primarily dictated by the max amount of virtual memory space that a process can address. This means that the dynamic memory limit for a process plus the shared memory size can't exceed 4GB.

With RSP440-SE and A9K-24X10GE-SE the max scale you can achieve is 32k subscribers per NP and 64k subscribers per LC. If you go for LC-based subscribers we support 128k dual-stack sessions per chassis. With RP-based subscribers we support 64k dual-stack sessions per chassis.

If you need a higher scale than this, please reach out to your account team at Cisco. I'm sure we can come up with a good proposal for a migration from RSP440+Typhoon to RSP880+Tomahawk, which would allow you to unlock the full power of the asr9k BNG solution. 

Happy New Year!

/Aleksandar

[Serg_tsk]

Hi, Aleksandar! Where will the session be created if i terminate subscriber on L1 connected 9000nv satellite? On LC or RSP ? Does the Cisco officially support termination of subscriber sessions on the phy TenGig ports on 24x10G-se with the SRG functionality (DHCP and unclassified src) or i must to create sessions on virtual interfaces ?

Thank you!

[Aleksandar Vidakovic]

If the subscriber access interface is a Bundle-Ether or PW-Ether (sub)interface, subscriber sessions are terminated on the RP CPU.

If the subscriber access interface is a physical (sub)interface, subscriber sessions are terminated on the LC CPU. BNG Geo-Redundancy on LC-based subscribers is supported starting with IOS XR release 6.5.1, but the recommended release is 6.5.2. We expect 6.5.2 to be on cisco.com by the end of January. If you want to give it a run in the lab, I can share a pre-release image with you. Please pass me your CCO user ID via private message if you want to try out the 6.5.2 pre-release image in your lab.

asr9000v satellite is completely transparent. None of the BNG features is offloaded to the satellite.

[Serg_tsk]

Hi Aleksandar!

Thank you for your offer of pre-release image, but we only have 2 chassis  in production :( If tell the truth ,

the RSP resource is enough for us in GEO RED scheme. But i'm trying to think about the future.

Thank you!

With best regards, Sergey.