You are always good respond. - Page 2

Andy Erickson · ‎03-10-2015

We currently have 7,500 broadband subscribers that we will be terminating on our ASR 9001.

Each one of our customers will be terminating on a sub-interface on a bundle.

On the 9k, there will be a QoS policy applied to rate-limit their broadband connection (see example below).

The challenge that we are running into right now is scaling beyond 4096 L3 sub-interfaces. When running through this in our lab, we receive the following fail message:

RP/0/RSP0/CPU0:BNG(config-subif)#show config failed

Tue Mar 10 18:32:07.552 UTC

!! SEMANTIC ERRORS: This configuration was rejected by

!! the system due to semantic errors. The individual

!! errors with each failed configuration command can be

!! found below.

interface Bundle-Ether10.6941171

!!% The L3 sub-interface limit for the trunk interface has been reached: Trunk limit for L3 subinterfaces on Bundle-Ether10 is 4096

We have added the following on to each of the sub-interfaces to "fake" out the NPU, but even with SPD configured, we are receiving the max 4096 message:

service-policy output <POLICY> subscriber-parent resource-id 0
service-policy output <POLICY> subscriber-parent resource-id 1
service-policy output <POLICY> subscriber-parent resource-id 2
service-policy output <POLICY> subscriber-parent resource-id 3

It is my understanding that we have a total of 4 resource ID's to use (0-3) and the ASR 9001 will support up to 32,000 sub-interfaces (system wide or 8,000 sub interfaces per resource-id).

See attached image for reference this design.

Main question to the community is what is the work around to scale beyond 4096 L3 sub-interfaces??

In our case it is not feasible to bring in additional bundles and spread the customers out.

Look forward to your responses.

Below is a sample configuration:

policy-map 10M_D
class class-default
shape average 10100000 bps
!
end-policy-map
!
policy-map 10M_U
class class-default
police rate 10300000 bps
exceed-action drop
!
!
end-policy-map
!

interface Bundle-Ether10.650102
description ---INT: GigabitEthernet0/0/1.650102 NAME: TEST #1---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 0
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 102
!
interface Bundle-Ether10.650103
description ---GigabitEthernet0/0/1.650103 NAME: TEST #2---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 1
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 103
!
interface Bundle-Ether10.650104
description ---INT: GigabitEthernet0/0/1.650104 NAME: TEST #3---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 2
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 104
!
interface Bundle-Ether10.650105
description ---INT: GigabitEthernet0/0/1.650105 NAME: TEST #4---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 3
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 105
!

interface Bundle-Ether10.650106
description ---INT: GigabitEthernet0/0/1.650106 NAME: TEST #5---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 0
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 106
!
interface Bundle-Ether10.650107
description ---INT: GigabitEthernet0/0/1.650107 NAME: TEST #6---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 1
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 107
!
interface Bundle-Ether10.650108
description ---INT: GigabitEthernet0/0/1.650108 NAME: TEST #7---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 2
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 108
!
interface Bundle-Ether10.650109
description ---INT: GigabitEthernet0/0/1.650109 NAME: TEST #8---
service-policy input 10M_U
service-policy output 10M_D subscriber-parent resource-id 3
ipv4 point-to-point
local-proxy-arp
ipv4 unnumbered Loopback10
encapsulation dot1q 650 second-dot1q 109
!

Andy Erickson · ‎04-01-2015

xander,

Thanks for sharing the QinQ username, works perfectly.

couple of design questions for you.

#1 - If i have >7500 subscribers that will be terminating on this bundle, would this be the best design to ensure that i can scale up to 32,000 subscribers on the BE <leveraging the subscriber-parent resource-id (0-4)>

EXAMPLE

interface Bundle-Ether10.100

 description BE10.100 – Area 1 - BNG customers - QinQ

 ipv4 point-to-point

 ipv4 unnumbered Loopback0
 service-policy output <POLICY> subscriber-parent resource-id 0

 service-policy type control subscriber IP_PM

 ipsubscriber ipv4 l2-connected

  initiator dhcp

 !

 encapsulation ambiguous dot1q 100 second-dot1q any

!

interface Bundle-Ether10.200

 description BE10.200 – Area 2 - BNG customers - QinQ

 ipv4 point-to-point

 ipv4 unnumbered Loopback0
 service-policy output <POLICY> subscriber-parent resource-id 1

 service-policy type control subscriber IP_PM

 ipsubscriber ipv4 l2-connected

  initiator dhcp

 !

 encapsulation ambiguous dot1q 200 second-dot1q any

!

interface Bundle-Ether10.300

 description BE10.300 – Area 3 - BNG customers - QinQ

 ipv4 point-to-point

 ipv4 unnumbered Loopback0
 service-policy output <POLICY> subscriber-parent resource-id 3

 service-policy type control subscriber IP_PM

 ipsubscriber ipv4 l2-connected

  initiator dhcp

 !

 encapsulation ambiguous dot1q 300 second-dot1q any

!

interface Bundle-Ether10.400

 description BE10.400 – Area 4 - BNG customers - QinQ

 ipv4 point-to-point

 ipv4 unnumbered Loopback0
 service-policy output <POLICY> subscriber-parent resource-id 4

 service-policy type control subscriber IP_PM

 ipsubscriber ipv4 l2-connected

  initiator dhcp

 !

 encapsulation ambiguous dot1q 400 second-dot1q any

!

#2 - How do I verify in XR the CoA speed profile that is pushed down from RADIUS to a given subscriber?

I thought I might see the dynamic policy using the command below, but no luck.

Do you know the correct command?

RP/0/RSP0/CPU0:bng-asr9001#show policy-map inter be10.1.ip5

Wed Apr  1 14:12:06.390 UTC



Bundle-Ether10.1.ip5 input: __sub_55ffffff8b7dffffffad



Class class-default

  Classification statistics          (packets/bytes)     (rate - kbps)

    Matched             :              126959/10831088             14

    Transmitted         : N/A

    Total Dropped       : N/A



  Policy __sub_55ffffff8b7dffffffad_child1 Class class-default

    Classification statistics          (packets/bytes)     (rate - kbps)

      Matched             :              126959/10831088             14

      Transmitted         : N/A

      Total Dropped       :                 325/322582               0

    Policing statistics                (packets/bytes)     (rate - kbps)

      Policed(conform)    :              126634/10508506             14

      Policed(exceed)     :                 325/322582               0

      Policed(violate)    :                   0/0                    0

      Policed and dropped :                 325/322582            

      Policed and dropped(parent policer)  : N/A



Bundle-Ether10.1.ip5 output: __sub_6effffff81ffffffbfffffffdb



Class class-default

  Classification statistics          (packets/bytes)     (rate - kbps)

    Matched             :              199642/280153690            453

    Transmitted         : N/A

    Total Dropped       : N/A



  Policy __sub_6effffff81ffffffbfffffffdb_child1 Class class-default

    Classification statistics          (packets/bytes)     (rate - kbps)

      Matched             :              199642/280153690            453

      Transmitted         : N/A

      Total Dropped       :               26930/38989025             61

    Policing statistics                (packets/bytes)     (rate - kbps)

      Policed(conform)    :              172712/241164665            392

      Policed(exceed)     :               26930/38989025             61

      Policed(violate)    :                   0/0                    0

      Policed and dropped :               26930/38989025          

      Policed and dropped(parent policer)  : N/A

RP/0/RSP0/CPU0:bng-asr9001#

#3 - CoA QoS profile -> I'm using the following avpair for ingress / egress qos. However when validating against a speed test server, my results are well above the 10Mbps / 10Mbps I have provisioned. Actual is more of in the ~15Mbps/15Mbps range.

Am I missing additional config in the policing section?

        cisco-avpair = "ip:qos-policy-in=add-class(sub, (class-default,class-default),police(10000))",

        cisco-avpair += "ip:qos-policy-out=add-class(sub, (class-default,class-default),police(10000))"

Appreciate it in advance xander!

-ae

xthuijs · ‎04-03-2015

you can use show qos int <bla> to see what is programmed in the hardware.

as for speed, probably have bundle and perfect loadbalancing... gives you member times the bw assigned.

so you want to set the bundle hash dest-ip on your bundle parent interface to hash only on dest-ip which is sub, to tie a sub to a member to prevent that.

qos resource ID is necessary when you have shapers in place and you need more then 8k per interface. (i.e. assign a chunk to a subinterface to give it access to 8k subs per subinterface).

you can't have more then 8k per subif.

xander

Pichet Piriyakitkosol · ‎04-25-2017

Hi Xander

Is this "subscriber-parent resource-id" command can only manual apply? What's should I do if we assign qos profile from AAA? My goal is want to reach 32K subscriber and assigned profile from AAA.

And also Is this "subscriber-parent resource-id" command are really apply qos to interface or is it fake?

Thank you

Pichet

xthuijs · ‎04-25-2017

hi pichet,

the qos policy you apply via radius is for the subscriber session itself.

the qos policy that is needed to assign the chunk is on the access (sub) interface.

in order to reach 32k subs with qos, you need to have 4 subinterfaces and you can terminate 8k subs per subinterface.

this subinterface can either be a vlan or a vlan range.

cheers

xander

Pichet Piriyakitkosol · ‎04-25-2017

You are always good respond. Thanks very much.

Pichet

xthuijs · ‎03-12-2015

The what I call "elephant trap" is done on a per mac bases, so before LPTS sees the discover, the elephant trap is already taking place which puts an abusing MAC/source access interface into the penalty bucket, almost like a super nanny type of approach (the naughty spot!).

You can always apply a policer on the subsciber directly also if you like, but for control plane protection, you don't need to, because it is there already.

I can dig up some info on the elephant trap if you like but it should be part of the docu also, if not, let me know and I'll get you sorted.

cheers!

xander

Andy Erickson · ‎03-13-2015

xander,

can you send the "elephant trap" info.

appreciate it!

-ae

xthuijs · ‎03-13-2015

I attached some pics for you from our training deck on BNG for this elephant flow trap.

And this is a ref to the official documentation:

http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r5-3/bng/configuration/guide/b-bng-cg53xasr9k/b-bng-cg53xasr9k_chapter_0111.html#concept_60A7D05B2248411BB2DECA90FC2E7ABC

xander

ASR9000/XR - BNG - L3 sub-interface limit for trunk (4096) error - what is the work around?