BGP Flowspec scale and performance

[peter.ivanov]

Good Day!

Up to 3000 simple rules are supported on ASR9000 per line card using the TCAM. Does it mean that if I have 4 line cards, it is possible to support 12000 rules per ASR9000? Of course each 3000 flows should be distributed through available 4xline cards. Is it possible to have in this case alone FlowSpec neighbor for receiving all 12000 rules? 

Or may be there is restriction of the max rules per platform?

[Nicolas Fevrier]

Hi Peter,

with current implementation, the decision is binary. You can decide to activate FS on the interface or not activate FS on the interface but it's not possible to filter which rules will be applied on each interface.

That's the purpose of the Interface-set draft / discussion: https://tools.ietf.org/id/draft-litkowski-idr-flowspec-interfaceset-02.txt

This feature is not yet implemented, so basically, if you receive 3000 rules and activate FS on 4 line cards, you can not get 12000 rules but only 3000.

The only filtering supported today is the NLRI filters on source and destination addresses, but it's per BGP FS peer (it's basically a safety net to prevent rules to impact traffic accidentally) and not per interface.

Cheers,

N.

[peter.ivanov]

Hi, Nicolas!

Thanks for your answer!

I would like to add that on the ASR 9K in my lab I see correct treatment 1600 ipv4 and 1600 ipv6 rules for IOS XR 5.3.2. I notice that it is possible to use any match criteria in the same time. But main restriction is value range. For example, if add second value for DSCP, then total quantity of rules two times less is supported. One more example, if use packet length with construction less or more then some value, then total quantity of rules dramatically fall down. The same results for src-port and dst-port ranges. 

I think that in many cases very important to use range of value for some match criteria. Also because it is not predictable result of total rules within value range using, it is difficult and risky to implement this technology on the SP network. There is any plan to resolve this situation in the following releases?

Also I notice that within IOS XR 5.3.1  Flow Spec rules not working correctly. For any kind of rules I see error message regarding tcam exhaust ('prm_server' detected the 'resource not available' condition 'TCAM resource exhausted.').        

[xthuijs]

Hi Peter,

yeah if the concern is the number of tcam entries that a rule creates that is difficult business indeed. I tried to capture some of that in the cisco live id 2904 of sandiego this year in the section about TCAM that shows how to validate and monitor that particular piece.

Especially port ranges can cause quite a bit of expansion in tcam entries especially when certain bit boundaries are crossed.

We are evaluating and looking at some sort of toolset to help with the planning of tcam usage but that is not yet committed to a release as yet.

xander

[Nicolas Fevrier]

Forgot to mention that 3000 is not a hard nor well defined limit.

It will highly depends on the complexity of your rules but also, keep in mind that they are stored in TCAM which is also a resource shared with QoS and ACL.

3000 simple rules is just the number used in our validation labs.

BR,

N.

[ercintorun]

Hi Nicolas, 

If FS pbr/filter is applied on a number of interfaces on one linecard (e.g. 100 customer interfaces), would it be counted once from the TCAM usage perspective, or would it be counted per interface ? Could you please explain the mechanism behind. 

Cheers

[Nicolas Fevrier]

Hi,

the 3072 limit (which is now enforced by software, contrary to what I wrote in 2015, things changed a bit) is not dependant on the number of interfaces where BGP FS is applied.

So, 100 interfaces but one rule/line : one rule/line used in TCAM.

Thanks,

N.

[Phongthep.S]

Hi Nicolas,

Is it still being limited at 3072 on ASR9K platform?

Does Cisco has any platform that can support more than 3072 rules?

Thanks,

Phongthep,

[kay.ehrlich]

Hi Nicolas,

is the 3072 limit of the ASR9k still up to date?
What is the limit of the newer Cisco 8000 Routers (8100/8200)?

Thanks and regards,

Kay 

[smilstea]

for the 8K https://community.cisco.com/t5/service-providers-blogs/bgp-flowspec-implementation-on-cisco-8000-platforms/ba-p/4019762

for the asr9k rules is still limited to 3k.

Sam