cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
50122
Views
25
Helpful
66
Comments
xthuijs
Cisco Employee
Cisco Employee

 

Introduction

BGP flow spec is a new tool that can be used to assist in DDOS mitigation in a dynamic fashion, levering BGP.

When I first started working and looking at BGP-FS I was a bit confused and in this article I would like to try and unravvle some of the specifics of BGP-FS, what you can do with it and how it works.

 

DDOS mitigation via RTBH (remote triggered blackhole)

Conceptually in this picture:

 

DDOS mitigation with the help of BGP is generally done via RTBH (remote triggered blackhole). And this comes basically in 2 key flavors: Source and Destination based.

The concept is that a BGP edge speaker has a static route defined to a "special next hop" to null0.

From a central point we inject a new route for a particular destination (eg our affected webserver) with a next hop pointing to that special next hop. This means that now at our border routers, our webserver address is set to a next hop, and this next hop is drop, and we prevent this traffic from hitting our server. This protects it nicely. But it removes the ability to reach this server completely!

The source based model is a similar concept whereby we KNOW where an attack comes from and now we send in a bgp path for that source with the special next hop, which then points to null.

Now when we use uRPF (reverse path forwarding), when a packet from that suspect source comes in, we apply RPF, we find the nexthop set to null0/discard. This would make RPF fail and we drop the packet from that host (or subnet for that matter).

Both options work great, but are big hammers.

BGP flowspec allows for a more granular appraoch and effectively construct instructions to match a particular flow with source AND destination, and L4 parameters and packet specifics such as length, fragment etc, and allow for a dynamic installation of an action at the border rotuers to either:

  • drop the traffic
  • inject it in a different vrf (for analysis)
  • or allow it, but police it at a specific defined rate.

Cool. Now that we got that theory out of the way, how about Flowspec then, what does it do and how is it different?!

 

BGP flowspec definitions

This conceptually describes how flowspec works:

So instead of sending/injecting a route for our server or server subnet with a special community that the border routers will associated with a nexthop to drop in their route policy language, in this case we are sending the new flowspec attribute format down to the border routers instructing them to create a sort of ACL with class-map and policy-map to implement this rule that we are advertising!

Very nifty, am lovin it. So in order to do that, we need to extend the BGP protocol a bit, and this is how the RFC is defining that approach:

BGP FS adds a new NLRI (network layer reachability info) to the BGP protocol:

NLRI defined (AFI=1, SAFI=133)

It adds the following packet/attribute format:

It allows for the following parameters to be defined:

1.Destination IP Address (1 component)
2.Source IP Address (1 component)
3.IP Protocol (+1 component)
4.Port (+1 component)
5.Destination port (+1 component)

6.Source Port (+1 component)

7.ICMP Type (asr9k has support for this yet, CRS does also)

8.ICMP Code (asr9k has support for this yet, CRS does also)

9.TCP Flags (asr9k supports lower byte, crs can not look at all bits)

10.Packet length

11.DSCP

12.Fragment

 

The actions that can be associated to a flow that we matched on by the parameters specified above are encoded as follows:

 

And the available actions that we can define are:

So with these new rules we can define a prefix or prefix set and not just that, we can base it on L3 source, L3 dest, its L4 specifics and even on packet length and L3 header options such as fragments! Then we can specify what actions we want to associate with that flow, redirect it, remark it, drop it or police it. And that is the full flow info.

All actions are supported by both CRS and ASR9000.

Ok this is all awesome, now that we got the theory out of the way, you want to go set it up I am sure!

Client/Server Model

In BGP-FS there is a client and a server. What that effectively means is that there is a device that injects the new flowspec entry (for the lack of a better word I guess) and there is a device (BGP speaker) that receives that NRLI and has to implement something in the hardware forwarding to act on that instruction.

So the device sending or injecting the NRLI is what I call the server, and the device that receive the NRLI and programs the hardware forwarding is called the client.

Visually:

CLIENT

In this concept the server in the grey box injects the flowspec NRLI and the client in the right box receives the information, sends it to the flow spec manager, configures the ePBR (enhance Policy based routing) infrastructure who then in turn programs the hardware from the underlaying platform in use.

SERVER

The server is configured either via CLI or XML to provide that entry for NRLI injection.

Note that XRv, virtual XR, can be a Server, it can inject those NRLI's for an asr9000 or CRS to act on those. Obviously XRv, not having a hardware forwarding layer is not a client

 

Configuration

conceptually

BGP

applicable to both client and server, we need to enable the new address family for advertisement:

router bgp 100

 address-family ipv4 flowspec

 ! Initializes the global address family

 address-family ipv6 flowspec

 !

 neighbor 1.1.1.1

  remote-as 100

  address-family ipv4 flowspec

  ! Ties it to a neighbor configuration

  address-family ipv6 flowspec

  !

 

Interface

You can disable flowspec on the interface as follows:

Interface gigabit0/0/0/0

 ipv4 flowspec disable

 !

Client

requires no special config other then a flowspec enabled peer.

Server

The server side configuration includes the policy-map definition and the association to the ePBR config consists of 2 portions, the class definition, and using that class in ePBR to define the action.

Classification

class-map type traffic match-all <class-name>

 <match statement> #Any combination of tuples 1-13 match statements goes here.

 end-class-map

 !

!

Tuple definition possibilities:

#Type 1: match  destination-address {ipv4| ipv6} <IPv4/v6 address>/<mask-length>

#Type 2: match  source-address {ipv4| ipv6}  <IPv4/v6 address>/<mask-length>

#Type 3: match protocol  { <value> | <min_value> -  <max_value> }

<!In case of IPv6, it will map to last next-header >

#Type 4: create two class-maps one with source-port and another with destionation-port

match source-port { <value> | <min_value> -  <max_value> }

match destination-port { <value> | <min_value> -  <max_value> }

<! Applicable only for TCP and UDP protocol !>

#Type 5: match destination-port { <value> | <min_value> -  <max_value> }

#Type 6: match source-port   { <value> | <min_value> -  <max_value> }

#Type 7: match {ipv4 | ipv6} icmp-type  { <value> | <min_value> - <max_value>}

#Type 8: match {ipv4 | Ipv6} icmp-code { <value> | <min_value> - <max_value>}

#Type 9: match tcp-flag   <value>  bit-mask <mask-value>

#Type 10: match packet length { <value> | <min_value> -  <max_value> }

#Type 11: match dscp { <value> | <min_value> -  <max_value> }

match ipv6 traffic-class  { <value> | <min_value> -  <max_value> }  

   [for providing 8 bit traffic class value]

#Type 12:  match fragment-type  {dont-fragment, is-fragment, first-fragment, last-fragment}

#Type 13:   match ipv6 flow-label { <value> | <min_value> -  <max_value> }

 

policy-map type pbr <policy-name>

 class type traffic  <class-name>

   <action>  #Any one of the extend community action listed below

 class class-default

 end-policy-map

ACTIONS:

##Traffic rate:

     police  rate < >  | drop

#Traffic action:

   sample-log

#Traffic marking:

     set dscp  <6 bit value> |

     set ipv6 traffic-class <8 bit value>

#VRF  redirect based on route-target:

  redirect {ipv6} extcommunity rt <route_target_string>

# Redirect IP nexthop support

 redirect {ipv6} next-hop <ipv4/v6 address> {ipv4/v6 address}

 

The following ties the flowspec to the PBR policies defined earlier.

flowspec

 [local-install interface-all]

 address-family ipv4

  [local-install interface-all]

  service-policy type pbr <policy-name>

  service-policy type pbr <policy-name>

 address-family ipv6

  [local-install interface-all]

  service-policy type pbr <policy-name>

  service-policy type pbr <policy-name>

  !

 ! 

 vrf  <vrf-name> 

  address-family ipv4

   [local-install interface-all]

   service-policy type pbr <policy-name>

   service-policy type pbr <policy-name>

  address-family ipv6

   [local-install interface-all]

   service-policy type pbr <policy-name>

   service-policy type pbr <policy-name>

   !

  !

 !

Example use case/configuration:

Target:

all packets to 10.0.1/24 from 192/8 and destination-port {range [137, 139] or 8080, rate limit to 500 bps in blue vrf and drop it in vrf-default. Also disable flowspec getting enabled on gig 0/0/0/0.

Associated copy paste config:

class-map type traffic match-all fs_tuple

 match destination-address ipv4 10.0.1.0/24

 match source-address ipv4 192.0.0.0/8

 match destination-port 137-139 8080

 end-class-map

 !

!

policy-map type pbr fs_table_blue

 class type traffic fs_tuple

  police rate 500 bps

  !

 !

 class class-default

 !

 end-policy-map

policy-map type pbr fs_table_default

 class type traffic fs_tuple

  drop

  !

 !

 class class-default

 !

 end-policy-map

flowspec

 local-install interface-all

 address-family ipv4

  service-policy type pbr fs_table_default

  !

 !

 vrf blue

  address-family ipv4

   service-policy type pbr fs_table_blue local

   !

  !

 !

!

Interface GigabitEthernet 0/0/0/0

 vrf blue

 ipv4 flowspec disable

 

 

BGP Flowspec and local QoS configuration

When flowspec is implemented on an interface that is also having local QoS configuration, local config will come before flowspec processing.
Local config will police and dscp-mark the packets and pass them to flowspec.
Flowspec will then do its processing (police, redirect) except dscp marking.

Flowspec will retain dscp marking as dictated by local qos config.

Say, we have the following:

inbound qos config : police 100Mbps, mark dscp af11
 
=============================================================
 
ipv4 access-list acl_ipv4_qos_stream
6 permit ipv4 any host 200.255.5.2
!
!
class-map match-any cm_ipv4_qos_stream
match access-group ipv4 acl_ipv4_qos_stream
 end-class-map
!
 
policy-map pm_ipv4_qos_stream
class cm_ipv4_qos_stream
  police rate 100 mbps
  !
  set dscp af11
!
 class class-default
!
 end-policy-map
!
 
interface hundredGigE 0/4/0/35
service-policy input pm_ipv4_qos_stream
 
=============================================================

Then we receive the following in flowspec advertisement.
flowspec config : police 50Mbps, mark dscp af43, redir vrf.
 
=============================================================
 
RP/0/RP0/CPU0:fretta-50#sh flowspec ipv4 detail | b 200.255.5.2
  Flow           :Dest:200.255.5.2/32
    Actions      :Traffic-rate: 50000000 bps DSCP: af43 Redirect: VRF honeypot Route-target: ASN2-4787:13  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :           116570713/12822778430        
      Transmitted         :            57360817/6309689870         
      Dropped             :            59209896/6513088560   
 
=============================================================
 
Then the outcome will be:

 

  1. traffic will be policed by flowspec at 50Mbps.
  2. traffic will be redirected by flowspec to VRF honeypot.
  3. flowspec will not overwrite dscp marking, traffic will be forwarded using dscp af11 instead of af43.

NCS5500 platform will also display this same behavior.

 

Special thanks to Nic Fevrier whose pictures I used for some of the visuals above.

 

Xander Thuijs CCIE#6775

Principal Engineer, ASR9000

 

Comments
nimmi.phasil
Beginner
Beginner

Hi Nicolas,

 

It worked. Removed some configurations and reloaded . Thanks for the support.

Regards

Nimmi MP

nimmi.phasil
Beginner
Beginner

RP/0/RSP0/CPU0:ios#sh platform
Tue Jul 28 01:43:39.843 UTC
Node            Type                      State            Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0     ASR9001-RP(Active)        IOS XR RUN       PWR,NSHUT,MON
0/0/CPU0        ASR9001-LC                IOS XR RUN       PWR,NSHUT,MON
0/0/0           A9K-MPA-4X10GE            OK               PWR,NSHUT,MON
0/0/1           A9K-MPA-20X1GE            OK               PWR,NSHUT,MON
RP/0/RSP0/CPU0:ios#

 

Thanks

Nimmi

Nicolas Fevrier
Cisco Employee
Cisco Employee

Hi Nimmi,

ASR9001 is Typhoon based, all supported for BGP FS.

But if you said a reload fixed it, you can keep on running your tests then.

BR,

N.

spapageorgiou
Beginner
Beginner

Hi all,

We have an ASR9001. I would like some info about the sizing of BGP-FS. For example, how many /32 routes can i send over it, without reaching tcam limits?

DDOS attacks can easily reach hundred of thousands attacking hosts...

thanx,

Sp

Nicolas Fevrier
Cisco Employee
Cisco Employee

Hi Sp,

The scale we support (ie that we test in our labs) is 3000 rules. In the context of DDoS mitigation it's usually way enough because we are not trying to identify every single source responsible for the attack, but more generally we identify the victim or the range of addresses under attack and we try to filter based on packet specificities.

For example: in the case of NTP amplification attack, we will drop packet following this description:
- Destination address: victim IP or range of the victim IPs
- protocol 17: UDP
- port source or destination: 123
- packet size: larger than 400 (it's a random number)

--> we don't define the source, so it could be from everywhere as long as these other conditions are met (also, it's common to define a second rule for fragmented packets targeted to the victim because the attack packets may be larger than the PMTU and will often be fragmented).

Hope it helps a bit,

N.

Aleksandar Vidakovic
Cisco Employee
Cisco Employee

On asr9k we support 3k simple rules per system. If you require a more advanced solution, you could consider the DDoS prevention solution that we developed in collaboration with Arbor. You would need a larger asr9k for that solution and a VSM line card.

/Aleksandar

spapageorgiou
Beginner
Beginner

Thank you Nicolas and Aleksandar for the answers!

Sp

costan
Beginner
Beginner

Given FS limite indication, you can mix and match with RPF unicast check and RTBH.

If it's a DoS or small source set DDoS, you use FS. If it's a massive DDoS (thousand of sources), it's better to RBTH, whose limit is route table size (way mooore), and its protection applies to all customers at the same time, and whose drawback is only that the DDoS source become unreachable (not a big deal if they are botnet infected machines).

Hi, can you please share some design guide for ASR9K and Arbor integration?. Specific for ASR9K VSM + BGP FlowSpec. I not found in Cisco Website.

xthuijs
Cisco Employee
Cisco Employee

hi mauricio, I dont have something pre-cut for that at the moment, but I know some sales guidance exists for it that explains the VSM+arbor solution for ddos mitigation.

There is a white paper that talks about the solution here, does that get you going?

cheers!

xander

Nicolas Fevrier
Cisco Employee
Cisco Employee

Hi Mauricio,

I suggest you watch the CiscoLive Berlin session "BRKSPG-3012 - SP Security - Leveraging BGP FlowSpec to protect your infrastructure (2016 Berlin)". 

You'll find recommandation for each type of attacks.

Also this one "BRKSPG-3335 - Advanced - DDoS Mitigation Deployment: Impact on your Architecture " still has a lot of good examples.

Cheers,

N.

satish.txt1
Beginner
Beginner

Do we need special router to run flowspacs? we have ASR1001-X does it support BGP flowspecs? 

also does ISP or provider need to enable flowspecs in their side to run flowspecs?

Nicolas Fevrier
Cisco Employee
Cisco Employee

Hi Satish,

ASR1001-X should support BGP FS as a client since IOS XE 3.15.

By nature, BGP is a peering relationship between two entities. So you need another end to control your ASR1000 and inject rules via BGP FS.

In short, it really depends what you are trying to do :)

BR,

N.

pvlrpm
Beginner
Beginner

 Hi Alexander,

   As per the RFC 5575 there are validation steps for the Flow Spec NLRI containing destination prefix component. What about the Flow Spec NLRIs that don't have the destination prefix component? Is there no validation for such kind of Flow Spec NLRIs? Please clarify.

Thanks,

Pavana.

xthuijs
Cisco Employee
Cisco Employee

hi pavana, it is basically for ebgp advertisements, it defines the "protocol" for advertising the flow rules to a peer based on the best path selection algorithm. if the dest prefix is missing it wouldn't match a "real" prefix, hence the flow spec rule is not advertised along since the best path doesnt match the flowspec rule.

eBGP usage of flowspec is topic of much discussion (that is me having a flowspec rule and passing it on to another AS for implementation at their borders) and today possibly not part of consideration (yet).

xander

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Recognize Your Peers
Quick Links