Buy or Renew
Buy or Renew
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Create a new blog
5264
Views
6
Helpful
0
Comments

BGP Flowspec implementation on Cisco 8000 platforms

yosef_manasseh
Cisco Employee
Cisco Employee
‎01-29-2020 01:21 PM

Feature Description

 

 


BGP flowspec in a nutshell is a feature that will allow you to receive IPv4/IPv6 traffic flow specification (source X, destination Y, protocol UDP, source port A .. etc) and actions that need to be taken on that traffic (drop, or police .. or redirect etc) via BGP update.
Inside the BGP update, the flowspec matching criteria is represented by BGP NLRI and the actions are represented by BGP extended communities.


This feature is based on RFC 5575 and can be used to mitigate against DDoS attack. When a certain host inside of a network is being attacked, we can send a flowspec update to edge routers so that attack traffic can be policed or dropped, or even redirected elsewhere, maybe to an appliance that can clean the traffic (filter out the bad traffic and forward only the good traffic toward the affected host).

Once flowspecs have been received by a router and programmed in applicable line cards, any active L3 ports on those line cards will start processing ingress traffic according to flowspec rules.
If needed, we can disable flowspec processing on specific ports of the LC via CLI config (discussed later).
Note also that flowspec will only affect ingress traffic, it won't interfere with egress direction.

Flowspec can be programmed on different kind of interfaces:

  1. regular interface, e.g. TenGigE0/0/0/0
  2. vlan interface, e.g. TenGigE0/0/0/0.1 , Bundle-Ether3.2
  3. bundle interface, e.g. Bundle-Ether3

Ingress traffic can be matched by many criteria as defined in the RFC.
We can define only one, some, or all criteria to match traffic.
Once a set of criteria in a flowspec is defined, then all criteria must match the packet for actions to take place.

 

BGP flowspec feature has been supported on ASR9K since XR 5.2.0, NCS5500 since XR 6.5.1, and Cisco 8000 platform supports the feature starting with 7.0.12 release.
Disclaimer:
7.0.12 might not be GA release.

 

Update 4/15/2021, this article will not be updated anymore going forward.
The information in this document is accurate as of IOS XR 7.0.12.
Newer version of IOS XR might have new features / enhancements that are not covered in this doc.
Whenever in doubt, please reach out to your friendly TAC team for assistance.

Available criteria:

     

How about a case where we receive multiple different flowspecs from controller matching the same traffic stream?
Say the controller sends us flowspec type1 (destination address), type2 (source address), and type3 (protocol) each in different update; and later we send a single traffic stream matching all three flowspec rules.
Which flowspec will match the traffic in hardware?
Answer:
The traffic will match the flowspec with lowest numeric type value.
In this case, traffic will match flowspec type 1 (because 1<2<3).
As a result, whatever action being defined along with flowspec type 1 will be taken.

Here's an example.
Say we receive following flowspecs from controller.

RP/0/RP0/CPU0:klaproject#sh flowspec ipv4 detail         
Wed Jan 29 10:22:33.807 UTC
 
AFI: IPv4          
  Flow           :Dest:51.1.1.1/32
    Actions      :Traffic-rate: 5000000 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :                   0/0                  
      Transmitted         :                   0/0                  
      Dropped             :                   0/0       
  Flow           :Source:14.84.0.1/32
    Actions      :Traffic-rate: 7000000 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :                   0/0                  
      Transmitted         :                   0/0                  
      Dropped             :                   0/0                  
  Flow           :Proto:=255
    Actions      :Traffic-rate: 2000000 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :                   0/0                  
      Transmitted         :                   0/0                  
      Dropped             :                   0/0     

Then we send traffic stream ingressing the router:

destination 51.1.1.1
source 14.84.0.1
proto: 255

This will result in the traffic matching flowspec type1 (destination address):

RP/0/RP0/CPU0:klaproject#sh flowspec ipv4 detail         
Wed Jan 29 10:30:19.188 UTC
 
AFI: IPv4
  Flow           :Dest:51.1.1.1/32
    Actions      :Traffic-rate: 5000000 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :                   8333868/2670601302                  
      Transmitted         :                   4612813/1402458454                 
      Dropped             :                   3721055/1268142848                 
  Flow           :Source:14.84.0.1/32
    Actions      :Traffic-rate: 7000000 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :                   0/0                  
      Transmitted         :                   0/0                  
      Dropped             :                   0/0                  
  Flow           :Proto:=255
    Actions      :Traffic-rate: 2000000 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :                   0/0                  
      Transmitted         :                   0/0                  
      Dropped             :                   0/0     



Available actions:

  • Drop
    Drop all traffic that matches the criteria.
    Feature availability:
    Supported in first release 7.0.12 (both IPv4 and IPv6).

  • Police
    Police all traffic that matches the criteria to a certain rate.
    Feature availability:
    Not supported in first release 7.0.12
    (earlier version of this doc mentioned that this is supported, but the support is taken off since)

  • Redirect VRF
    Redirect all traffic that matches the criteria to a different VRF.
    Feature availability:
    Not supported in first release 7.0.12.

  • Set DSCP
    Mark all traffic that matches the criteria with a certain DSCP value.
    Feature availability:
    Not supported in first release 7.0.12.

  • Redirect IPv4/IPv6 next-hop
    Redirect all traffic that matches the criteria to a different IP next-hop.
    Feature availability:
    Not supported in first release 7.0.12.

 

Handy support matrix table for reference:

 

Match Criteria

 

  Cisco 8000 platform NCS5500 platform ASR9K platform
Type 1 - Destination Prefix IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 2 - Source Prefix IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 3 - IP Protocol IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 4 - Port IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 5 - Destination port IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 6 - Source port IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 7 - ICMP type Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 8 - ICMP code Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 9 - TCP flags Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 10 - Packet length Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 11 - DSCP

IOS XR 7.0.12 for IPv4

Future release for IPv6

 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Type 12 - Fragment Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported

 

 

Actions

 

  Cisco 8000 platform NCS5500 platform ASR9K platform
Drop IOS XR 7.0.12
(both IPv4 and IPv6)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Police Future release
(earlier version of this doc mentioned that this is supported, but the support is taken off since)		 IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Redirect VRF Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Set DSCP Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported
Redirect IPv4/IPv6 next-hop Future release IOS XR 6.5.1
(both IPv4 and IPv6)		 Supported

 

 

 

We call the router that receives the BGP update as the "client", and the router/appliance that advertises the update as the "server" or "controller".
A router than runs IOS XR can function as either client or server, and even as both server/client at the same time. For instance, when the router also needs to take actions on the attack traffic in addition to propagating the flowspec rules to clients.

This article will not discuss further details about working of BGP flowspec, because we already have great documentations that does that already:

1. basic description of BGP flowspec on ASR9K:
https://supportforums.cisco.com/t5/service-providers-documents/asr9000-xr-understanding-bgp-flowspec-bgp-fs/ta-p/3139916

2. detailed description of BGP flowspec on ASR9K, complete with config examples.

Please refer to CiscoLive session presented in Las Vegas (2017).

https://www.youtube.com/watch?v=dbsNf8DcNRQ

 



Rather, we will focus more on specific information pertaining to Cisco 8000 platform.

This is also a live document, we will update if new info becomes available.




Supported Hardware

    
Since all variants of Cisco 8000 routers and LCs run the same Cisco Silicon One forwarding ASICs with on-chip High Bandwidth Memory (HBM), all variants will support flowspec out of the box, in all role (as server and/or client).



Supported Scale

The way that the HBM TCAM is carved allows up to 2,000 TCAM entries for BGP Flowspec, but some constraints that might cause flowspec not to be able to reach the 2,000 entries theoretical limit will apply:

  • This same HBM TCAM space is shared by other features like QoS and ACL.
  • When the flowspec rules are using multi-value ranges for BGP tuples then it will consume more TCAM cells and will reduce overall scale, for instance when specifying a range of destination port match instead of specific port match.
  • On 88xx modular-platform, IPv4 flowspec can go up to 2,000 with drop action.
    On 82xx fixed-platform, IPv4 flowspec can go up to 1,500 with drop action.
  • IPv6 flowspec can't go more than 980.
    This is because space for 1,000 entries are reserved for IPv6 flowspec and LPTS, and LPTS by default will consume up to 20 entries.
  • These constraints applies for first release 7.0.12 and might be removed at later release.

 

For first release 7.0.12, Cisco recommends the following combination for deployment (assuming we're not using multi-value ranges).
(Earlier version of this doc also listed recommended numbers for flowspec with police action. Since this action ends up not supported on first release, the content below has been updated)

  • on 82xx fixed-platform
    1400 IPv4 flowspec with drop action.
    300 IPv6 flowspec with drop action.
    for a total of 1,700 flowspecs.

  • on 88xx modular-platform
    1700 IPv4 flowspec with drop action.
    300 IPv6 flowspec with drop action.
    for a total of 2,000 flowspecs.


Config example
"attack traffic ingressing on interface in Global Routing Table (GRT)"

Use case:

Attack IPv4 and IPv6 traffic received on GRT interface.
Drop both IPv4 and IPv6 traffic.

 

Topology:

attack traffic
|
|
|
|
|
\/
GRT interface
Cisco 8000
|   |
|   |
|   +------ iBGP ----- flowspec server
|
|
\/
attacked host


Client config

 

... activate flowspec programming in the LC

flowspec
 local-install interface-all
!
        
... optional, disable flowspec processing on specific ingress interfaces.
In case disablement is desired, we need to disable both address-families, cisco 8000 platform doesn't support disablement on IPv4 or IPv6 only.

interface Bundle-Ether3.1
 ipv4 flowspec disable
 ipv6 flowspec disable
!
        
... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!

route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec server
    
router bgp <>
 nsr
 bgp router-id <>
 address-family ipv4 flowspec
 !
 address-family ipv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family ipv4 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  address-family ipv6 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  update-source <>
 !
!



Server Config

... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!

route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec client

router bgp <>
 address-family ipv4 flowspec
 !
 address-family ipv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family ipv4 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
  address-family ipv6 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
 !
!

... let's configure flowspecs to be advertised to client

...... address-family ipv4 flowspec

    class-map type traffic match-all ipv4_attack
     match destination-address ipv4 70.2.1.1 255.255.255.255
     match source-address ipv4 80.2.1.1 255.255.255.255
     end-class-map
    !

    policy-map type pbr scale_ipv4
     class type traffic ipv4_attack
      drop
     !
     class type traffic class-default
     !
     end-policy-map
    !

    flowspec
     address-family ipv4
      service-policy type pbr scale_ipv4
     !
    !

...... address-family ipv6 flowspec

    class-map type traffic match-all ipv6_tcp
     match destination-address ipv6 70:1:1::5a/128
     match source-address ipv6 80:1:1::5a/128
     match protocol tcp
     match destination-port 22
     match source-port 4000
     end-class-map
    !

    policy-map type pbr scale_ipv6
     class type traffic ipv6_tcp
      drop
      !
     !
     class type traffic class-default
     !
    !

    flowspec
     address-family ipv6
      service-policy type pbr scale_ipv6
     !
    !

 



Config example
"attack traffic ingressing on VRF interface"

Use case:


Attack traffic received on VRF "customer_1" interface.
Drop this traffic.



Topology:

attack traffic
|
|
|
|
|
\/
VRF "customer_1" interface
Cisco 8000
|   |
|   |
|   +------ iBGP ----- flowspec server
|
|
\/
attacked host (VRF "customer_1")


Client Config

... activate flowspec programming in the LC

flowspec
 local-install interface-all
!
        
... optional, disable flowspec processing on specific ingress interfaces.
In case disablement is desired, we need to disable both address-families, cisco 8000 platform doesn't support disablement on IPv4 or IPv6 only.

interface Bundle-Ether3.1
 ipv4 flowspec disable
 ipv6 flowspec disable
!
        
... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!

route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec server
    
router bgp <>
 nsr
 bgp router-id <>
 address-family vpnv4 unicast
 !
 address-family vpnv6 unicast
 !
 address-family vpnv4 flowspec
 !
 address-family vpnv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family vpnv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv6 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv4 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  address-family vpnv6 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  update-source <>
 !
 vrf customer_1
  rd auto
  address-family ipv4 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv6 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv4 flowspec
  !
  address-family ipv6 flowspec
  !
 !
!

... define VRF "customer_1"

vrf customer_1
 address-family ipv4 unicast
  import route-target
   4787:1313
  !
  export route-target
   4787:1313
  !
 !
 address-family ipv4 flowspec
  import route-target
   4787:1313
  !
  export route-target
   4787:1313
  !
 !
 address-family ipv6 unicast
  import route-target
   4787:1313
  !
  export route-target
   4787:1313
  !
 !
 address-family ipv6 flowspec
  import route-target
   4787:1313
  !       
  export route-target
   4787:1313
  !
 !
!


Server Config

... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!
route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec client

router bgp <>
 nsr
 bgp router-id <>
 address-family vpnv4 unicast
 !
 address-family vpnv6 unicast
 !
 address-family vpnv4 flowspec
 !
 address-family vpnv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family vpnv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv6 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv4 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv6 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
  update-source <>
 !
 vrf customer_1
  rd auto
  address-family ipv4 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv6 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv4 flowspec
  !
  address-family ipv6 flowspec
  !
 !
!

... let's configure flowspecs to be advertised to client



...... address-family vpnv4 flowspec

    class-map type traffic match-all l3vpn_ipv4_attack
     match destination-address ipv4 202.158.3.2 255.255.255.255
     match source-address ipv4 202.158.1.2 255.255.255.255
     end-class-map
    !

    policy-map type pbr pm_cust_VRF
     class type traffic l3vpn_ipv4_attack
      drop
     !
     class type traffic class-default
     !
     end-policy-map
    !

    flowspec
     vrf customer_1
      address-family ipv4
       service-policy type pbr pm_cust_VRF
     !
    !
    
...... address-family vpnv6 flowspec

    class-map type traffic match-all l3vpn_ipv6_attack
     match destination-address ipv6 200:158:3::2/128
     match source-address ipv6 200:158:1::2/128
     match protocol tcp
     match destination-port 22
     match source-port 4000
     end-class-map
    !

    policy-map type pbr pm_cust_VRF_ipv6
     class type traffic l3vpn_ipv6_attack
      drop
     !
     class type traffic class-default
     !
     end-policy-map
    !

    flowspec
     vrf customer_1
      address-family ipv6
       service-policy type pbr pm_cust_VRF_ipv6
     !
    !

 

 

 

Config example
XR platforms as flowspec controllers

 

More detailed config examples to define packet matching criteria (class-maps) when using XR platforms as flowspec controller can be found here:

https://supportforums.cisco.com/t5/service-providers-blogs/ncs5500-bgp-flowspec-packet-matching-criteria/ba-p/3387457

 

 


BGP Flowspec and IPv6 BGP neighbors

 

The use-cases and config examples provided previously are when we have BGP neighbors that runs on IPv4.
What about if we have IPv6 BGP neighbors instead of IPv4?
We can still advertise flowspec to that neighbor, but only for address-family "ipv6 flowspec".

The following flowspec address-families are not supported:

  • ipv4 flowspec
  • vpnv4 flowspec
  • vpnv6 flowspec

This is by design and same limitation applies to other platforms that run IOS-XR like ASR9000 and NCS5500.

 



BGP Flowspec and BGP route-policy

We can configure BGP route-policy to filter flowspec updates.
One use case is maybe to limit flowspec updates with only specific subnet masks.

For instance:

router bgp <>
!
address-family ipv4 flowspec
!
address-family ipv6 flowspec
!
neighbor <>
  remote-as <>
  update-source <>
  address-family ipv4 flowspec
   route-policy routes_flowspec in
   route-policy drop-all out
  !
  ...

route-policy routes_flowspec
  if destination in ge_17 then
    pass
  endif
end-policy

prefix-set ge_17
  0.0.0.0/0 ge 17
end-set
!

Just be careful when doing route-policy.
If you commit the above config, and the coming flowspec updates don't have Type 1 (Destination address) criteria, then no updates would match the policy and it would be implicit deny for all flowspec updates.

 

 

BGP Flowspec and ACL

When flowspec is implemented on an interface that is also having ingress ACL, ACL will come before flowspec processing.

 

  • When the ACL is permitting the traffic:
    Flowspec will take whatever passed by ACL and run flowspec processing.
  • When the ACL is denying the traffic:
    Flowspec will not process any traffic since ACL has discarded them.

 

 

 BGP Flowspec and local QoS configuration

When flowspec is implemented on an interface that is also having local QoS configuration, local config will come before flowspec processing.
Local config will police and dscp-mark the packets and pass them to flowspec.
Flowspec will then do its processing (police).

Flowspec will retain dscp marking as dictated by local qos config.

Say, we have the following:

inbound qos config : police 100Mbps, mark dscp af11
 
=============================================================
 
ipv4 access-list acl_ipv4_qos_stream
6 permit ipv4 any host 200.255.5.2
!
!
class-map match-any cm_ipv4_qos_stream
match access-group ipv4 acl_ipv4_qos_stream
 end-class-map
!
 
policy-map pm_ipv4_qos_stream
class cm_ipv4_qos_stream
  police rate 100 mbps
  !
  set dscp af11
!
 class class-default
!
 end-policy-map
!
 
interface hundredGigE 0/4/0/35
service-policy input pm_ipv4_qos_stream
 
=============================================================

Then we receive the following in flowspec advertisement.
flowspec config : police 50Mbps.
(Note that this info is not applicable for first release 7.0.12, this is because flowspec with police action is only supported in future release).
 
=============================================================
 
RP/0/RP0/CPU0:fretta-50#sh flowspec ipv4 detail | b 200.255.5.2
  Flow           :Dest:200.255.5.2/32
    Actions      :Traffic-rate: 50000000 bps (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :           116570713/12822778430        
      Transmitted         :            57360817/6309689870         
      Dropped             :            59209896/6513088560   
 
=============================================================
 
Then the outcome will be:

 

  1. traffic will be policed by flowspec at 50Mbps.
  2. flowspec will not overwrite dscp marking, traffic will be forwarded using dscp af11.

 

 


BGP Flowspec and NSR


NSR RP switchover is hitless for flowspec provided all underlay protocol (BGP, ISIS, OSPF, etc) has been configured for NSR.

 


Caveats


BGP flowspec will NOT process packets when it's received on GRE tunnel.

BGP flowspec is NOT supported on BVI interface.

BGP flowspec is NOT supported with multicast traffic.

BGP flowspec polices traffic at L2, not L1.
What this means is that the policing will not take into account L1 headers.
So, say we police at 500Mbps, then what being transmitted on the wire would be more than 500Mbps since the traffic will be transmitted with L1 headers on top of it.
(Note that this info is not applicable for first release 7.0.12, this is because flowspec with police action is only supported in future release).

Once received via BGP update, flowspec will take longer time to program to the LC if we compare with regular IPv4/IPv6 route updates.
Enhancement is being planned to make the programming faster, but it will only be provided after 7.0.12.

 

 


Related Show Commands

 

The following commands are captured from client side.

=============================================================

 

RP/0/RP0/CPU0:klaproject#sh bgp ipv4 flowspec
Wed Jan 29 11:32:48.903 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 2703
BGP main routing table version 2703
BGP NSR Initial initsync version 0 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>iDest:51.1.1.1/32,Source:61.1.1.1/32,Proto:=6,DPort:=80,SPort:=3000,DSCP:=10/200
0.0.0.0 100 0 i

...


          
=============================================================
          

RP/0/RP0/CPU0:klaproject#sh bgp ipv6 flowspec
Wed Jan 29 11:34:03.270 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 303
BGP main routing table version 303
BGP NSR Initial initsync version 0 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>iDest:56:1:1::1/0-128,Source:66:1:1::1/0-128,NH:=6,DPort:=194,SPort:=8500/384
:: 100 0 i

....


          
=============================================================
          

RP/0/RP0/CPU0:klaproject#sh bgp vpnv4 flowspec
Wed Jan 29 11:36:02.648 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 56507
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 202.158.0.1:1 (default for vrf customer_2001)
*>iDest:55.1.1.1/32,Source:65.1.1.1/32,Proto:=17,DPort:=108,SPort:=7000/176
0.0.0.0 100 0 i

...


          
=============================================================
          

RP/0/RP0/CPU0:klaproject#sh bgp vpnv6 flowspec
Wed Jan 29 11:37:14.642 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 1803
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 202.158.0.1:1 (default for vrf customer_2001)
*>iDest:52:1:1::1/0-128,Source:62:1:1::1/0-128,NH:=17,DPort:=69,SPort:=6000/384
0.0.0.0 100 0 i

...


          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh bgp ipv4 flowspec summary
Wed Jan 29 11:43:48.324 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 2703
BGP main routing table version 2703
BGP NSR Initial initsync version 0 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 2703 2703 2703 2703 2703 0

Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
202.158.0.3 0 4787 330040 329911 0 0 0 1d01h Idle (Admin)
202.158.0.4 0 4787 9643 9583 2703 0 0 01:24:09 900 <-- this many flowspecs were received from server

RP/0/RP0/CPU0:klaproject#
          
=============================================================

RP/0/RP0/CPU0:klaproject#sh bgp ipv6 flowspec summary
Wed Jan 29 11:54:55.845 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 303
BGP main routing table version 303
BGP NSR Initial initsync version 0 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 303 303 303 303 303 0

Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
202.158.0.3 0 4787 330040 329911 0 0 0 1d01h Idle (Admin)
202.158.0.4 0 4787 10311 10251 303 0 0 01:35:17 100 <-- this many flowspecs were received from server
202:158::4 0 4787 0 0 0 0 0 00:00:00 Idle (Admin)

RP/0/RP0/CPU0:klaproject#


          
=============================================================

RP/0/RP0/CPU0:klaproject#sh bgp vpnv4 flowspec summary
Wed Jan 29 11:54:13.773 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 56507
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 56507 56507 56507 56507 56507 0

Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
202.158.0.3 0 4787 330040 329911 0 0 0 1d01h Idle (Admin)
202.158.0.4 0 4787 10269 10209 56507 0 0 01:34:35 100 <-- this many flowspecs were received from server

RP/0/RP0/CPU0:klaproject#


          
=============================================================

RP/0/RP0/CPU0:klaproject#sh bgp vpnv6 flowspec summary
Wed Jan 29 11:56:59.829 UTC
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 1803
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 1803 1803 1803 1803 1803 0

Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
202.158.0.3 0 4787 330040 329911 0 0 0 1d01h Idle (Admin)
202.158.0.4 0 4787 10435 10375 1803 0 0 01:37:21 900 <-- this many flowspecs were received from server

RP/0/RP0/CPU0:klaproject#


          
=============================================================

RP/0/RP0/CPU0:klaproject#sh flowspec ipv4 detail
Wed Jan 29 11:58:03.665 UTC

AFI: IPv4
Flow :Dest:51.1.1.1/32,Source:61.1.1.1/32,Proto:=6,DPort:=80,SPort:=3000,DSCP:=10
Actions :Traffic-rate: 0 bps (bgp.1)
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

...


          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec ipv4 internal
Wed Jan 29 11:58:59.926 UTC

AFI: IPv4
Flow :Dest:51.1.1.1/32,Source:61.1.1.1/32,Proto:=6,DPort:=80,SPort:=3000,DSCP:=10
Actions :Traffic-rate: 0 bps (bgp.1)
Client Version: 0
Local: FALSE <--- this is flowspec advertised from server
Unsupported: FALSE <--- flowspec data is supported
RT:
VRF Name Cfg: 0x00
RT Cfg: 0x00
RT Registered: 0x00
RT Resolved: 0x00
Class handles:
Handle [0]: 3000000076017d25
Class Handle Version: 1
Sequence: 4096
Match Unsupported: None <--- flowspec data is supported
Synced: FALSE
Ref Count: 1
Last Error: 0:Success <--- no error is seen
Last Batch: 7207
Time Init: Jan 29 10:22:33
Time iClass Update: Jan 29 10:22:33
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

 ...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec ipv6 detail
Wed Jan 29 12:49:46.211 UTC

AFI: IPv6
Flow :Dest:56:1:1::1/0-128,Source:66:1:1::1/0-128,NH:=6,DPort:=194,SPort:=8500
Actions :Traffic-rate: 80000000 bps (bgp.1)
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec ipv6 internal
Wed Jan 29 12:50:34.410 UTC

AFI: IPv6
Flow :Dest:56:1:1::1/0-128,Source:66:1:1::1/0-128,NH:=6,DPort:=194,SPort:=8500
Actions :Traffic-rate: 80000000 bps (bgp.1)
Client Version: 0
Local: FALSE <--- this is flowspec advertised from server
Unsupported: FALSE <--- flowspec data is supported
RT:
VRF Name Cfg: 0x00
RT Cfg: 0x00
RT Registered: 0x00
RT Resolved: 0x00
Class handles:
Handle [0]: 300000007601e3dd
Class Handle Version: 1
Sequence: 4096
Match Unsupported: None <--- flowspec data is supported
Synced: FALSE
Ref Count: 1
Last Error: 0:Success <--- no error is seen
Last Batch: 7733
Time Init: Jan 29 10:25:09
Time iClass Update: Jan 29 10:25:09
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

...        
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec vrf customer_2001 ipv4 detail
Wed Jan 29 12:54:09.509 UTC

VRF: customer_2001 AFI: IPv4
Flow :Dest:55.1.1.1/32,Source:65.1.1.1/32,Proto:=17,DPort:=108,SPort:=7000
Actions :Traffic-rate: 80000000 bps (bgp.1)
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

... 
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec vrf customer_2001 ipv4 internal
Wed Jan 29 12:54:59.561 UTC

VRF: customer_2001 AFI: IPv4
Flow :Dest:55.1.1.1/32,Source:65.1.1.1/32,Proto:=17,DPort:=108,SPort:=7000
Actions :Traffic-rate: 80000000 bps (bgp.1)
Client Version: 0
Local: FALSE <--- this is flowspec advertised from server
Unsupported: FALSE <--- flowspec data is supported
RT:
VRF Name Cfg: 0x00
RT Cfg: 0x00
RT Registered: 0x00
RT Resolved: 0x00
Class handles:
Handle [0]: 3000000076017aec
Class Handle Version: 1
Sequence: 4096
Match Unsupported: None <--- flowspec data is supported
Synced: FALSE
Ref Count: 1
Last Error: 0:Success <--- no error is seen
Last Batch: 7196
Time Init: Jan 29 10:22:24
Time iClass Update: Jan 29 10:22:24
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec vrf customer_2001 ipv6 detail
Wed Jan 29 12:56:54.106 UTC

VRF: customer_2001 AFI: IPv6
Flow :Dest:52:1:1::1/0-128,Source:62:1:1::1/0-128,NH:=17,DPort:=69,SPort:=6000
Actions :Traffic-rate: 0 bps (bgp.1)
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0

...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec vrf customer_2001 ipv6 internal
Wed Jan 29 12:57:59.460 UTC

VRF: customer_2001 AFI: IPv6
Flow :Dest:52:1:1::1/0-128,Source:62:1:1::1/0-128,NH:=17,DPort:=69,SPort:=6000
Actions :Traffic-rate: 0 bps (bgp.1)
Client Version: 0
Local: FALSE <--- this is flowspec advertised from server
Unsupported: FALSE <--- flowspec data is supported
RT:
VRF Name Cfg: 0x00
RT Cfg: 0x00
RT Registered: 0x00
RT Resolved: 0x00
Class handles:
Handle [0]: 3000000076017d44
Class Handle Version: 1
Sequence: 256
Match Unsupported: None <--- flowspec data is supported
Synced: FALSE
Ref Count: 1
Last Error: 0:Success <--- no error is seen
Last Batch: 7208
Time Init: Jan 29 10:22:34
Time iClass Update: Jan 29 10:22:34
Statistics (packets/bytes)
Matched : 0/0
Transmitted : 0/0
Dropped : 0/0
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec ipv4 nlri
Wed Jan 29 12:59:42.384 UTC

AFI: IPv4
NLRI (hex) :0x01203301010102203d01010103810605815006910bb80b810a
Actions :Traffic-rate: 0 bps (bgp.1)

...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec ipv6 nlri
Wed Jan 29 13:00:22.041 UTC

AFI: IPv6
NLRI (hex) :0x01800000560001000100000000000000000001028000006600010001000000000000000000010381060581c206912134
Actions :Traffic-rate: 80000000 bps (bgp.1)

...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec vrf customer_2001 ipv4 nlri
Wed Jan 29 13:01:19.624 UTC

VRF: customer_2001 AFI: IPv4
NLRI (hex) :0x01203701010102204101010103811105816c06911b58
Actions :Traffic-rate: 80000000 bps (bgp.1)

...
          
=============================================================

 

RP/0/RP0/CPU0:klaproject#sh flowspec vrf customer_2001 ipv6 nlri
Wed Jan 29 13:02:09.521 UTC

VRF: customer_2001 AFI: IPv6
NLRI (hex) :0x018000005200010001000000000000000000010280000062000100010000000000000000000103811105814506911770
Actions :Traffic-rate: 0 bps (bgp.1)


=============================================================

 

RP/0/RP0/CPU0:klaproject#sh policy-map transient type pbr
Wed Jan 29 13:02:50.211 UTC
...
policy-map type pbr __bgpfs_default_IPv4
handle:0x36000006
table description: L3 IPv4 and IPv6
class handle:0x76017d25 sequence 4096
match destination-address ipv4 51.1.1.1 255.255.255.255
match source-address ipv4 61.1.1.1 255.255.255.255
match protocol tcp
match destination-port 80
match source-port 3000
match dscp 10
drop
!

...
          
=============================================================

 

 

 


Logs to provide to Cisco TAC for BGP flowspec related issues on Cisco 8000 platform


Gather the following set of logs from client router.

Replace "NAME_OF_ROUTER"with the name of your router.

 

logs to grab:

 

  1. Timestamp when the problem occurs (e.g. 16:25:15.095 GMT-7 Fri Dec 15 2017), the more exact, the better.
    It's best if the timestamp can be copied from a specific line of "show log" output.
  2. show tech flowspec file harddisk:/NAME_OF_ROUTER_sh_tech_flowspec
  3. show tech platform-fwd file harddisk:/NAME_OF_ROUTER_sh_tech_platf_fwd
  4. show tech pbr file harddisk:/NAME_OF_ROUTER_sh_tech_pbr
  5. show dpa objects pbr base location <ingress LC> | file harddisk:/NAME_OF_ROUTER_sh_dpa_obj_pbr_base_loca_ingress_lc.txt
  6. show pbr trace ea all location <ingress LC> | file harddisk:/NAME_OF_ROUTER_sh_pbr_trace_ea_all_loca_ingress_lc.txt
  7. show ofa trace loca <ingress LC> | file harddisk:/NAME_OF_ROUTER_sh_ofa_trace_location_ingress_lc.txt
  8. show log | file harddisk:/NAME_OF_ROUTER_show_log.txt
    (showing the events when the problem occurs)

 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Popular Articles
Customers Also Viewed These Support Documents