cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
75747
Views
21
Helpful
73
Comments
xthuijs
Cisco Employee
Cisco Employee

Introduction

IOS devices have the concept of control plane policing. IOS-XR doesn't use that concept but instead uses a very comprehensive and powerful Local Packet Transport Services. In this document it is explained how LPTS works and how you can work with it, monitor and verify it.

Core Issue

LPTS is the concept of reflexive ACL's, punt policers and has an "internal" FIB or iFIB that directs certain packets to various nodes. IOS-XR can handle certain traffic on the linecard (such as BFD, Netflow and ARP) and these packets are instructed by LPTS to be handled by the local CPU rather then the RSP CPU.

At the same time, there are ACL's in place that allow for instance the punting of Telnet traffic and then per host if configured so, but another component of LPTS called MPP, the Management Plane Protection.

Generally, the default values for LPTS provide the level of protection you are after. However there are some rare circumstances whereby you want to tune the values of LPTS in order to get the service levels you need. LPTS is very dynamic in nature and pierces holes into the protection system as particular items are configured.

The LPTS policers work on a per NP basis. So if the LPTS police value is set to 1000pps that means that every NP on the LC can punt with 1000pps to the RSP CPU or LC CPU. This is something to take into consideration when evaluating the A9K-8T-x cards who have 8 NPU's per LC.

Take extreme care when changing the LPTS policer values.

High level overview

From a birds eye view, LPTS looks like this:

overview.jpg

The NPU has a table that tells it where to send packet to (LC or RSP) as part of the "internal FIB" or iFIB. These packets are punted at a pre-defined rate, they can be tuned in XR release 4.x and later. Also in the TCAM which is used in the ASR9K for ACLs (amongst others), are lists that define which packets we want to allow and not. This will be discussed in the MPP section of this document.

LPTS is composed of a (set of) dynamic ACL's (which are created as part of user configuration dynamically, or automitcally inserted as peerings establish), an internal "routing table" (iFIB) and a set of policers for different punt reasons.

LPTS Characteristics

0) LPTS has Hardware policers on line cards to limit traffic sent to local or remote nodes
0) LPTS entries in TCAM classifies packets to select a policer to apply
0) The policer value can be tuned to 0 (to drop all packet matching classification criteria)
0) Polices on protocol (BGP, OSPF, SSH) and flow state (BGP established, BGP configured, and BGP listen)
0) Policing done on the LC Hardware ASIC before packets hit RP/LC CPU
0) Filters are automatically and dynamically installed by the IOS XR infrastructure

So for-me packets are undergoing the Pre iFIB classification and policing upon which they are directed by the iFIB, which is the second level of filtering to the destination node.

LPTS Firewalling

One of the great strenghts with LPTS is the dynamic ACL creation. This is configuration driven and no user intervention is required.

In addition to that, LPTS has different flow categories based on the state of the protocol. For instance, BGP has 3 different states:

  1. Unknown
  2. Configured
  3. Established

Unknown is the flow whereby we have TCP port 179 traffic, but we have no neighbor configured from that source. Policed very heavily.

Configured is the entry whereby we know the source address of the peer, but the session is not yet established (no known source port from the peer), Policed moderately.

Established is where we have all the L3 and L4 data from the session. Lightly policed.

The entries for configured is driven by the configuration of the neighbor statement under the router BGP section.

Established is dynamically inserted when the peer establishes.

You could theoretically police the unknown to a rate of zero.

Example:

Router bgp

   neighbor 192.168.1.1

   …

!

The following table can be seen with the output of the command:

show lpts pifib hardware entry brief loc 0/3/cpu0 | i 179

LocalPortRemotePortRateState
any179ANYANY100unknown
any179192.168.1.1ANY1,000configured
192.168.1.2179192.168.1.1222310,000established

If you use the command

RP/0/RSP0/CPU0:A9K-TOP#show lpts pifib hardware entry location 0/3/CPU0 | be 33.33.1  

You can check the detailed entry of the PiFIB (policer)

Source IP         : 33.33.1.1                        the remote address

Is Fragment       : 0                                fragments allowed

Interface         : any                              expected source interface

M/L/T/F           : 0/IPv4_STACK/0/BGP-known

DestNode          : 48                               where the packets are sent to

DestAddr          : 48

L4 Protocol       : TCP

TCP flag byte     : any                              additional security checks at TCP level

Source port       : Port:179

Destination Port  : 11293

Accepted/Dropped  : 117866/0                         packets accepted and denied

# of TCAM entries : 1                                number of tcam entries burnt for this PiFIB entry

HPo/HAr/HBu/Cir   : 1924676/2500pps/2500ms/2500pps

State             : Entry in TCAM                    status of the entry


Configuring LPTS police rates

You can configure the LPTS Policers on a PiFIB bases and also the punt policers can be adjusted.

The following commands apply. Note that this is on a per linecard basis. All NPU's on that linecard will get reconfigured.

RP/0/RSP0/CPU0:A9K-BNG(config)#lpts punt police location 0/0/CPU0 protocol ?

  arp              ARP packets

  bfd              Bidirectional Forwarding Detection packets

  cdp              Cisco Discovery Protocol packets

  cfm              Connectivity Fault Management Protocol packets

  cgmp             Cisco Group Management Protocol packets

  dhcp             Dynamic Host Configuration Protocol packets

  efm              Ethernet in the First Mile Protocol packets

  igmp-snoop       Internet Group Management Protocol Snoop packets

  ipiw-arp         L2VPN IPIW ARP packets

  ipv4             IPv4 packets

  ipv6             IPv6 packets

  lacp             Bundle Protocol packets

  mofrr            Multicast-only FRR packets

  mpls             MPLS punt packets

  mstp             Multiple Spanning Tree Protocol packets

  mvrp             Multiple VLAN Registration Protocol packets

  ppp              Point-to-Point Protocol packets

  pppoe            Point-to-Point Protocol over Ethernet packets

  rarp             Reverse ARP packets

  vccv             Virtual Circuit Connection Verification packets

  vidmon           Video Monitoring packets

  vidmon-flow-add  Video Monitoring flow add packets

Exception packets can be reconfigured by the following command: lpts punt police location 0/0/CPU0 exception

Glean adjacency or ACL-deny packets can be tuned for instance via that command.

The PIFIB can be reconfigured via the following commands:

RP/0/RSP0/CPU0:A9K-BNG(config)#lpts pifib hardware ...

  • In there you can enter the linecard you wish to specifically reconfigure
  • The policer flow values
  • And the TCAM entries (this is new in XR420)   
    • As you've seen LPTS can dynamically create "ACL" entries for dynamic firewalling and for MPP. This command limits the number of TCAM entries that LPTS can use so that space is available for other purposes such as regular ACL's, QOS matching, EFP matching etc.

LPTS static-police and police differences

The command “police” is used to check policer values, accept/drop counts for packets matching LPTS TCAM(mostly L3 packets) entries whereas “static-police” is used to check policer values.

Accept/drop counts for packets matching static punt reasons programmed in search structures (Mostly L2 and exception packets).

policer” is for dynamic flows (like BGP, OSPF, etc protocols directed  by RSP)

static-policer” is for pseudo Static flows (like BFD, CFM directed by the LC) These are hard-coded and include Exception processing packets.

There is a CLI to change few of the exception processing as well (for e.g. ICMP unreachable)

Monitoring LPTS

LPTS is not SNMP enabled (request has been filed and is in the works, no target release defined at time of writing). Though there are very inventive ways to monitor LPTS and generate alerts. There is a TCL script that you can use with EEM in order to get some level of alerting.

Attached to this article is the script package and here is how you set it up:

event manager environment EEM_LPTS_CHECK_INTERVAL 300

event manager environment EEM_LPTS_CHECK_FLOWTYPES BGP-known *

event manager environment EEM_LPTS_CHECK_LOCATIONS 0/0/CPU0 0/4/CPU0

event manager environment EEM_LPTS_CHECK_THRESHOLD 1 50%

event manager directory user policy disk0:/scripts/

event manager policy lpts-threshold-alerting.tcl username scripts

How to clear LPTS statistics

LPTS stats cannot be cleared by LPTS commands or qos counter clearing.

You can clear LPTS stats by clearing hte np controller stats:

"clear controllers np counters all location <>”

MPP: Managed Plane Protection

In the standard configuration all interfaces have access to the Telnet, SSH and SNMP daemons.

Inband vs Out of band

All linecard interfaces are designated to be inband, meaning they can transport user traffic as well as management traffic.

The mgmt interfaces on the RSP are designated out of band. This means that they can't transport user traffic but only management traffic.

Out-of-band interfaces can't "speak" to other interfaces as they are desginated for managment traffic. So eventhough there is a route in the system that would send traffic out of the mgmt interface, Fabric enabled interfaces on the LC can't

Here an example of out of band and the restrictions that it imposes on the forwarding

oob.jpg

Configuring MPP

By default when the service is configured, there are no mpp restrictions. All interfaces are able to accept the mgmt traffic for the service you defined. For example, when the telnet server is configured, LPTS reports the following binding:

RP/0/RSP0/CPU0:A9K-BNG#show lpts bindings brief | i (any.23 )

Tue Feb 28 12:00:55.195 EDT

0/RSP0/CPU0 TCP  LR IPV4 TCP    default   any          any,23 any

This means that every for me packet with port 23 as the dest port will get serviced.

Now when configuring MPP the bindings output changes:

control-plane

management-plane

  inband

   interface TenGigE0/1/0/0

    allow Telnet peer

     address ipv4 3.3.3.3

     address ipv4 5.5.5.0/28

    !

   !

   interface GigabitEthernet0/0/0/10

    allow Telnet

   !

  !

!

In this configuration example I am designating two interfaces as inband, so they will still be able to forward transient traffic and allow inbound telnet traffic. At the same time I allow telnet from any host on Gig0/0/0/10 and only telnet from a few peers on Te0/1/0/0.

The LPTS bindings are dynamically changed as per following output:

RP/0/RSP0/CPU0:A9K-BNG#show lpts bindings brief | i (any.23 )

Tue Feb 28 12:06:48.339 EDT

0/RSP0/CPU0 TCP  LR IPV4 TCP    default   Gi0/0/0/10   any,23 any                 << Any source can access my telnet on this intf

0/RSP0/CPU0 TCP  LR IPV4 TCP    default   Mg0/RSP0/CPU0/0 any,23 any            << Dedicated inband

0/RSP0/CPU0 TCP  LR IPV4 TCP    default   Te0/1/0/0    any,23 3.3.3.3          << /32 host access for telnet on dedicated intf

0/RSP0/CPU0 TCP  LR IPV4 TCP    default   Te0/1/0/0    any,23 5.5.5.0/28    << Hosts from this subnet on this intf

Powerful eh!?!

We can also look at the pre internal fib (piFIB) and check the entries there:

RP/0/RSP0/CPU0:A9K-BNG#show lpts pifib hardware entry bri location 0/1/cpu0 | i (.23 )

Tue Feb 28 12:27:46.389 EDT

7      IPV4 default      TCP    Te0/1/0/0       LU(48) any,23 3.3.3.3,any

10     IPV4 default      TCP    Te0/1/0/0       LU(48) any,23 5.5.5.0/28,any


Decoding the Destnode in LPTS entries

In the example above you see the following detail: LU(48). This section explains that number and detail.

The LU means local unicast fabric. The 48 is a very interesting number.

The device that this output is taken from is an ASR9010. Which has 8 LC slots and 2 RSP slots. On both sides of the RSP's in the middle

are the 4 LC's

If I were to decode the 30 into binary it looks like this:

+---+---+---+---+---+---+---+---+

| 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |     Position

+---+---+---+---+---+---+---+---+

| 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |     Bit value for Decimal 48

+---+---+---+---+---+---+---+---+

|LC |LC |RSP|RSP|LC |LC |LC |LC |     Slot position filling (note 2 left most LC's not drawn)

+---+---+---+---+---+---+---+---+

Now you can see that the 1's are in position 5 and 4, and if you look at the slot numbering of the ASR9006, you can see that these are

the RSP's!! So telnet is delivered to the RSP.

Restrictions for MPP

1) Currently MPP doesn’t keep track of the denied or dropped protocol requests
2) The management protocols need to be enabled explicitly
–MPP configuration doesn’t enable the protocol services.
–MPP is only responsible for making the services available on different interfaces
3) Management requests received on in-band interfaces may not be acknowledged on in-band interfaces
4) RSP Management Ethernet interfaces are by default out-of-band interfaces
5) MPP configuration changes doesn’t affect active sessions established before the changes

6) No MIB Support

Related Information

Cisco Guide to Harden Cisco IOS XR Devices

LPTS Considerations

If you can use only p2p OSPF network type

flow ospf-uc-known rate 0

flow ospf-uc-default rate 0

Note that OSPF p2p network type is the recommended setting even on Ethernet interfaces unless you have multiple routers on the same segment.
Do we really need BGP, LDP-TCP, MSDP, default – for unconfigured sessions

flow bgp-default rate 0

flow ldp-tcp-default rate 0

flow msdp-default rate 0

Xander Thuijs, CCIE #6775

Sr. Tech Lead ASR9000


Comments
Aleksey Pyankov
Level 1
Level 1

Hi.

Is Fragment       : 0  

 means there is no fragmented packets allowed or we allow them?

We use Cacti to monitor ASR by snmp. And when Cacti did bulk.SNMP request at 100 OID - server fragment it and ASR receive this packets fragmented. And we think it drops them.

xthuijs
Cisco Employee
Cisco Employee

hi aleksey,

this flag identifies whether frags are allowed or not. the issue is with a port level filter, subsequent frags are hard to identify as part of the flow since subsequent frags do not have the port level info anymore.

you could do a quick debug snmp packets and see if the request is recevied from that mgmt station.

reassembly would happen in netio, so before it is handed to udp process and subsequently snmp.

cheers

xander

Aleksey Pyankov
Level 1
Level 1

Thank you. Is there is a way to set "Is Fragment       : 0" flag to "1"(enable)?  

xthuijs
Cisco Employee
Cisco Employee

It is a system parameter that is not configurable.

it is a bit dangerous also because you effectively open everything up towards a particular destination.

best is to fix the packet fragment from the mgmt station...

cheers

xander

Aleksey Pyankov
Level 1
Level 1

Thanks again! I'l try. 

 

netops044
Level 1
Level 1

Hi Xander,

What is the primary difference between the protocol-Default and protocol-known in the Flow type of the LPTS? Applying the ACL and rate limiting it does not work. Please guide us with in-depth understanding on this. 

In MPP, the IP address is not accepting with subnet length and shows error 'Stray bits in prefix-length of address' in router running version 5.2.0. Is this is still a bug CSCts11467 which is being hit.

Warm Regards.

xthuijs
Cisco Employee
Cisco Employee

see also cisco live id 2904 sanfran 2014 with some extensive detail on lpts.

-default is hit for the protocol for which we have no explicit peer configured.

-known would be hit in case we have say an ntp server defined, an ssh client, bgp neighbor etc.

all non configured specific peers will hit the -default rate.

xander

aditya.nugraha
Community Member

Hi Xander,

according to this link :

https://supportforums.cisco.com/discussion/12270901/block-traceroute-through-ios-xr

i wanna block icmp ttl-expired, with setting rate limit lpts policer for specific icmp ttl-expired to zero, and let another icmp type to be processed. is that possible?

what the difference between flow type ICMP-local, ICMP-app, ICMP-control, ICMP-default, ICMP-app-default?

Regards,

Aditya

Tom Marcoen
Level 1
Level 1

Alexander, you said:

The mgmt interfaces on the RSP are designated out of band. This means that they can't transport user traffic but only management traffic.

Is it possible to convert these mgmt interfaces to inband, like this?

control-plane
management-plane
inband
interface MgmtEth0/RSP1/CPU0/0
!
!
!

And, if not possible, any thoughts on how we can tackle the following problem?

We want to manage our ASR routers via an out-of-band network on the mgmt interfaces. As the ME switches in the datacenter are too thinly spread to connect their out-of-band interfaces to this dedicated network, we want to manage them inband using a dedicated L3VPN. This requires L3 connectivity between the L3VPN and the mgmt interface...

xthuijs
Cisco Employee
Cisco Employee

hi tom, you can't convert them to inband, but you can allow them to route through, however I would not recommend for this.

By default the system will NOT route through the mgmt ethernet port.

Say, you have a static default route pointing out the mgmt ethernet. That means that traffic arriving from a linecard for which we have no destination specifically in the FIB will NOT get forwarded to the RSP for egressing the mgmt port.

Traffic ingress on LC will get dropped by the NP in that case.

Same deal for traffic arriving in the MGMT port, it will not get forwarded out to a linecard either.

You can override that behavior, for testing purposes with the command :

rp mgmtethernet forwarding

xander

Tom Marcoen
Level 1
Level 1

Thank you for your quick response! It seems you also explained it here: https://supportforums.cisco.com/discussion/11840376/use-management-ethernet-port-asr9000

Is it possible to use LPTS (or maybe CoPP / MPP) to reduce the impact of DNS DDoS attacks?  Basically, we want to perform rate limiting in hardware.

Thanks.

-ben

xthuijs
Cisco Employee
Cisco Employee

sure thing that is possible Ben!!

check cisco live id 2904 from sandiego 2015 and sanfran 2014 I had a specific section there on the NTP DDOS, but similar approach exists for DNS (if locally targeted) too.

if not locally targeted, but transit, then you can use BGP flowSPEC to define a rule and police or redirect that particular traffic very powerful!

xander

Found it!  Pages 42-45 from 2014 SanFran.  Thanks Xander!

-ben

juamart2
Community Member

Hi Xander, thanks for sharing this doc, it's very useful.

Is it possible to know the LPTS punt police default NPU values for ARP protocol?

We have to migrate a service with faster arp learning and we need to know if the default it's ok or if we need to configure LPTS before migration.

Thanks again!

Juan.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Quick Links