Introduction
IOS devices have the concept of control plane policing. IOS-XR doesn't use that concept but instead uses a very comprehensive and powerful Local Packet Transport Services. In this document it is explained how LPTS works and how you can work with it, monitor and verify it.
Core Issue
LPTS is the concept of reflexive ACL's, punt policers and has an "internal" FIB or iFIB that directs certain packets to various nodes. IOS-XR can handle certain traffic on the linecard (such as BFD, Netflow and ARP) and these packets are instructed by LPTS to be handled by the local CPU rather then the RSP CPU.
At the same time, there are ACL's in place that allow for instance the punting of Telnet traffic and then per host if configured so, but another component of LPTS called MPP, the Management Plane Protection.
Generally, the default values for LPTS provide the level of protection you are after. However there are some rare circumstances whereby you want to tune the values of LPTS in order to get the service levels you need. LPTS is very dynamic in nature and pierces holes into the protection system as particular items are configured.
The LPTS policers work on a per NP basis. So if the LPTS police value is set to 1000pps that means that every NP on the LC can punt with 1000pps to the RSP CPU or LC CPU. This is something to take into consideration when evaluating the A9K-8T-x cards who have 8 NPU's per LC.
Take extreme care when changing the LPTS policer values.
High level overview
From a birds eye view, LPTS looks like this:
The NPU has a table that tells it where to send packet to (LC or RSP) as part of the "internal FIB" or iFIB. These packets are punted at a pre-defined rate, they can be tuned in XR release 4.x and later. Also in the TCAM which is used in the ASR9K for ACLs (amongst others), are lists that define which packets we want to allow and not. This will be discussed in the MPP section of this document.
| LPTS is composed of a (set of) dynamic ACL's (which are created as part of user configuration dynamically, or automitcally inserted as peerings establish), an internal "routing table" (iFIB) and a set of policers for different punt reasons. |
|---|
LPTS Characteristics
So for-me packets are undergoing the Pre iFIB classification and policing upon which they are directed by the iFIB, which is the second level of filtering to the destination node.
LPTS Firewalling
One of the great strenghts with LPTS is the dynamic ACL creation. This is configuration driven and no user intervention is required.
In addition to that, LPTS has different flow categories based on the state of the protocol. For instance, BGP has 3 different states:
- Unknown
- Configured
- Established
Unknown is the flow whereby we have TCP port 179 traffic, but we have no neighbor configured from that source. Policed very heavily.
Configured is the entry whereby we know the source address of the peer, but the session is not yet established (no known source port from the peer), Policed moderately.
Established is where we have all the L3 and L4 data from the session. Lightly policed.
The entries for configured is driven by the configuration of the neighbor statement under the router BGP section.
Established is dynamically inserted when the peer establishes.
You could theoretically police the unknown to a rate of zero.
Example:
Router bgp
neighbor 192.168.1.1
…
!
The following table can be seen with the output of the command:
show lpts pifib hardware entry brief loc 0/3/cpu0 | i 179
| Local | Port | Remote | Port | Rate | State |
|---|---|---|---|---|---|
| any | 179 | ANY | ANY | 100 | unknown |
| any | 179 | 192.168.1.1 | ANY | 1,000 | configured |
| 192.168.1.2 | 179 | 192.168.1.1 | 2223 | 10,000 | established |
If you use the command
RP/0/RSP0/CPU0:A9K-TOP#show lpts pifib hardware entry location 0/3/CPU0 | be 33.33.1
You can check the detailed entry of the PiFIB (policer)
Source IP : 33.33.1.1 the remote address
Is Fragment : 0 fragments allowed
Interface : any expected source interface
M/L/T/F : 0/IPv4_STACK/0/BGP-known
DestNode : 48 where the packets are sent to
DestAddr : 48
L4 Protocol : TCP
TCP flag byte : any additional security checks at TCP level
Source port : Port:179
Destination Port : 11293
Accepted/Dropped : 117866/0 packets accepted and denied
# of TCAM entries : 1 number of tcam entries burnt for this PiFIB entry
HPo/HAr/HBu/Cir : 1924676/2500pps/2500ms/2500pps
State : Entry in TCAM status of the entry
Configuring LPTS police rates
You can configure the LPTS Policers on a PiFIB bases and also the punt policers can be adjusted.
The following commands apply. Note that this is on a per linecard basis. All NPU's on that linecard will get reconfigured.
RP/0/RSP0/CPU0:A9K-BNG(config)#lpts punt police location 0/0/CPU0 protocol ?
arp ARP packets
bfd Bidirectional Forwarding Detection packets
cdp Cisco Discovery Protocol packets
cfm Connectivity Fault Management Protocol packets
cgmp Cisco Group Management Protocol packets
dhcp Dynamic Host Configuration Protocol packets
efm Ethernet in the First Mile Protocol packets
igmp-snoop Internet Group Management Protocol Snoop packets
ipiw-arp L2VPN IPIW ARP packets
ipv4 IPv4 packets
ipv6 IPv6 packets
lacp Bundle Protocol packets
mofrr Multicast-only FRR packets
mpls MPLS punt packets
mstp Multiple Spanning Tree Protocol packets
mvrp Multiple VLAN Registration Protocol packets
ppp Point-to-Point Protocol packets
pppoe Point-to-Point Protocol over Ethernet packets
rarp Reverse ARP packets
vccv Virtual Circuit Connection Verification packets
vidmon Video Monitoring packets
vidmon-flow-add Video Monitoring flow add packets
Exception packets can be reconfigured by the following command: lpts punt police location 0/0/CPU0 exception
Glean adjacency or ACL-deny packets can be tuned for instance via that command.
The PIFIB can be reconfigured via the following commands:
RP/0/RSP0/CPU0:A9K-BNG(config)#lpts pifib hardware ...
- In there you can enter the linecard you wish to specifically reconfigure
- The policer flow values
- And the TCAM entries (this is new in XR420)
- As you've seen LPTS can dynamically create "ACL" entries for dynamic firewalling and for MPP. This command limits the number of TCAM entries that LPTS can use so that space is available for other purposes such as regular ACL's, QOS matching, EFP matching etc.
LPTS static-police and police differences
The command “police” is used to check policer values, accept/drop counts for packets matching LPTS TCAM(mostly L3 packets) entries whereas “static-police” is used to check policer values.
Accept/drop counts for packets matching static punt reasons programmed in search structures (Mostly L2 and exception packets).
“policer” is for dynamic flows (like BGP, OSPF, etc protocols directed by RSP)
“static-policer” is for pseudo Static flows (like BFD, CFM directed by the LC) These are hard-coded and include Exception processing packets.
There is a CLI to change few of the exception processing as well (for e.g. ICMP unreachable)
Monitoring LPTS
LPTS is not SNMP enabled (request has been filed and is in the works, no target release defined at time of writing). Though there are very inventive ways to monitor LPTS and generate alerts. There is a TCL script that you can use with EEM in order to get some level of alerting.
Attached to this article is the script package and here is how you set it up:
event manager environment EEM_LPTS_CHECK_INTERVAL 300 event manager environment EEM_LPTS_CHECK_FLOWTYPES BGP-known * event manager environment EEM_LPTS_CHECK_LOCATIONS 0/0/CPU0 0/4/CPU0 event manager environment EEM_LPTS_CHECK_THRESHOLD 1 50% event manager directory user policy disk0:/scripts/ event manager policy lpts-threshold-alerting.tcl username scripts |
|---|
How to clear LPTS statistics
LPTS stats cannot be cleared by LPTS commands or qos counter clearing.
You can clear LPTS stats by clearing hte np controller stats:
"clear controllers np counters all location <>”
MPP: Managed Plane Protection
In the standard configuration all interfaces have access to the Telnet, SSH and SNMP daemons.
Inband vs Out of band
All linecard interfaces are designated to be inband, meaning they can transport user traffic as well as management traffic.
The mgmt interfaces on the RSP are designated out of band. This means that they can't transport user traffic but only management traffic.
Out-of-band interfaces can't "speak" to other interfaces as they are desginated for managment traffic. So eventhough there is a route in the system that would send traffic out of the mgmt interface, Fabric enabled interfaces on the LC can't
Here an example of out of band and the restrictions that it imposes on the forwarding
Configuring MPP
By default when the service is configured, there are no mpp restrictions. All interfaces are able to accept the mgmt traffic for the service you defined. For example, when the telnet server is configured, LPTS reports the following binding:
RP/0/RSP0/CPU0:A9K-BNG#show lpts bindings brief | i (any.23 )
Tue Feb 28 12:00:55.195 EDT
0/RSP0/CPU0 TCP LR IPV4 TCP default any any,23 any
This means that every for me packet with port 23 as the dest port will get serviced.
Now when configuring MPP the bindings output changes:
control-plane
management-plane
inband
interface TenGigE0/1/0/0
allow Telnet peer
address ipv4 3.3.3.3
address ipv4 5.5.5.0/28
!
!
interface GigabitEthernet0/0/0/10
allow Telnet
!
!
!
In this configuration example I am designating two interfaces as inband, so they will still be able to forward transient traffic and allow inbound telnet traffic. At the same time I allow telnet from any host on Gig0/0/0/10 and only telnet from a few peers on Te0/1/0/0.
The LPTS bindings are dynamically changed as per following output:
RP/0/RSP0/CPU0:A9K-BNG#show lpts bindings brief | i (any.23 )
Tue Feb 28 12:06:48.339 EDT
0/RSP0/CPU0 TCP LR IPV4 TCP default Gi0/0/0/10 any,23 any << Any source can access my telnet on this intf
0/RSP0/CPU0 TCP LR IPV4 TCP default Mg0/RSP0/CPU0/0 any,23 any << Dedicated inband
0/RSP0/CPU0 TCP LR IPV4 TCP default Te0/1/0/0 any,23 3.3.3.3 << /32 host access for telnet on dedicated intf
0/RSP0/CPU0 TCP LR IPV4 TCP default Te0/1/0/0 any,23 5.5.5.0/28 << Hosts from this subnet on this intf
Powerful eh!?!
We can also look at the pre internal fib (piFIB) and check the entries there:
RP/0/RSP0/CPU0:A9K-BNG#show lpts pifib hardware entry bri location 0/1/cpu0 | i (.23 )
Tue Feb 28 12:27:46.389 EDT
7 IPV4 default TCP Te0/1/0/0 LU(48) any,23 3.3.3.3,any
10 IPV4 default TCP Te0/1/0/0 LU(48) any,23 5.5.5.0/28,any
Decoding the Destnode in LPTS entries
In the example above you see the following detail: LU(48). This section explains that number and detail.
The LU means local unicast fabric. The 48 is a very interesting number.
The device that this output is taken from is an ASR9010. Which has 8 LC slots and 2 RSP slots. On both sides of the RSP's in the middle
are the 4 LC's
If I were to decode the 30 into binary it looks like this:
+---+---+---+---+---+---+---+---+
| 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 | Position
+---+---+---+---+---+---+---+---+
| 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | Bit value for Decimal 48
+---+---+---+---+---+---+---+---+
|LC |LC |RSP|RSP|LC |LC |LC |LC | Slot position filling (note 2 left most LC's not drawn)
+---+---+---+---+---+---+---+---+
Now you can see that the 1's are in position 5 and 4, and if you look at the slot numbering of the ASR9006, you can see that these are
the RSP's!! So telnet is delivered to the RSP.
Restrictions for MPP
6) No MIB Support
Related Information
Cisco Guide to Harden Cisco IOS XR Devices
LPTS Considerations
If you can use only p2p OSPF network type
flow ospf-uc-known rate 0
flow ospf-uc-default rate 0
flow bgp-default rate 0
flow ldp-tcp-default rate 0
flow msdp-default rate 0
Xander Thuijs, CCIE #6775
Sr. Tech Lead ASR9000
Thank you for your answer Xander!
I assume that in your example the interface g0/1/0/0.20 is assigned to vrf RED?
If it is the case, so I think your example and mine are different. I am inserting a diagram for clarification :
The 4 routers are forming a MPLS backbone ( inter-router links are MPLS links). I try for example to establish a management session (telnet or ssh) to R2 from the station on the left ( or from any of the other routers , specifying that I am using the vrf RED to establish that session).
In that case, I see the "reject" counter incrementing in the command "show lpts ifib statistics", and R2 effectively refuses the connection.
But if I try to establish a management session to R1 from the station (or from R3 , but using the LAN link assigned to VRF RED), behaviour is OK, connexions are established. For me, it seems that connections are rejected because the "inband" interface used to reach the loopback interface, and that same loopback interface are not both assigned to vrf RED (inband interface is a MPLS interface, shared between many services).
telnet/ssh services are correctly activated for correct VRFs as you reminded.
I hope my explanation is enough clear
Best regards,
Brahim
This is becoming more and more interesting..
well I checked those constraints :
- I am running a 4.1.2 release so for the moment I can not disable SVD. I'll check when I can.
- But in my topology R1 and R3 already have their "LAN" interface (assigned to VRF) and their MPLS backbone interface located in the same card (see topology below). And behaviour is the same when I try to manage R3 from the host on the left (trafic of course flowing through R1... If I try to manage R3 from the LAN, directly to VRF interface, I do not have the problem: telnet/ssh sessions correctly establish)
And the same if I try to establish the same remote session from R1 : connections are dropped.
- I looked for a debug, I used a "debug lpts packet drops" , which gives me this kind of message each time a connection is dropped :
netio[310]: lpts ifib [0xddc545b8/66 if 0x000001a0 VRF 0x60000001 IP4 SOURCE_IP_ADDRESS.62854 -> DESTINATION_IP_ADDRESS
.23 TCP S] to local stack for reject, dropping
I really appreciate your interest !!
Thank you for that,
Regards
Brahim
