cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
38523
Views
15
Helpful
92
Comments
xthuijs
Cisco Employee
Cisco Employee

Introduction

This document tries to assist in an easy and smooth migration from IOS to IOS-XR. Because of the fundamental different nature of the IOS XR operating system and the way things have been implemented specifically by the ASR9000 platform, this article tries to collect a couple of key items to think about and providing some pointers that prevent issues down the road and prepare for proper planning to the great ASR9000.

In this article various topics are separated out per main topic.

  • Operating System     
    • Monolithic for MicroKernel
    • Understanding the IOS-XR prompt and privilege levels/taskgroups

    • Memory architecture

    • NETIO and "slow switching"
    • Using show commands and the location keyword
    • RIB, FIB and adjacencies
    • Committing configurations and rollback points
    • Commit options
    • Rollback options
  • OSPF     
    • Processes and Using OSPF as a PE-CE protocol

  • BGP     
    • Capability advertisement
    • Using neighbor, peer and session groups
    • RPL
    • RPL and changes to the policy
    • InterAS
  • L2VPN
    • Matching configuration from 7600 to ASR9K for L2 Services:
    • Spanning Tree
    • SVI and BVI
    • EFP
    • Converting IOS trunks into XR
    • SNMP

    This is a "living document", we'll add more and more items as we see questions coming in that have not been covered before, so watch the revision of the document to see if new items have been added. I realize that this document is not complete, but more to be added as we go.

    Operating System

    Monolithic vs Microkernel

    One of the key differences between IOS and IOS-XR is the base operating system. Legacy IOS is known to be a "monolithic" operating system. Effectively it is a run to completion whereby some timesharing is done between processes. This model has proven to be working out very well for over 25 years given the success of Cisco IOS based routers and switches. Also IOS uses a complete shared memory space.

    Of course there are also drawbacks which IOS-XR focusses on to address.

    One of these enhancements is that XR is running on a microkernel (qnx based) and on top of that we are running the IOS XR processes.

    These processes are running similar to a process on a linux based operating system. Effectively the QNX gives us a K-Shell from which we can do similar things as a unix based OS.

    When seeing the IOS-XR prompt, if you type "run" it will give you access to the K-Shell. Although it is not supported officially, sometimes it is handy and useful to access the kshell to get hardware level counters or access the file system to copy things around etc.

    The flexibility that IOS-XR gives with these processes are:

    • ability to restart a process
    • ability to patch a process (Via a SMU, the software maintenance update)
    • complete control plane and data plane separation (if eg OSPF crashes it doesn't affect the forwarding)
    • control plane distribution (some functionality can be offloaded to the linecards like netflow or BFD for scale increase)

    Understanding the IOS-XR prompt and privilege levels/taskgroups

    IOS has a very simple prompt with a host name followed by a sign that identifies the "mode" that you are in, whether that is privileged exec or regular exec etc.

    For instance:

    CPE#

    or

    CPE>

    IOS-XR prompt looks like this:

    RP/0/RSP0/CPU0:A9K-BNG#

    The way to interpret it is as follows:

    RP : We are looking at a route processor

    0    : Currently we are attached to shelf 0. In the case of multichassis (CRS) or Clustering (ASR9000) we can link multiple chassis together functioning as a single entity, this number identifies which shelf from that same logical node we are looking at.

    RSP0: Which RSP we are connecting to. In the case of dual RSP the lower slot ID is RSP0 and the higher slotID is RSP1. Generally you always logon to the active RSP via telnet which can then either be RSP1 or RSP0.

    CPU0: today we only have a single (multicore) CPU  on the RSP and linecards. This would identify the CPU we are working with in the case that we are adding CPU's on the system.

    :hostname : this is the well known part, the hostname.

    Note that the suffix of the complete prompt is always with a hash '#' sign. Which suggests that you are in privilige 15 mode.

    IOS-XR does NOT have the concept of privilege levels but instead uses task group authorization.

    To learn more about using task groups in IOS-XR check you can see in this picture.

    Some key differences and highlights between 7600/IOS and ASR9000/XR

    Common part
    –Both share the same EVC SW infrastructure
    –Feature parity for the flexible VLAN tag classification, VLAN tag rewrite and service mappin§
    7600 IOS
    –VLAN tag classification, rewrite, service mapping are all done on the port level (with some exceptions), which is classic IOS CLI
    –Introduced “service instance” configuration mode for better L2VPN scale
    –Legacy switchport feature support in parallel (but can’t co-exist with EVC on the same port)§
    ASR 9000 IOS-XR
    –De-couple port level and service configuration. VLAN tag classification and rewrite are done at port level. L2VPN services are configured at “l2vpn” module
    –Uniform “sub-interface” CLI for both L2 and L3 service, no additional “service instance” structure
    –Common Infrastructure for native L2 and MPLS based L2VPN service

    Matching configuration from 7600 to ASR9K for L2 Services

    evc.jpg

    A very comprehensive overview of the EVC model is found on this link.

    Spanning Tree

    The ASR9000 only supports full MSTP and no other spanning tree protocol.

    There is the possibility to use the PVST in PVST-AG mode or Access Gateway.

    The "AG" version of the MST or PVST gives you the ability to run these protocols in an P2MP VPLS deployment without the need to run the full protocol set. It basically is designed around the 9K PE's being the root, advertising pre-canned BPDU's and receive the TCN's from the access switches to trigger MAC withdrawl.

    More info on VPLS and ASR9000 is here.

    Running Spanning Tree (not the AGG) version together with IOS requires you to be aware of the concept of VLAN pruning that IOS does and XR is not aware of.

    Migrating spanning tree from 7600 to ASR9000 can be a complex task. IOS switches run STP by default, and you need to disable it explicitly if you don't want to run it. ASR9000 does not run any spanning tree protocol by default and you need to enable it explicitly.

    Also the way that BPDU's are handled in XR/ASR9000 is dependant on your configuration.

    The following scenarios cover a few of these design migrations you need to be aware of.

    This section tries to cover both MSTP and PVST. The key difference for these 2 protocols is that MSTP sends BPDU's untagged and PVST sends tagged BPDU's on the vlans that are PVST enabled.

    One of the first decisions you need to make is whether you want the A9K's to be part of the Spanning Tree design or be transparent to them.

    There are pros and cons to each option.

    In this first picture below shows a design whereby the ASR9000's are NOT part of the spanning tree topology.

    If you have defined an untagged EFP like this:

    int Gig0/0/0/P.1 l2trans

    encap untagged

    you will capture the MSTP BPDU's and put them subject to the service that is attached to this untagged EFP.

    This can either be a Cross connect (p2p) or a Bridge domain (p2mp). The difference between XCON and BD is that XCON transparently takes whatever comes in on the Attachment Circuit (AC) and send it to the other side (whether that is a phyiscal interface again or a PseudoWire). An Xcon can only have 2 interfaces.

    A bridge domain can have multiple EFP's and also employs mac-learning. If the Destination MAC is not know or part of a broadcast/multicast mac address it will get "Flooded" over to all EFP's in the Bridge Domain, except for the originating EFP (split horizon).

    Ok so in this design, with that knowledge from above, the BPDU's from switch X are sent via interfaces X and Y to PE1 and PE2.

    PE1 would take the BPDU from the untagged EFPand sends them transparently to PE2 over interface M to Switch B's interface U.

    In other words Switch A and B see each other as directly connected neighbors. The A9k's are completely transparent and acting as a transparent L2 wire.

    This STP design will block one of the 4 (X, Y, U or V) interfaces to break the loop.

    Design 1

    Slide3.JPG

    If you were to have a bridge domain on PE1 and a pseudowire between PE1 and PE4, the BPDU *also* gets sent to PE4 and arriving on interface V.

    This model whereby the 9k's are transparent to STP cannot be used with a full mesh of PseudoWires.

    This design that you see above is generally seen by "accident" when it is forgotten that the switches run STP by default and the 9k would transparently pass everything on.

    "Solutions" are to break to loop manually and using an L2ACL to block the MSTP BPDU's from traversing your 9K's.

    In the scenario that you do want the 9k's to participate in spanning tree you basically create to STP "islands" on the left and right side.

    The 9k's now terminate the spanning tree coming from the switches. A full PW mesh is possible and this is also one of the designs where the AG version of the STP protocol becomes very useful.

    Switch A sees PE1 and PE2 as neighbor.

    Design it such that the PE1 and PE2 are root and back up root.

    The configuration for this design is to put the interface P into the STP Configuration so that BPDU's are sent and received.

    Design 2

    Slide4.JPG

    The effects of the design scenarios and the relation to the spanning-tree protocol in use are pretty much the same for both MSTP and PVST.

    What happens when you follow design 1 or 2 in relation to the EFP configuration associated with it, will be discussed below separated out between the two key STP's.

    More detailed configurations and VPLS designs are discussed in this article.

    Let us evaluate the various configuration options that you have when defining your EFP's with and without Spanning tree.

    MSTP

    Slide1.JPG

    Scenario 1 in this picture above is the model that you want to use in the design option "2". There is no untagged EFP necessary in this case, and BPDU's received are punted and locally generated BPDU's are injected directly into the port to the switch.

    Scenario 2 describes a situation whereby sometimes people want to peel out their untagged traffic and transport it while still running MSTP on the 9k as in design "2". This is problematic today for a few reasons:

         1) received BPDU's are subject to the untagged EFP service defintion and will get forwarded. The local MSTP configuration injects BPDU's.

         2) this causes the locally connected switch to see BPDU's from the VPLS remote side (switch B) as well as PE1.

         3) it will cause MSTi mismatches and unexpected blocked ports.

    Scenario 3 can be used for "design option 1". We don't have any local configuration for STP, so we're not injecting anything, we are sending the BPDU's across as per the EFP service definition.

    Note:

    Scenario 2 is however a design that is recognized as a design we need to support. Starting XR421 scenario 2 will work as follows:

    If there is an untagged EFP *and* local STP config on the PE, THEN we will NOT forward the BPDU, but punt them for local STP handling.

    We will continue to inject local BPDU's towards the locally connected switch.

    In other words if you have untagged traffic that you want to transport but not the BPDU's this will work in XR421. Today, you will get the behavior as described above in scenario 2.

    If your intend is to use design option 1, and you want to forward untagged traffic (config scenario 3), but you don't want to forward the BPDU's then you must apply an L2 ACL onto the untagged EFP to block and deny the DMAC used for (MSTP) BPDU's.

    The ACL definition is discussed in this article in the related information section.

    PVST

    The story above doesn't change that much when we are considering PVST.

    However there are some minor tweaks caused by the fact that PVST BPDU's are vlan tagged.

    Slide2.JPG

    Scenario 1 is used in design option "2" whereby you want your A9K's to participate in PVST. Note that we don't do full PVST, but PVST-AG or access gateway, which means that we are sending the bpdu's on the EFP's for the respective vlans and take the BPDU's from these vlans and react on them with mac widthdrawl. The configuration scenario looks like this:

    !EFP's

    interface g0/0/0/P.10 l2trans

    encap dot1q 10

    ...etc

    !service definitions

    l2vpn

    bridge group VLANS

    bridge-domain vlan-10

    interface g0/0/0/P.10

    bridge-domain vlan-20

    interface g0/0/0/P.20

    ...etc

    !spanning-tree config

    spanning-tree pvst-ag

    interface g0/0/0/P.10

    interface g0/0/0/P.20

    interface g0/0/0/P.30

    HOT HOT HOT HOT

    Scenario 2 is a common issue we see happening causing a lot of trouble. This config scenario does NOT have any local PVST configuration, but if the adjacent switches have PVST enabled (and that can be the default!!) then we'd be transparently passing on the vlans as part of the EFP's service definition! The PVST BPDU's are arriving at the remote side and what can be worse is that if we are doing vlan manipulation in terms of tag rewriting with pop or push operations, then the remote side received BPDU's meant to describe vlan 10, but received as VLAN X after the rewrite!

    This scenario can be the intended design as described in design option "1" above.

    Scenario 3 is a remedy for scenario 2. Basically we are using an L2ACL blocking any bpdu's on the EFP's received so that we are not confusing switches on either end. Alternatively you can also disable STP on the switches connected to the 9k PE's. We are applying L2 ACL's that are blocking a particular DMAC that is used for the PVST bpdu's (see

    This issue described here above is something you MUST be aware of.

    The ACL definition is discussed in this article in the related information section.

    SVI and BVI

    The concept between a Switch Virtual Interface and a Bridge Virtual Interface is the same: and L3 endpoint in an L2 environment.

    The SVI is a switch concept and the BVI is an L3 concept generally seen on routers.

    The BVI interface in IOS-XR/ASR9000 has some restrictions well documented in the CCO documentation for BVI.

    Use this reference to setup IRB (Integrated Route Bridging) using the BVI.

    EFP

    When you set up your Ethernet Flow Point (EFP), especially the untagged one, it can make you run into unexpected scenarios.

    For instance, when you have an untagged EFP and you are running full MSTP, the 9K will be able to inject BPDU's to the peer, but the peer's BPDU's are subject to the service of the untagged EFP and may get forwarded. This results in MSTP conflicts on your peer device.

    With XR 4.2.1 we'll have the auto ability to peel out the BPDU's from the untagged EFP when MSTP configuration is present.

    More info here.

    Also the forwarding of vlan traffic out of an EFP and vlans has a few things that you need to be aware of documented in this article

    Converting IOS trunks into XR

    Because the IOS-XR EVC model is not aware of trunks like IOS devices are, the conversion from an IOS trunk to an XR EVC based config can be a bit confusing at first. This configuration example documents how to convert an IOS trunk to an XR EVC model:

    IOS:

    interface TenGigabitEthernet13/3

    description my-trunk

    switchport

    switchport trunk encapsulation dot1q

    switchport trunk allowed vlan 4,130,133

    switchport mode trunk

    no ip address

    interface Vlan 4

    ip add 10.11.2.1 255.255.255.0

    XR:

    The translation will be:

    interface TenGigabitEthernet 0/0/0/0

    description my-trunk-like-xr-interface

    Define the EFP's with their respective vlan tags. Because a BVI is used we need to pop the tag so that "inside" the bridge-domain we see untagged packets. On egress, the vlan tag will be slapped on as per EFP definition. Effectively, we create a bridge-domain per vlan.

    interface ten0/0/0/0.4 l2transport

    encapsulation dot1q 4

    rewrite ingress tag pop 1 symmetric

    interface ten0/0/0/0.130 l2transport

    encapsulation dot1q 130

    rewrite ingress tag pop 1 symmetric

    int ten0/0/0/0.133 l2transport

    encapsulation dot1q 133

    rewrite ingress tag pop 1 symmetric

    The L2transport command makes these switchports for L2 services

    For the switchport trunk allowed vlans, and the interface vlan X, you need to do the following:

    First create the bvi interface:

    interface BVI4

      ipv4 address 10.4.1.10 255.255.0.0

    interface BVI130

      ipv4 address 10.130.1.1 255.255.0.0

    interface BVI133

      ipv4 address 10.130.1.1 255.255.0.0

    Note that the BVI interface number doesn't necessarily need to be the same as the VLAN identifier, same goes for the subinterface number of the l2transport interface. Though for this example, the practice is followed to make the BVI number, the same as the dot1q TAG value and the same as the EFP subinterface number for clarity.

    Then you need to create the bridge group to tide all together.

    l2vpn

    bridge group MyTrunks

      bridge-domain VLAN4

        interface ten0/0/0/0.4

        routed-interface bvi4

    bridge-domain VLAN130

      interface ten0/0/0/0.130

      routed-interface bvi130

    bridge-domain VLAN133

      interface ten0/0/0/0.133

      interface bvi133

    The Bridge group is just a non functional configuration hierarchy to tie several bridge-domains together in part of the same functional group. It functionaly is no different then creating multiple individual groups with their domains, as opposed to one group with multiple domains.

    SNMP

    Because as you've seen throughout this document XR is heavily distributed, SNMP being a component that requests data from every feature or functioanlity potentially is very heavily relient on IPC's to get its info. Sometimes it feels that show commands or SNMP performs slower in a next generation OS like XR, but this is because of these IPC's.

    Also because IOS-XR employs the concept of "SDR" or Secure Domain Routers (CRS specific), some restrictions apply to the way that SNMP operates.

    Significant performance options have been put in place. For instance, when you get the stats for an interface, rather then sending an IPC for one interface, we collect a "bulk" of info for the next X interfaces also as you might do a getnext for the next if inline.

    Some "standard" data like the Entity info is subject to the load of the MGBL pie that gives access to these MIBS as well as special config is needed to expose this info to the SNMP agent. See here for more detail on that.

    Related Information

    Xander Thuijs, CCIE #6775

    Sr Tech Lead ASR9000

    Comments
    shiras k a
    Level 1
    Level 1

    Hi Alexander,

                           In ASR 9 k, which feature is using for multichasis end point pwseudowires in case of Inter-AS PEs or Metro networks? Is it LACP ? To mitigate MAC duplication and loop , can I  categorise the MAC as protected and non-protected ?

    xthuijs
    Cisco Employee
    Cisco Employee

    If there are 2 separate nodes then the access side can use MC-LAG and on the core side you can use (backup) Pseudo wires.

    You can also choose to have a link between the 2 nodes and active active PW so that if the traffic arrives on the PW that terminates on the node that does not have the active member for the mclag, it can switch it down to the adj node.

    Or you can consider using cluster. Single PW, regular LAG and no difficult tricks needed.

    regards

    xander

    shiras k a
    Level 1
    Level 1

    Hi Alexander,

                         May I know how to mitigate MAC duplication and network loop using the features of asr 9k ? In addition to that I have another doubt, In PEs Can we categorize mac based on the location (ie, local :- learned from attched circuit and remote :- leared from network side) ?? I am expecting your valid response.

    xthuijs
    Cisco Employee
    Cisco Employee

    Shiras, that functionality to prevent mac duplication or mac moving is prevented by a feature called mac security.

    If we learnt a mac on one port and it moves to another, we can associate an action to that. Syslog, port shutdown things like that.

    Looping traffic is prevented by using PW's in a VFI or using split horizon groups and of course spanning tree.

    Mac categorization on PW or AC, there is no differentiation based on that when it comes to SHG or MAC security.

    regards

    xander

    shiras k a
    Level 1
    Level 1

    Hi Alexander,

                         Do you mean the below steps for preventing mac duplication or mac movement ??

    RP/0/RSP0/CPU0:router(config-l2vpn-bg-bd)#mac

    RP/0/RSP0/CPU0:router(config-l2vpn-bg-bd-mac)#limit

    RP/0/RSP0/CPU0:router(config-l2vpn-bg-bd-mac-limit)#action shutdown

    RP/0/RSP0/CPU0:router(config-l2vpn-bg-bd-mac-limit)#maximum 1

    In case of network where stp/rstep/mstp is not running , can't we prevent loop traffic using mac flushing or

    something ??

    xthuijs
    Cisco Employee
    Cisco Employee

    MAC duplication within the same bridge domain is a mac move. And that you can protect yourself against with mac security.

    Mac limit, what you have as example is the number of macs that can live within the bridge domain as a total. I believe the granularity starts with 5, but that is a minor point.

    Mac duplication, that is the same mac in different bridge domains is not an issue, that should be allowed.

    I think I see your confusion on the mac-limit command, that is not the number of *instances* per mac in that BD.

    It is the total number of unique MAC's allowed in that BD.

    MAC security is the command to define an action when the SAME mac within the SAME BD is moving from one AC/PW to another.

    regards

    xander

    shiras k a
    Level 1
    Level 1
    Hi Alexander,
                    
                 Ooh, right,that was my confusion. But what am I thinking even is if the mac 
    belongs to diffrenet bridge domains, Shouldn't the mac be unique? because otherwise Can't
    one customer does DoS attack to another customer ?
    Please let me know your response.
    Is it right ?,
    RP/0/RSP0/CPU0:router(config-l2vpn)#bridge group b1 RP/0/RSP0/CPU0:router(config-l2vpn-bg)#bridge-domain bar RP/0/RSP0/CPU0:router(config-l2vpn-bg-bd)#mac secure RP/0/RSP0/CPU0:router(config-l2vpn-bg-bd-mac-secure)#action shutdown
    xthuijs
    Cisco Employee
    Cisco Employee

    Theoretically MAC's are unique. But they only need to be unique in the subnet that they exist!

    A VLAN is (technically) the equivalent of a subnet, and a Bridge-Domain (as per design intent) is the equivalent of a vlan.

    With that, you can have the same mac in 2 different bridge-domains. The same mac in the same BD will constitute a mac move.

    l2vpn

    bridge group xme

      bridge-domain xme

       interface Bundle-Ether100.999

       !

      !

      bridge-domain xme2

       interface Bundle-Ether100.1000

       !

      !

    !

    Mac Address    Type    Learned from/Filtered on    LC learned Resync Age

    Mapped to

    --------------------------------------------------------------------------------

    b4a4.e392.208d routed  BD id: 0                    N/A        N/A

    N/A

    0006.2aaa.2438 dynamic BE100.999                   0/RSP0/CP  0d 0h 0m 13s  N/A

    0006.2aaa.2438 dynamic BE100.1000                  0/RSP0/CP  0d 0h 0m 12s N/A

    RP/0/RSP0/CPU0:A9K-BNG#

    rakeshsekhar
    Level 1
    Level 1

    Hi Xander,

                         Can I enable and configure proxy server  in DHCP of L2vpn ??

    xthuijs
    Cisco Employee
    Cisco Employee

    Native L2VPN has no L3 component, but we can do snooping for some additional security.

    the other (trusted) port is where the dhcp server will be found, but there is no proxy configuration to that extend in that mode.

    What you can do is enable a BVI interface in the l2vpn bridge domain and configure dhcp proxy on that bvi.

    that means that the BVI will pick up the dhcp broadcast and relay that to the dhcp server as part of the configuration and the BVI interface is then the proxy between the clients and the dhcp server.

    You can still use snooping in this case also to limit where the dhcp messages will go to.

    regards

    xander

    rakeshsekhar
    Level 1
    Level 1

    Hi Xander,

                   Thanks for your reply. I would like to clarify one more thing,  Does L2vpn support " IGMP host tracking", because as my knowledge it will run only under  IGMP VRF & IGMP interface configuration modes. Is there any possibility to track and to find (eg: using expiry time) inactive hosts under igmp of L2vpn ?

    xthuijs
    Cisco Employee
    Cisco Employee

    For that Rakesh, you want to look into the feature called IGMP snooping.

    similar as dhcp snooping but then for mcast related traffic.

    (mcast like bcast is flooded if not implemented smoothly)

    regards!

    xander

    manuv1984
    Level 1
    Level 1

    Hi ,

                        Please let me know how l2vpn is forwarding frames. I know L3vpn a little bit. In l3vpn, RD adds to ipv4 to get vpnv4(96 bits) and sends the vpnv4 packets to destination which is identified by RT and there is each vrf for each customer also. Can we make l2vpn(VPLS)  connection between the sites via MPLS if one customer site uses ethernet and other end site uses ATM/framerelay technologies? Is vc label (inside label) and vfi concept in vpls equivalent to RT and vrf in l3vpn resp. ? How a PE router differentiate its different customers and their different services in vpls ? According to my knowledge vc label is using for identifying the PWs as ingress and egress (just like RT in l3 vpn, I don't know can we compare like this, please accept my apologies if it is not correct.). If anyone know please share the knowledge.

    xthuijs
    Cisco Employee
    Cisco Employee

    L2VPN is no different then a regular L2 switch: we forward based on the knowledge where the DMAC is.

    We learn the macs based on where the source mac was seen.

    If we dont know the DMAC we flood the packet, which also occurs for broadcast and multicast.

    When a packet is to be forwarded out a pseudowire we prepend the packet with the rewrite string belonging to that

    PW, this includes the following frame:

    L2 header according to the destination mac of the next hop, source mac of the egress interface connecting to that next hop. MPLS header according to the next hop information. Next MPLS heder indicating the psuedowire information.

    What follows next is the original L2 packet in EoMPLS, with the smac/dmac of the originally received packet.

    VLAn information is dependent on the Type of the PW 4 or 5 and what the rewrite of the vlans was set for on the EFP access side.

    xander

    manuv1984
    Level 1
    Level 1

    Hi Xander,

               Thanks for your precious response. Could you please tell me how the PE router identifies its different client sites and their different services. Can we do vpls between two sites even if they use different technolgies(eg: ethernet and frame relay or ATM) on either sides ? May I know the use of VFI also. I am expecting your response.

    Getting Started

    Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

    Quick Links