1. L2VPN Overview 2. Why L2VPN Auto Discovery? 2.1 VPLS auto-discovery 2.2 VPWS auto-discovery 3. VPLS Operation with BGP Auto-Discovery and Signaling 3.1 Responsibilities of BGP & L2VPN 3.1.1. BGP 3.1.2. L2VPN 3.2 Configuring VPLS with BGP AD & Signaling 3.3 Example of NLRI for VPLS with BGP-AD & Signaling 3.4 Verification of VPLS with BGP-AD & Signaling 3.5 Adding a third PE (PE3) 3.5.1.  L2VPN config for PE3 3.5.2.  Verification of PE3 4. VPWS Operation with BGP Auto-Discovery and Signaling 4.1 Responsibilities of BGP & L2VPN 4.1.1. BGP 4.1.2. L2VPN 4.2 Configuring VPWS with BGP AD & Signaling 4.3 Example of NLRIs of VPWS with BGP AD & Signaling 4.4 Verification of VPWS with BGP-AD & Signaling 5. Troubleshooting 6. Glossary   1. L2VPN Overview Layer 2 Virtual Private Network (L2VPN) emulates the behavior of a LAN across an L2 switched, IP or MPLS-enabled IP network, allowing Ethernet devices to communicate with each other as they would when connected to a common LAN segment. Point-to-point L2 connections are vital when creating L2VPNs.   As Internet service providers (ISPs) look to replace their Frame Relay or Asynchronous Transfer Mode (ATM) infrastructures with an IP infrastructure, there is a need to provide standard methods of using an L2 switched, IP or MPLS-enabled IP infrastructure. These methods provide a serviceable L2 interface to customers; specifically, to provide virtual circuits between pairs of customer sites.   Building a L2VPN system requires coordination between the ISP and the customer. The ISP provides L2 connectivity; the customer builds a network using data link resources obtained from the ISP. In an L2VPN service, the ISP does not require information about the customer's network topology, policies, routing information, point-to-point links, or network point-to-point links from other ISPs.   There are two fundamentally different kinds of Layer 2 VPN service that a service provider could offer to a customer: Virtual Private Wire Service (VPWS) and Virtual Private LAN Service (VPLS). There is also the possibility of an IP-only LAN-like Service (IPLS).   A VPWS is a VPN service that supplies an L2 point-to-point service. As this is a point-to-point service, there are very few scaling issues with the service as such. Scaling issues might arise from the number of end-points that can be supported on a particular PE.       A VPLS is an L2 service that emulates LAN service across a Wide Area Network (WAN). With regard to the amount of state information that must be kept at the edges in order to support the forwarding function, it has the scaling characteristics of a LAN. Other scaling issues might arise from the number of end-points that can be   supported on a particular PE.     2. Why L2VPN Auto Discovery?   Discovery refers to the process of finding all the PEs that participates in a given VPLS/VPWS instance. A PE either can be configured with the identities of all the other PEs in a given L2VPN service or can use some protocol to discover the other PEs. The later is called auto-discovery.   The former approach is fairly configuration-intensive, especially since it is required that the PEs participating in a given VPLS is fully meshed (i.e., that every PE in a given VPLS establish pseudowires to every other PE in that VPLS). Furthermore, when the   topology of a VPLS changes (i.e., a PE is added to, or removed from, the VPLS), the VPLS configuration on all PEs in that VPLS must be changed.       In the auto-discovery approach, each PE "discovers" which other PEs are part of a given VPLS/VPWS by means of some protocol, in this case BGP. This allows each PE's configuration to consist only of the identity of the VPLS/VPWS instance established on this PE, not the identity of every other PE in that VPLS/VPWS instance -- that is auto-discovered. Moreover, when the topology changes, only the affected PE's configuration changes; other PEs automatically find out about the change and adapt.   2.1 VPLS auto-discovery   Conventional VPLS implementation requires manual configuration of each neighbor (VPLS PE) in the VPLS domain. When a new PE is added or removed from the VPLS domain, manual configuration of each PE in the VPLS domain is required. Manual configuration changes add operational costs and increase the chance of network mis-configuration. VPLS Auto Discovery eliminates the need to manually provision a VPLS neighbor. VPLS auto discovery automatically detects when new PEs are added or removed from the VPLS domain. Auto-discovery by nature requires the information to be distributed to all members of a VPN - multipoint mechanism - which BGP is well suited for. BGP is also used for signaling to exchange label bindings and signal MTU and state changes. Although LDP is better suited for signaling between two endpoints, it is needed for interoperability with other vendors.   2.2 VPWS auto-discovery   There is no true auto-discovery in VPWS as it is in VPLS. In VPWS, to connect CEs, user has to explicitly configure at each PE. All what is discovered in VPWS is the existence of other PEs.     3. VPLS Operation with BGP Auto-Discovery and Signaling   There are two primary functions of the VPLS control plane: auto-discovery, and setup and teardown of the pseudowires that constitute the VPLS, often called signaling. Both of these functions are accomplished with a single BGP Update advertisement.   When the L2VPN address-family (AF) and VPLS/VPWS subsequent address-family (SAF) are configured, BGP will connect to L2VPN to receive configured VPLS bridge domains. In the case of distributed BGP and the presence of multiple BGP speakers, L2VPN still communicates with one active BGP instance only. Therefore, BGP distribution is completely hidden from L2VPN.   When a VPLS Bridge domain is configured with BGP auto-discovery and signaling enabled, BGP needs to distribute NLRI for the VPLS bridge domain with the PE as the BGP next-hop and appropriate VE-ID. Additionally, the VPLS is associated with one or more BGP export Route Targets (RTs) that are also distributed (along with NLRI). VPLS SAFI NLRI uses AFI = 25 and SAFI = 65. The keywords "l2vpn" and "vpls-vpws" will be introduced to represent AF and SAF respectively in the BGP configuration.   If a PE receiving VPLS NLRIs is configured with the VPLS associated with a particular import RT, it can then import all the NLRIs tagged with the same RT. Generic BGP RPL policies for RT filtering will be supported for the VPLS/VPWS SAFI. No specific NLRI policy will be added for VPLS/VPWS SAFIs.   The NLRI format for VPLS BGP-AD & BGP Signaling is shown in the diagram below:     Length (2 octets) Route Distinguisher (8 octets) VE ID (2 octets) VE Block Offset (2 octets) VE Block Size (2 octets) Label Base (3 octets)                             Figure 1: NLRI format for VPLS with BGP Auto-discovery and Signaling   3.1 Responsibilities of BGP & L2VPN 3.1.1. BGP Advertise LRI, RTs, VE-IDs and label blocks using AFI = 25 SAFI = 65. Learn VE-ID, range, as well as the RD/RT configured under a VPLS bridge domain. <ve-id, range, rd> along with RT. Import NLRIs based on RT(s) and passes {VFI_ID, local label, remote-label and next-hop, layer2info} to L2VPN_MGR. Replay the necessary information for the imported VFIs on the request of L2VPN_MGR. Provide an API for L2VPN to retrieve AS number.   3.1.2. L2VPN Learns the configured VFIs from Sysdb. Obtains the configured data such as RT, VPLS-ID, VPN-ID, VE-ID, VE-ID range, CE-ID and CE-ID from Sysdb. Obtains label block from LSD and maps the local label range (block size, label base, offset) per VFI. Notifies BGP of the configured parameters such RT, etc. Also, L2VPN_MGR shall replay this information upon request from BGP. Receives information such as local label, remote label, etc., pertaining to the PWs from BGP, creates appropriate entries in the bridge database, and notifies L2FIB to setup forwarding plane. Display auto-discovered data via show output. 3.2 Configuring VPLS with BGP AD & Signaling   3.3 Example of NLRI for VPLS with BGP-AD & Signaling   3.4 Verification of VPLS with BGP-AD & Signaling PE1:   PE1# show l2vpn discovery bridge-domain Service Type: VPLS, Connected List of VPNs (1 VPNs): Bridge group: bg1, bridge-domain: bd1, id: 0    List of Local Edges (1 Edges):      Local Edge ID: 3, Label Blocks (1 Blocks)        Label base Offset   Size     Time Created              ---------- ------   ----     -------------------        16015     1         10       01/24/2009 16:23:27    List of Remote Edges (1 Edges):      Remote Edge ID: 5, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------        16120     1         10       1.1.1.1         01/24/2009 16:23:46   PE1# show l2vpn bridge-domain detail VFI vf1      VPN-ID: 100, Auto Discovery: BGP, state is Provisioned (Service Connected)      Route Distinguisher: (auto) 3.3.3.3:32770      Import Route Targets:         2.2.2.2:100      Export Route Targets:        2.2.2.2:100      Signaling protocol: BGP      Local VE-ID: 3 , Advertised Local VE-ID : 3      VE-Range: 10      PW: neighbor 1.1.1.1, PW ID 100, state is up (established)        Encapsulation MPLS, Auto-discovered (BGP), protocol BGP          MPLS         Local                         Remote                                ------------ ------------------------------ -------------------------          Label       16019                        16122                                  MTU         1500                           1500                                  Control word disabled                       disabled                              PW type     VPLS                          VPLS                                  VE-ID       3                             5                         PE1# show bgp l2vpn vpls BGP router identifier 3.3.3.3, local AS number 100 BGP generic scan interval 60 secs BGP table state: Active Table ID: 0x0 BGP main routing table version 898 BGP NSR converge version 3 BGP NSR converged BGP scan interval 60 secs   Status codes: s suppressed, d damped, h history, * valid, > best              i - internal, S stale Origin codes: i - IGP, e - EGP, ? - incomplete    Network           Next Hop       Rcvd Label     Local Label Route Distinguisher: 1.1.1.1:32775 *>i5:1/32             1.1.1.1         16120           nolabel         Route Distinguisher: 3.3.3.3:32770 (default for vrf bg1:bd1) *> 3:1/32            0.0.0.0         nolabel         16015           *>i5:1/32             1.1.1.1         16120           nolabel           Processed 3 prefixes, 3 paths     PE2:     PE2# show l2vpn discovery bridge-domain Service Type: VPLS, Connected List of VPNs (1 VPNs): Bridge group: bg1, bridge-domain: bd1, id: 0    List of Local Edges (1 Edges):      Local Edge ID: 5, Label Blocks (1 Blocks)        Label base Offset   Size     Time Created              ---------- ------   ----     -------------------         16120     1         10       01/24/2009 16:03:26    List of Remote Edges (1 Edges):      Remote Edge ID: 3, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------        16015     1         10       3.3.3.3         01/24/2009 16:03:26   PE2# show l2vpn bridge-domain detail VFI vf1      VPN-ID: 100, Auto Discovery: BGP, state is Provisioned (Service Connected)      Route Distinguisher: (auto) 1.1.1.1:32775      Import Route Targets:        2.2.2.2:100      Export Route Targets:        2.2.2.2:100      Signaling protocol: BGP      Local VE-ID: 5 , Advertised Local VE-ID : 5      VE-Range: 10      PW: neighbor 3.3.3.3, PW ID 100, state is up ( established )        Encapsulation MPLS, Auto-discovered (BGP), protocol BGP          MPLS         Local                         Remote                                ------------ ------------------------------ -------------------------          Label       16122                         16019                                  MTU         1500                           1500                                  Control word disabled                       disabled                             PW type     VPLS                           VPLS                                  VE-ID       5                             3                                      ------------ ------------------------------ -------------------------   PE2# show bgp l2vpn vpls BGP router identifier 1.1.1.1, local AS number 100 BGP generic scan interval 60 secs BGP table state: Active Table ID: 0x0 BGP main routing table version 802 BGP NSR converge version 7 BGP NSR converged BGP scan interval 60 secs   Status codes: s suppressed, d damped, h history, * valid, > best              i - internal, S stale Origin codes: i - IGP, e - EGP, ? - incomplete    Network           Next Hop       Rcvd Label     Local Label Route Distinguisher: 1.1.1.1:32775 (default for vrf bg1:bd1) *>i3:1/32             3.3.3.3         16015           nolabel         *> 5:1/32             0.0.0.0         nolabel         16120           Route Distinguisher: 3.3.3.3:32770 *>i3:1/32             3.3.3.3         16015           nolabel           Processed 3 prefixes, 3 paths   3.5 Adding a third PE (PE3) A third PE (PE3) is added to the same VPLS domain with BGP AD & signaling.   3.5.1.  L2VPN config for PE3 Following is the L2VPN config for PE3:     3.5.2.  Verification of PE3   PE1# show l2vpn discovery bridge-domain Service Type: VPLS, Connected List of VPNs (1 VPNs): Bridge group: bg1, bridge-domain: bd1, id: 0    List of Local Edges (1 Edges):      Local Edge ID: 3, Label Blocks (1 Blocks)        Label base Offset   Size     Time Created             ---------- ------   ----     -------------------        16015     1         10       01/24/2009 16:23:27    List of Remote Edges (2 Edges):      Remote Edge ID: 5, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------        16120     1         10       1.1.1.1         01/24/2009 16:23:46      Remote Edge ID: 7, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------        16145     1         10       5.5.5.5         01/24/2009 16:40:32     PE1# show l2vpn bridge-domain detail VFI vf1      VPN-ID: 100, Auto Discovery: BGP, state is Provisioned (Service Connected)      Route Distinguisher: (auto) 3.3.3.3:32770      Import Route Targets:        2.2.2.2:100      Export Route Targets:        2.2.2.2:100      Signaling protocol: BGP      Local VE-ID: 3 , Advertised Local VE-ID : 3      VE-Range: 10      PW: neighbor 1.1.1.1, PW ID 100, state is up ( established )        Encapsulation MPLS, Auto-discovered (BGP), protocol BGP          MPLS         Local                         Remote                               ------------ ------------------------------ -------------------------          Label       16019                         16122                                  MTU         1500                           1500                                  Control word disabled                       disabled                              PW type     VPLS                           VPLS                                  VE-ID       3                             5                                    ------------ ------------------------------ -------------------------      PW: neighbor 5.5.5.5, PW ID 100, state is up ( established )        Encapsulation MPLS, Auto-discovered (BGP), protocol BGP          MPLS         Local                          Remote                                ------------ ------------------------------ -------------------------          Label       16021                         16147                                  MTU         1500                          1500                                  Control word disabled                       disabled                              PW type     VPLS                           VPLS                                  VE-ID       3                             7                                      ------------ ------------------------------ -------------------------     PE1# show bgp l2vpn vpls BGP router identifier 3.3.3.3, local AS number 100 BGP generic scan interval 60 secs BGP table state: Active Table ID: 0x0 BGP main routing table version 898 BGP NSR converge version 3 BGP NSR converged BGP scan interval 60 secs Status codes: s suppressed, d damped, h history, * valid, > best              i - internal, S stale Origin codes: i - IGP, e - EGP, ? - incomplete    Network           Next Hop       Rcvd Label     Local Label Route Distinguisher: 1.1.1.1:32775 *>i5:1/32             1.1.1.1         16120           nolabel         Route Distinguisher: 3.3.3.3:32770 (default for vrf bg1:bd1) *> 3:1/32            0.0.0.0         nolabel         16015           *>i5:1/32             1.1.1.1         16120           nolabel         *>i7:1/32             7.7.7.7         16145           nolabel         Route Distinguisher: 5.5.5.5:32780 *>i7:1/32             7.7.7.7         16145           nolabel           Processed 5 prefixes, 5 paths   4. VPWS Operation with BGP Auto-Discovery and Signaling Similar to VPLS, two primary functions of the VPWS control plane is: auto-discovery, and setup and teardown of the pseudowires that constitute the VPWS to build a full mesh of CEs, often called signaling. Both of these functions are accomplished with a single BGP Update advertisement.   When a VPWS cross-connect is configured with BGP auto-discovery and signaling enabled, BGP needs to distribute NLRI for the xconnect with the PE as the BGP next-hop and appropriate CE-ID. Additionally, the cross-connect is associated with one or more BGP export Route Targets (RTs) that are also distributed (along with NLRI). VPLS SAFI NLRI uses AFI = 25 and SAFI = 25 [5]. The keywords "l2vpn" and "vpls-vpws" will be introduced to represent AF and SAF respectively in the BGP configuration.   The configured attributes are similar to VPLS with the following differences: • CE-IDs instead of VE-IDs • ce-id-range instead of ve-id-range. • ACs are configured with remote CE-IDs. This association is save in L2VPN database and used to establish P2P xconnects.   If a PE receiving VPWS NLRIs is configured with the cross-connect associated with a particular import RT, it can then import all the NLRIs tagged with the same RT.   The NLRI is in the format shown in diagram below:   Length   (2 octets) Route   Distinguisher (8 octets) CE   ID (2 octets) CE   Block Offset (2 octets) CE   Block Size (2 octets) Label   Base (3 octets) Status   Vector (SubTLV)                                       Figure 2: NLRI format for VPWS BGP Auto-discovery and Signaling     4.1 Responsibilities of BGP & L2VPN 4.1.1. BGP Advertising NLRI, RTs, CE-IDs and labelblocks using AFI = 25 SAFI = VPWS. Learns CE-IDs, range, as well as the RD/RT configured under a vpls domain. <MP2MP_ID, ce-id, range, rd> along with RT for ce-id locally configured (rd and RT remain the same for a VPLS) Obtains label block from LSD and maps the local label range (block size, label base, offset) per MP2MP_ID (xconnect group) Imports NLRIs based on RT(s) and passes {MP2MP_ID, local label, remote-label and nexthop, remote CE-ID, l2info} to L2VPN_MGR for each local CE_ID Replay the necessary information for the imported VFIs on the request of L2VPN_MGR.   4.1.2. L2VPN Learns the configured Xconnect from Sysdb. Obtains the configured data such as RT, VPLS-ID, VPN-ID, VE-ID, VE-ID range, CE-ID and CE-ID from Sysdb. Notifies BGP of the configured parameters such RT, etc. Also, L2VPN_MGR shall replay this information upon request from BGP. Receives information such as local label, remote label, etc., pertaining to the PWs from BGP, update xconnect database entry, and notifies L2FIB to setup forwarding plane. Display auto-discovered data via show output.   4.2 Configuring VPWS with BGP AD & Signaling   4.3 Example of NLRIs of VPWS with BGP AD & Signaling     4.4 Verification of VPWS with BGP-AD & Signaling PE1:   PE1# show l2vpn discovery xconnect Service Type: VPWS, Connected List of VPNs (1 VPNs): XC Group: gr1, MP2MP mp1    List of Local Edges (1 Edges):      Local Edge ID: 1, Label Blocks (1 Blocks)        Label base Offset   Size     Time Created              ---------- ------   ----     -------------------        16030     1         10       01/24/2009 21:23:04         Status Vector: 9f ff    List of Remote Edges (2 Edges):      Remote Edge ID: 2, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------       16045     1         10       1.1.1.1         01/24/2009 21:29:35        Status Vector: 7f ff      Remote Edge ID: 3, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------        16060     1         10       1.1.1.1         01/24/2009 21:29:35        Status Vector: 7f ff         PE1# show l2vpn xconnect mp2mp detail Group gr1, MP2MP mp1, state: up VPN ID: 100 VPN MTU: 1500 L2 Encapsulation: VLAN Auto Discovery: BGP, state is Advertised (Service Connected)      Route Distinguisher: (auto) 3.3.3.3:32770    Import Route Targets:        2.2.2.2:100    Export Route Targets:        2.2.2.2:100    Signaling protocol:BGP     CE Range:10 …. Group gr1, XC mp1.1:2, state is up; Interworking none Local CE ID: 1, Remote CE ID: 2, Discovery State: Advertised AC: GigabitEthernet0/1/0/1.1, state is up    Type VLAN; Num Ranges: 1    VLAN ranges: [1, 1]    MTU 1500; XC ID 0x2000013; interworking none PW: neighbor 1.1.1.1, PW ID 65538, state is up ( established )    PW class not set, XC ID 0x2000013    Encapsulation MPLS, Auto-discovered (BGP), protocol BGP      MPLS         Local                         Remote                             ------------ ------------------------------ -----------------------------      Label       16031                         16045                              MTU         1500                           1500                            Control word enabled                       enabled                            PW type     Ethernet VLAN                 Ethernet VLAN                      CE-ID       1                             2                                  ------------ ------------------------------ -----------------------------   ... Group gr1, XC mp1.1:3, state is up; Interworking none Local CE ID: 1, Remote CE ID: 3, Discovery State: Advertised AC: GigabitEthernet0/1/0/1.2, state is up    Type VLAN; Num Ranges: 1    VLAN ranges: [2, 2]    MTU 1500; XC ID 0x2000014; interworking none PW: neighbor 1.1.1.1, PW ID 65539, state is up ( established )    PW class not set, XC ID 0x2000014    Encapsulation MPLS, Auto-discovered (BGP), protocol BGP      MPLS        Local                         Remote                            ------------ ------------------------------ -----------------------------      Label       16032                         16060                              MTU         1500                          1500                              Control word enabled                       enabled                            PW type     Ethernet VLAN                 Ethernet VLAN                      CE-ID       1                              3                                  ------------ ------------------------------ -----------------------------   PE1# show bgp l2vpn vpws BGP router identifier 3.3.3.3, local AS number 100 BGP generic scan interval 60 secs BGP table state: Active Table ID: 0x0 BGP main routing table version 913 BGP NSR converge version 3 BGP NSR converged BGP scan interval 60 secs Status codes: s suppressed, d damped, h history, * valid, > best              i - internal, S stale Origin codes: i - IGP, e - EGP, ? - incomplete    Network           Next Hop       Rcvd Label     Local Label Route Distinguisher: 1.1.1.1:32775 *>i2:1/32             1.1.1.1         16045           nolabel         *>i3:1/32             1.1.1.1         16060           nolabel         Route Distinguisher: 3.3.3.3:32770 (default for vrf gr1:mp1) *> 1:1/32             0.0.0.0         nolabel         16030           *>i2:1/32             1.1.1.1         16045           nolabel         *>i3:1/32             1.1.1.1         16060           nolabel           Processed 5 prefixes, 5 paths     PE2:   PE2# show l2vpn discovery xconnect Service Type: VPWS, Connected List of VPNs (1 VPNs): XC Group: gr1, MP2MP mp1    List of Local Edges (2 Edges):      Local Edge ID: 2, Label Blocks (1 Blocks)        Label base Offset   Size     Time Created              ---------- ------   ----     -------------------        16045     1         10       01/24/2009 21:09:14        Status Vector: 7f ff      Local Edge ID: 3, Label Blocks (1 Blocks)        Label base Offset   Size     Time Created              ---------- ------   ----     -------------------        16060     1         10       01/24/2009 21:09:14        Status Vector: 7f ff    List of Remote Edges (1 Edges):      Remote Edge ID: 1, NLRIs (1 NLRIs)        Label base Offset   Size     Peer ID         Time Created              ---------- ------   ----     --------------- -------------------        16030     1         10       3.3.3.3         01/24/2009 21:09:16      Status Vector: 9f ff   PE2# show l2vpn xconnect mp2mp detail Group gr1, MP2MP mp1, state: up VPN ID: 100 VPN MTU: 1500 L2 Encapsulation: VLAN Auto Discovery: BGP, state is Advertised (Service Connected)      Route Distinguisher: (auto) 1.1.1.1:32775    Import Route Targets:        2.2.2.2:100    Export Route Targets:        2.2.2.2:100    Signaling protocol:BGP      CE Range:10 ... Group gr1, XC mp1.2:1, state is up; Interworking none Local CE ID: 2, Remote CE ID: 1, Discovery State: Advertised AC: GigabitEthernet0/1/0/2.1, state is up    Type VLAN; Num Ranges: 1    VLAN ranges: [1, 1]    MTU 1500; XC ID 0x2000008; interworking none PW: neighbor 3.3.3.3, PW ID 131073, state is up ( established )    PW class not set, XC ID 0x2000008    Encapsulation MPLS, Auto-discovered (BGP), protocol BGP      MPLS         Local                         Remote                            ------------ ------------------------------ -----------------------------      Label       16045                         16031                              MTU         1500                           1500                              Control word enabled                       enabled                            PW type     Ethernet VLAN                  Ethernet VLAN                      CE-ID       2                             1                                  ------------ ------------------------------ ----------------------------- ...   PE2# show bgp l2vpn vpws BGP router identifier 1.1.1.1, local AS number 100 BGP generic scan interval 60 secs BGP table state: Active Table ID: 0x0 BGP main routing table version 819 BGP NSR converge version 7 BGP NSR converged BGP scan interval 60 secs Status codes: s suppressed, d damped, h history, * valid, > best              i - internal, S stale Origin codes: i - IGP, e - EGP, ? - incomplete    Network           Next Hop       Rcvd Label     Local Label Route Distinguisher: 1.1.1.1:32775 (default for vrf gr1:mp1) *>i1:1/32             3.3.3.3         16030           nolabel         *> 2:1/32             0.0.0.0         nolabel         16045           *> 3:1/32             0.0.0.0         nolabel         16060           Route Distinguisher: 3.3.3.3:32770 *>i1:1/32             3.3.3.3        16030           nolabel           Processed 4 prefixes, 4 paths     5. Troubleshooting L2VPN discovery not working   Check the router bgp configs, as sample configs shown below:   RP/0/RSP1/CPU0:PE1#show run router bgp router bgp 100 nsr bgp router-id 2.2.2.2 bgp graceful-restart address-family l2vpn vpls-vpws ! neighbor 3.3.3.3   remote-as 100   update-source Loopback0   address-family l2vpn vpls-vpws   ! ! !   RP/0/RSP1/CPU0:PE1# RP/0/RSP0/CPU0:PE2#show run router bgp router bgp 100 nsr bgp router-id 3.3.3.3 bgp graceful-restart address-family l2vpn vpls-vpws ! neighbor 2.2.2.2   remote-as 100   update-source Loopback0   address-family l2vpn vpls-vpws   ! ! !   RP/0/RSP0/CPU0:PE2#   Check the configs for BGP AD under l2vpn BD : -> Check the VPN-ID matches with the other side PE -> Check the ve-id is different from the other side PE -> Check the rd, generally it will be auto -> Check the signalling protocol configured the same correctly on both the ends   Sample Cfgs : On PE1 : l2vpn bridge group bg1   bridge-domain bg1_bd1    interface PW-Ether2.1    !    interface GigabitEthernet0/1/1/10.1    !    vfi bgp_ad1     vpn-id 1001     autodiscovery bgp      rd auto      route-target 10.1.1.1:1      signaling-protocol bgp       ve-id 1001      !     !   On PE2 : l2vpn bridge group bg1   bridge-domain bg1_bd1    interface GigabitEthernet0/2/1/11.101    !    vfi bgp_ad1     vpn-id 1001     autodiscovery bgp      rd auto      route-target 10.1.1.1:1      signaling-protocol bgp       ve-id 2001      !     !    !   If still the VFI is down --> Start from IGP Neighborship   RP/0/RSP1/CPU0:PE1#show ospf neighbor   * Indicates MADJ interface   Neighbors for OSPF 100   Neighbor ID     Pri   State           Dead Time   Address         Interface 3.3.3.3         1     FULL/DR         00:00:35    30.2.1.2        Bundle-Ether3    Neighbor is up for 23:38:15 3.3.3.3         1     FULL/DR         00:00:38    30.1.1.2        TenGigE0/1/0/1     Neighbor is up for 1d22h 4.4.4.4         1     FULL/DR         00:00:35    60.1.1.2        GigabitEthernet0/1/1/9     Neighbor is up for 1d22h   Total neighbor count: 3 RP/0/RSP1/CPU0:PE1#   --> Then check the BGP l2vpn Neighbors   RP/0/RSP1/CPU0:PE1#show bgp l2vpn vpls  summary BGP router identifier 2.2.2.2, local AS number 100 BGP generic scan interval 60 secs Non-stop routing is enabled BGP table state: Active Table ID: 0x0   RD version: 0 BGP main routing table version 1 BGP NSR Initial initsync version 1 (Reached) BGP scan interval 60 secs   BGP is operating in STANDALONE mode.     Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer Speaker               1          1          1          1           1           1   Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd 3.3.3.3           0   100       4       4        1    0    0 00:01:21          0   RP/0/RSP1/CPU0:PE1#   --> Check the BD Detail for more details when the PW is down.   RP/0/RSP1/CPU0:PE1#show l2vpn bridge-domain bd-name bg1_bd1                                                 Legend: pp = Partially Programmed. Bridge group: bg1, bridge-domain: bg1_bd1, id: 20, state: up, ShgId: 0, MSTi: 0   Aging: 300 s, MAC limit: 4000, Action: none, Notification: syslog   Filter MAC addresses: 0   ACs: 2 (2 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up)   List of ACs:     Gi0/1/1/10.1, state: up, Static MAC addresses: 0     PE2.1, state: up, Static MAC addresses: 0   List of Access PWs:   List of VFIs:     VFI bgp_ad1 (up)       Neighbor 3.3.3.3 pw-id 1001, state: up, Static MAC addresses: 0 RP/0/RSP1/CPU0:PE1# RP/0/RSP1/CPU0:PE1#show l2vpn bridge-domain bd-name bg1_bd1 Legend: pp = Partially Programmed. Bridge group: bg1, bridge-domain: bg1_bd1, id: 20, state: up, ShgId: 0, MSTi: 0   Aging: 300 s, MAC limit: 4000, Action: none, Notification: syslog   Filter MAC addresses: 0   ACs: 2 (2 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up)   List of ACs:     Gi0/1/1/10.1, state: up, Static MAC addresses: 0     PE2.1, state: up, Static MAC addresses: 0   List of Access PWs:   List of VFIs:     VFI bgp_ad1 (up)       Neighbor 3.3.3.3 pw-id 1001, state: up, Static MAC addresses: 0 RP/0/RSP1/CPU0:PE1#show l2vpn bridge-domain autodiscovery bgp detail Legend: pp = Partially Programmed. Bridge group: bg1, bridge-domain: bg1_bd1, id: 20, state: up, ShgId: 0, MSTi: 0   Coupled state: disabled   MAC learning: enabled   MAC withdraw: enabled     MAC withdraw for Access PW: enabled     MAC withdraw sent on: bridge port up     MAC withdraw relaying (access to access): disabled   Flooding:     Broadcast & Multicast: enabled     Unknown unicast: enabled   MAC aging time: 300 s, Type: inactivity   MAC limit: 4000, Action: none, Notification: syslog   MAC limit reached: no   MAC port down flush: enabled   MAC Secure: disabled, Logging: disabled   Split Horizon Group: none   Dynamic ARP Inspection: disabled, Logging: disabled   IP Source Guard: disabled, Logging: disabled   DHCPv4 snooping: disabled   IGMP Snooping: enabled   IGMP Snooping profile: none   MLD Snooping profile: none   Storm Control: disabled   Bridge MTU: 1500   MIB cvplsConfigIndex: 21   Filter MAC addresses:   Create time: 26/06/2013 12:36:30 (00:14:13 ago)   No status change since creation   ACs: 2 (2 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up)   List of VFIs:     VFI bgp_ad1 (up)       VPN-ID: 1001, Auto Discovery: BGP, state is Provisioned (Service Connected)      Route Distinguisher:  (auto) 2.2.2.2:32768       Import Route Targets:         10.1.1.1:1       Export Route Targets:         10.1.1.1:1       Signaling protocol: BGP       Local VE-ID: 1001 ,  Advertised Local VE-ID : 1001       VE-Range: 10       PW: neighbor 3.3.3.3, PW ID 1001, state is up ( established )         PW class not set, XC ID 0xc0000414         Encapsulation MPLS, Auto-discovered (BGP), protocol BGP         Source address 2.2.2.2         PW type VPLS, control word disabled, interworking none         Sequencing not set             MPLS         Local                          Remote                                 ------------ ------------------------------ -------------------------           Label        289975                         16046                                  MTU          1500                           1500                                   Control word disabled                       disabled                               PW type      VPLS                           VPLS                                   VE-ID        1001                           2001                                   ------------ ------------------------------ -------------------------         MIB cpwVcIndex: 3221226516         Create time: 26/06/2013 12:37:26 (00:13:18 ago)         Last time status changed: 26/06/2013 12:37:26 (00:13:18 ago)         MAC withdraw messages: sent 0, received 0         Static MAC addresses:         Statistics:           packets: received 0, sent 0           bytes: received 0, sent 0       DHCPv4 snooping: disabled       IGMP Snooping profile: none       MLD Snooping profile: none       VFI Statistics:         drops: illegal VLAN 0, illegal length 0 RP/0/RSP1/CPU0:PE1#       6. Glossary   PSN (Packet Switched Network): a network using IP or MPLS as the mechanism for packet forwarding PE (Provider Edge): a device connected to customer devices through virtual circuits and providing L2VPN service VE (VPLS Edge): a PE participating in VPLS CE (Customer Edge): a customer device connected to the PE. AC (Attachment Circuit): the connection between the CE and the PE. It is either a port interface or a sub-interface (VLAN, ATM VPI/VCI, Frame Relay) PW (Pseudo Wire): an emulated circuit between two PE’s through a PSN. XC (Cross-Connect): a configured connection between two segments in a PE. A segment can be either an AC or a PW. VFI (Virtual Forwarding Instance): the set of Pseudowires facing the core network NLRI (Network Layer Reachability Information): VPN information exchanged between PEs for auto-discovery and signaling. RD (Route Distinguisher): is an address qualifier used only within a single VPN. It is used to distinguish the distinct VPN routes of separate customers who connect to the provider. BGP extended community: an 8 byte encoded value used to provide extra functionality and avoid routing loops. RT (Route Target): a BGP extended community to tag VPN routes with unique values in order to determine which routes belong to particular VPN. ... View more

White Paper on ABF (ACL Based Forwarding) on ASR9K has been published. Refer this link - https://supportforums.cisco.com/docs/DOC-34081 ... View more

  Introduction   In today’s high performance internetworks, organizations need the freedom to implement packet forwarding and routing according to their own defined polices in a way that goes beyond traditional routing protocol concerns. In XR platform this is achieved by using ACL Based Forwarding (ABF). ABF is a subset of PBR (Policy Based Routing) infrastructure in the XR platform. In summary this features allows traffic matching specific ACL rule to be forwarded to user specified nexthop instead of route selected by a routing protocol.  ASR9K supports ABF v4 and ABF v6 for forwarding IPv4 and IPv6 traffic respectively using ABF rules. Note that ABF is an ingress only feature, i.e., traffic matching ingress ACL can be forwarded using ABF.   The Benefits of ACL Based Forwarding   Source-Based Transit Provider Selection – Internet service providers and other organizations can use ABF to route traffic originating from different sets of users through different internet connections across the policy routers. Cost Savings – Organizations can achieve cost savings by distributing interactive and batch traffic among low-bandwidth, low-cost permanent paths and high-bandwidth, high-cost, switched paths. Load Sharing – In addition to the dynamic load-sharing capabilities offered by destination-based routing that the Cisco IOS-XR software provides network manager can implement policies to distribute traffic among multiple paths based on the traffic characteristics. ABF Configuration   In XR platforms ABF is supported using ACL infrastructure. Enhancements are done to allow specification of ABF nexthop in the ACE rule. There are different variants of specifying ABF nexthop in a ACE. Following illustrates different options for specifying ABF nexthops.   ABF Nexthop: We can specify list of specified IP addresses (up to 3 IP addresses will be supported for ABF along with VRF) in the ACE rule. In this case each nexthop IP address can specify the next hop router in the path towards the destination to which the packets must be forwarded. The first IP address associated with a currently “up” connected interface will be used to route the packets. Following is the example ABFv4 Example :   ipv4 access-list abf-1 10 permit ipv4 any 100.100.100.0/24 nexthop1 VRF RED ipv4 1.1.1.1 nexthop2 VRF BLUE ipv4 2.2.2.2 nexthop3 ipv4 3.3.3.3   ABFv6 Example :   Ipv6 access-list abf-2 10 permit ipv4 any 2010::1/64 nexthop1 VRF A ipv6 2020::1 nexthop2 VRF B ipv6 2030::1 nexthop3 VRF C ipv6 2040::1   In the above example, there are 3 nexthops specified in the ACE rule. nexthop1 has the highest priority and nexthop3 has the lowest priority. if nexthop1 is reachable, i.e, there exist a route for the nexthop1 then it will be selected for forwarding the traffic matching the ACE. Nexthop2 is only selected if nexthop 1 is not reachable, i.e., there is no route present for the nexthop1 and nexthop2 is reachable, i.e., nexthop2 has a route and so on.   ABF Default Route   List of default IP addresses (up to 3 IP addresses will be supported for ABF) – IP address can specify the next hop router in the path towards the destination to which the packets must be forwarded, if there is no explicit route for the destination address of the packet in the routing table. The packet DA lookup should result in default-route. Under this condition ABF will be used to select the nexthop. The first IP address associated with a currently “up” connected interface will be used to route the packets. Following is the example for the same.  Please note “default” keyword in the ACE rule.              ABFv4 Example:        ipv4 access-list abf-1       10 permit ipv4 any 100.100.100.0/24 default nexthop1 VRF RED ipv4 1.1.1.1 nexthop2 VRF BLUE 2.2.2.2 nexthop3 VRF GREEN 3.3.3.3              ABFv6 Example:              ipv6 access-list abf-2      10 permit ipv4 any 2010::1/64 default nexthop1 VRF A ipv6 2020::1   In the above example if matching traffic does not have any route present for the packet DA and it has only default route, in that case ABF nexthop1 is selected for forwarding. If None of the  ABF nexthops are UP then packet is routed using  default route as expected. Note that if Packet DA has route learnt by routing protocol in that case ABF default Nexthop are not selected, instead packet is forwarded using route for packet DA and incoming interface VRF. Lets illustrate this with an example:   In the above example for ABFv4, lets says incoming packet DA is 100.100.100.100   If 100.100.100.100 lookup results in default path, then ABF nexthop is selected in the order of priority when UP, where nexthop1 has highest priority and nexthop3 has lowest priority.   If 100.100.100.100 has a non-default route, then packet will be forwarded using packet DA lookup, ABF path is not taken in this case.     If 100.100.100.100 has default route and none of the ABF nexthops are UP then packet will be forwarded to the default route.   ABF VRF select   This allows to specify only nexthop VRF instead of specific IP address for the nexthop. With this packet DA lookup will be performed in the ABF Nexthop VRF. If route is present then packet is forwarded otherwise dropped. This allows packet arriving on ingrsss VRF to get routed on egress interface which is in different VRF.   ABFv4 VRF select Example: Ingress vrf BLUE , Egress vrf RED        ipv4 access-list abf-1        10 permit ipv4 any 100.100.100.0/24 nexthop1 VRF RED   Int Ten Gig 0/1/0/0 Vrf BLUE Ipv4 addres 30.30.30.1/24 ipv4 access-list abf-1 ingress   In this case packet arriving in VRF BLUE will be forwarded to nexthop in VRF RED. Note that, VRF select feature can be used to send packet arriving on default VRF ton non-default VRF nexthop. Similarly, packet arriving on non-default VRF interface can be forwarded to nexthop in default VRF. Please see following example   Example: Ingress vrf BLUE, and egress vrf DEFAULT.   ipv4 access-list abf-default 10 permit ipv4 any 100.100.100.0/24 nexthop1     Int Ten Gig 0/1/0/0 vrf BLUE Ipv4 addres 30.30.30.1/24 ipv4 access-list abf-default ingress   In the above case, packet arriving on VRF BLUE will be forwarded to nexthop in default route. Packet DA lookup will be performed in the default vrf table for forwarding. ABF with track option   Track option in ABF enables track object to be specified along with nexthop VRF and IP address. Status of track object is used along with reachability of the ABF Nexthop to determine if corresponding nexthop should be selected or not for forwarding.  At present ABFv4 is supported with track option. ABFv6 track option will be supported from 5.1.0 release onwards.   The following simple topology will be used to show this interaction:     Description   From the above diagram the requirement is to have traffic coming into the RED interfaces/VLANs/VRFs on A9K-PE1 to be forwarded with ABF to IGW1, if IGW1 is to fail or not become reachable, the traffic should then get forwarded to IGW2, the same requirement is present on A9K-PE2. There might be multiple Internet gateways in this network, so it’s important to associate each PE with the primary and secondary IGW. In the examples below we will use the loopback addresses of the IGWs, equally one can use any IP address on the GW. It some designs it might be more suitable to use the IP address of the internet facing interface.   Solution       Configuring IPSLA   Pre-requisite for IPSLA is the MGBL pie, without this pie the IPSLA CLI will not be available, this is only required on the PEs, as the IGW is acting as a responder. Step one is to configure IPSLA tracking towards the IGWs from the PEs, this is an example from A9K-PE1.   RP/0/RSP0/CPU0:A9K-PE1#sh run ipsla   Thu Mar 28 21:54:21.934 UTC   ipsla   operation 1   type icmp echo      destination address 10.10.10.10      frequency 5   !   !   operation 2   type icmp echo      destination address 10.20.20.20      frequency 5   !   !   reaction operation 1   react timeout      action logging   !   !   reaction operation 2   react timeout   action logging   !   !   schedule operation 1   start-time now   life forever   !   schedule operation 2   start-time now   life forever   !   !       RP/0/RSP0/CPU0:A9K-PE1#       Verify the IPSLA operation with show ipsla statistics       RP/0/RSP0/CPU0:A9K-PE1#show ipsla statistics   Thu Mar 28 21:54:53.052 UTC   Entry number: 1      Modification time: 21:19:42.937 UTC Thu Mar 28 2013      Start time       : 21:16:55.158 UTC Thu Mar 28 2013      Number of operations attempted: 456      Number of operations skipped : 0      Current seconds left in Life : Forever      Operational state of entry   : Active      Operational frequency(seconds): 5      Connection loss occurred     : FALSE      Timeout occurred             : FALSE      Latest RTT (milliseconds)     : 2      Latest operation start time   : 21:54:50.367 UTC Thu Mar 28 2013      Next operation start time     : 21:54:55.367 UTC Thu Mar 28 2013      Latest operation return code : OK      RTT Values:        RTTAvg : 2         RTTMin: 2         RTTMax : 2                NumOfRTT: 1         RTTSum: 2         RTTSum2: 4       Entry number: 2      Modification time: 21:46:26.132 UTC Thu Mar 28 2013      Start time       : 21:46:26.116 UTC Thu Mar 28 2013      Number of operations attempted: 102      Number of operations skipped : 0      Current seconds left in Life : Forever      Operational state of entry   : Active      Operational frequency(seconds): 5      Connection loss occurred     : FALSE      Timeout occurred             : FALSE      Latest RTT (milliseconds)     : 5      Latest operation start time   : 21:54:51.325 UTC Thu Mar 28 2013      Next operation start time     : 21:54:56.325 UTC Thu Mar 28 2013      Latest operation return code : OK      RTT Values:        RTTAvg : 5         RTTMin: 5         RTTMax : 5                NumOfRTT: 1         RTTSum: 5         RTTSum2: 25   RP/0/RSP0/CPU0:A9K-PE1#   Configuring Tracking       The next step is to configure tracking , so to have IPSLA associated with the tracker. Tracked objects could be Line protocol, Routes, IPSLA reachability, lists etc.. In below configuration we are associating the above IPSLA operation 1 through rtr 1.       RP/0/RSP0/CPU0:A9K-PE1#sh run track   Thu Mar 28 21:33:44.466 UTC   track IGW1   type rtr 1 reachability   !   track IGW2   type rtr 2 reachability   !       RP/0/RSP0/CPU0:A9K-PE1#       Verify the track operation status:       RP/0/RSP0/CPU0:A9K-PE1#show track brief   Thu Mar 28 21:55:53.004 UTC   Track                           Object                                   Parameter       Value                 --------------------------------------------------------------------------------------------------------   IGW2                             IPSLA Operation 2                         reachability     Up     IGW1                             IPSLA Operation 1                         reachability     Up     RP/0/RSP0/CPU0:A9K-PE1#   Configuring ABF on IOS XR       The next step is to associate the Tracker with the ABF statement; it’s done through the standard ABF CLI. The nexthop ip address does not need to be the same as the tracked ip address, as showing in this example.       RP/0/RSP0/CPU0:A9K-PE1#sh run ipv4 access-list   Thu Mar 28 21:50:23.672 UTC   ipv4 access-list ABF_IGW   10 permit ipv4 any any nexthop1 track IGW1 ipv4 10.10.10.1 nexthop2 track IGW2 ipv4 10.20.20.2   !   Bind the ABF statement to an ingress interface.       RP/0/RSP0/CPU0:A9K-PE1#sh run int bundle-e1   Thu Mar 28 21:50:28.751 UTC   interface Bundle-Ether1.100   ipv4 address 192.168.100.1 255.255.255.0   ipv4 access-group ABF_IGW ingress   !       RP/0/RSP0/CPU0:A9K-PE1#   Notes, the above configuration can be applied to an interface in a VRF or a global routing table.     Null0 to drop traffic   The  current functionality of ABF does not allow us to use recursive static routes to blackhole traffic locally. In the example given below  if nexthop1 fails the suitability criteria then packet is forwarded  based on the DA (normal forwarding) rather than using the recursive  route pointing to null.   RP/0/RSP0/CPU0:asr9010-e11-1#sh access-list ABF-TEST ipv4 access-list ABF-TEST 10 permit ipv4 host 109.159.249.66 host 10.10.10.10 nexthop1 ipv4 109.159.249.74 track NH1 nexthop2 ipv4 169.254.0.254 20 permit ipv4 any any RP/0/RSP0/CPU0:asr9010-e11-1#   RP/0/RSP0/CPU0:asr9010-e11-1#sh route 169.254.0.254 Routing entry for 169.254.0.254/32   Known via "static", distance 1, metric 0 (connected)   Installed Oct 10 13:28:57.613 for 01:35:15   Routing Descriptor Blocks     directly connected, via Null0       Route metric is 0   No advertising protos.   RP/0/RSP0/CPU0:asr9010-e11-1#sh track Track NH1         Response Time Reporter 61 reachability         7 changes, last change 14:57:09 BST Thu Oct 10 2013         Latest operation return code: TimeOut         Latest RTT (millisecs) : 0   RP/0/RSP0/CPU0:asr9010-e11-1#sh access-lists ABF-TEST hardware ingress location 0/0/CPU0 ipv4 access-list ABF-TEST 10 permit ipv4 host 109.159.249.66 host 10.10.10.10 (50 hw matches) 20 permit ipv4 any any (23559 hw matches)   A  possible workaround for this is to set a next hop VRF instead of  specific IP address and configure fixed default route (or destination  prefix itself) pointing to null0 within that VRF.   ipv4 access-list ABF-TEST 10 permit ipv4 host 109.159.249.66 host 10.10.10.10 nexthop1 ipv4 109.159.249.74 track NH1 nexthop2 vrf DROP_VRF ! router static vrf DROP_VRF address-family ipv4 unicast   0.0.0.0/0 Null0   RP/0/RSP0/CPU0:asr9010-e11-1#sh track                                                      Track NH1         Response Time Reporter 61 reachability         16 changes, last change 21:30:59 BST Thu Oct 10 2013         Latest operation return code: TimeOut         Latest RTT (millisecs) : 0   RP/0/RSP0/CPU0:asr9010-e11-1#sh access-lists ABF-TEST hardware ingress location 0/0/CPU0   ipv4 access-list ABF-TEST 10 permit ipv4 host 109.159.249.66 host 10.10.10.10 (20 hw matches) (next-hop: addr=0.0.0.0, vrf name=DROP_VRF) 20 permit ipv4 any any (146412 hw matches) RP/0/RSP0/CPU0:asr9010-e11-1#   ABF Support Matrix     Following provide the support matrix for ABF IPv4 and ABF IPv6 on different linecards on the ASR9k platform. Classification is done based on combination  of different interface types on ingress and egress linecards for ABFv4 and ABFv6. Aforementioned, ABF is ingress only feature, i.e., ABF rule is always applied on the ingress interface. TCAM resources are only allocated on line ingress cards where ABF is configured which is used for classification of incoming traffic. For virtual interface such as GRE TCAM resources will be allocated on all LCs where virtual interface is present.  In the following figure ingress LC determines that packet will be routed using ABF rule. Packet will be sent to egress LC with ABF type in the fabric header, which will be used by Egress LC to route packet based on ABF nexthop instead of packet DA. As a result both ingress and egress LCs need to be aware of the ABF feature that can be supported on a particular LC.   Line Cards Support   ABFv4 is supported on   Ethernet, Enhanced Ethernet, and SIP-700 Linecards. ABFv6 is supported only on Enhanced Ethernet and SIP-700. Following support matrix provide the different combinations of ingress and egress LCs that can be supported for ABFv4 and ABFv6 with different interface types. Ethernet: ETH Enhanced Ethernet: E-ETH SIP-700: SI         Ethernet as Ingress LC ABFv4       ABFv6   SIP-700 as Ingress LC ABFv4     ABFv6     Scale Support   We have tested 1000 unique ABF nexthop (IPv4 and IPv6 combined). Note that this does not include duplicate nexthops, we can use same nexthops across multiple ACLs, ACEs and it will be counted once towards the supported scale. System does not enforce this limitation through configuration, user can configure more than 1000 unique nexthops but any such use-case should be tested thoroughly by the customer before the deployment. Load Balancing   If ABF nexthop has multiple ECMP path then multiple streams matching the corresponding ACE rule will get load balanced across the ECMP paths. The raw-hash will be computed on the ingress LC based on the incoming packet and that will be used to select the outgoing path for the ABF nexthop like regular IPv4 or IPv6 forwarding. This will ensure that all the packets from a given stream will always take the same path when packet is routed using ABF. As a result for multiple streams using same ABF nexthop we should expect to see load balancing across multiple ECMP paths.  Figure 2 shows load balancing for 3 incoming streams matching same ACE rule with ABF nexthop.     Troubleshooting   Verify ABF is programmed correctly in the hardware.   RP/0/RSP0/CPU0:ios#show access-lists ipv4 abf-1 hardware ingress location 0/1/cpu0 Mon May 13 06:06:58.609 UTC ipv4 access-list abf-1 10 permit ipv4 any 20.20.20.20 (next-hop: addr=40.40.40.2, vrf name=default) If nexthop is programmed in the hardware then verify that nexthop is resolved, you can confirm that by pinging nexthop. Verify that traffic is matching the ACL for which ABF rule need to be applied.   ABF rule is applied if the traffic matches corresponding ACE entry RP/0/RSP0/CPU0:ios#show access-lists ipv4 abf-perf-1 hardware ingress location$ Mon May 13 06:12:16.203 UTC ipv4 access-list abf-perf-1 10 permit ipv4 any 20.20.20.20 (294096734 hw matches) (next-hop: addr=40.40.40.2, vrf name=default) Note that if incoming traffic is IPv4 or IPv6 then that will be only matched with ACL rules. However, if incoming traffic is labeled traffic then that traffic won’t match ACL rules and hence no ABF forwarding will take place for that scenario. Verify that Ingress and Egress interface and line card are supported as per support matrix given in Section 2   For e.g. ABFv6 is not supported on Ethernet Linecard. Ingress and Egress cannot be Ethernet LC for ABF v6.   Verify that ABF NH is pingable and is in resolved state.   If ABF is programmed correctly in the hardware and traffic is also matching the ACE. Verify that ABF NH is in resolved state. Confirm that you are able to ping ABF nexthop which is programmed in the hardware.   Verify path using traceroute   Traceroute can be used to determine the end-to-end path taken by the packet stream. If ABF match happen for the traceroute probe packet then you should expect to see ABF nexthop in the traceroute output.   Author: Tarun Banka ... View more

ABF (ACL Based Forwarding) on ASR9K ... View more